Abstract.

The paper considers the problem of paintings identification in photos acquired with mobile devices under the conditions of museum exhibition. The proposed approach is based on compact description of an image with a constellation of keypoints and corresponding local descriptors. Two-step comparison scheme is described for finding the best reference image matching the query. Bag-of-features approach is used as a first step, then mutual disposition of points is analyzed. Rejection of the query is performed if no suitable matches are found. Geometrical normalization of the query image is proposed to achieve higher robustness against scale and viewpoint variations. Advantages of the described approach over state-of-the-art solutions are considered. The results of the experiments conducted on the open WikiArt dataset are presented along with processing times for different hardware platforms.

Keywords:

paintings recognition, feature points, image processing.

PP. 5-14. 

DOI: 10.14357/20790279180501

References

1. Pérez-Sanagustín M. et al. Using QR codes to increase user engagement in museum-like spaces // Computers in Human Behavior. – 2016. – Т. 60. – С. 73-85.
2. Antoschuk S.G., Godovichenko N.A. Image local features analysis for “Mobile virtual guide” system. //Pratsi. – 2013. – №. 1 (40). – С. 67-72.
3. Andreatta C., Leonardi F. Appearance based paintings recognition for a mobile museum guide //International Conference on Computer Vision Theory and Applications, VISAPP. – 2006.
4. Leonard Wein. 2014. Visual recognition in museum guide apps: do visitors want it?. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 635-638.
5. Ivanova K. et al. Features for art painting classification based on vector quantization of mpeg-7 descriptors //Data Engineering and Management. – Springer, Berlin, Heidelberg, 2012. – С. 146-153.
6. Keren D. Painter identification using local features and naive bayes //Pattern Recognition, 2002. Proceedings. 16th International Conference on. – IEEE, 2002. – Т. 2. – С. 474-477.
7. Jou J., Agrawal S. Artist identification for renaissance paintings. – 2012.
8. Zujovic J. et al. Classifying paintings by artistic genre: An analysis of features & classifiers // Multimedia Signal Processing, 2009. MMSP’09. IEEE International Workshop on. – IEEE, 2009. – С. 1-5.
9. Arora R.S. Towards automated classification of fine-art painting style: A comparative study : diss. – Rutgers University-Graduate School-New Brunswick, 2012.
10. Lecoutre A., Negrevergne B., Yger F. Recognizing Art Style Automatically in painting with deep learning //Asian Conference on Machine Learning. – 2017. – С. 327-342.
11. Tan W. R. et al. Ceci n’est pas une pipe: A deep convolutional network for fine-art paintings classification //Image Processing (ICIP), 2016 IEEE International Conference on. – IEEE, 2016. – С. 3703-3707.
12. Hong Y., Kim J. Art Painting Identification using Convolutional Neural Network //International Journal of Applied Engineering Research. – 2017. – Т. 12. – №. 4. – С. 532-539.
13. Taverriti G. et al. Real-time Wearable Computer Vision System for Improved Museum Experience //Proceedings of the 2016 ACM on Multimedia Conference. – ACM, 2016. – С. 703-704.
14. Zhang R., Tas Y., Koniusz P. Artwork Identification from Wearable Camera Images for Enhancing Experience of Museum Audiences //arXiv preprint arXiv:1806.09084. – 2018
15. Ruf B., Kokiopoulou E., Detyniecki M. Mobile museum guide based on fast SIFT recognition // International Workshop on Adaptive Multimedia Retrieval. – Springer, Berlin, Heidelberg, 2008. – С. 170-183.
16. The State Tretyakov Gallery, https://www.tretyakovgallery.ru/collection/
17. Davies D.L., Bouldin D.W. A cluster separation measure //IEEE transactions on pattern analysis and machine intelligence. – 1979. – №. 2. – С. 224-227.
18. Yu G., Morel J.M. A fully affine invariant image comparison method //Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. IEEE International Conference on. – IEEE, 2009. – С. 1597-1600.
19. Bay H., Tuytelaars T., Van Gool L. Surf: Speeded up robust features //European conference on computer vision. – Springer, Berlin, Heidelberg, 2006. – С. 404-417.
20. Turcot P., Lowe D.G. Better matching with fewer features: The selection of useful features in large database recognition problems //Computer Vision Workshops (ICCV Workshops), 2009 IEEE 12th International Conference on. – IEEE, 2009. – С. 2109-2116.
21. Lukoyanov A.S., Nikolaev D.P., Konovalenko I.A. Modification of YAPE keypoint detection algorithm for wide local contrast range image //Information technologies and nanotechnology. – 2018. – С. 1193-1204.
22. Fan B. et al. Receptive fields selection for binary feature description //IEEE Transactions on Image Processing. – 2014. – Т. 23. – №. 6. – С. 2583-2595.
23. Muja M., Lowe D.G. Fast matching of binary features //Computer and Robot Vision (CRV), 2012 Ninth Conference on. – IEEE, 2012. – С. 404-410.
24. Skoryukina N., Nikolaev D.P., Sheshkus A., Polevoy D. (2015, February). Real time rectangular document detection on mobile devices. In Seventh International Conference on Machine Vision (ICMV 2014) (Vol. 9445, p. 94452A). International Society for Optics and Photonics.
25. Skoryukina N. et al. Snapscreen: TV-stream frame search with projectively distorted and noisy query //Ninth International Conference on Machine Vision (ICMV 2016). – International Society for Optics and Photonics, 2017. – Т. 10341. – С. 103410Y.
26. Fischler M.A., Bolles R.C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography //Communications of the ACM. – 1981. – Т. 24. – №. 6. – С. 381-395
27. WikiArt http://www.wikiart.org/
28. Karpenko S. et al. UAV control on the basis of 3D landmark bearing-only observations //Sensors. – 2015. – Т. 15. – №. 12. – С. 29802-29820.