Журнал «Труды Института системного анализа Российской академии наук» - A.V. Vokhmintcev The solution of the variational problem point-plane ICP based on the fusion of visual and semantic characteristics of a three-dimensional scene

A new accurate combined method for orthogonal transformations is developed for solving the pointplane variational problem in a closed form. This method is used to reconstruct a three-dimensional model of the environment from a set of images and depth map obtained from sensors which located on mobile platforms. The suggested method was compared with the Horn method for the point-to-point metric. The results of computer simulation showed that the proposed method is better than the state-of-art methods of registration both in accuracy and in terms of computational complexity in uncontrolled conditions.

registration task, point-plane metric, localization, orthogonal transformation, two-dimensional descriptors, iterative closest points algorithm

1. Davison, A. J., Reid, I. D., Molton, N. D., and Stasse, O. 2007. MonoSLAM: Real-Time single camera SLAM. Journal of IEEE Transactions on Pattern Analysis and Machine Intelligence. 29 (6): 1052-1067. doi: 10.1109/TPAMI.2007.1049.

2. Hertzberg, C., Wagner, R., and Birbach, O. 2011. Experiences in building a visual slam system from open source components. 2011. IEEE International Conference on Robotics and Automation Proceedings. Shanghai, China. 2644-2651.

3. Vokhmintsev, A., and Yakovlev, K. 2016. A Realtime Algorithm for Mobile Robot Mapping Based on Rotation-invariant Descriptors and ICP. The 5th International Conference on Analysis of Images, Social Networks and Texts / Eds. D. I. Ignatov, M. Yu. Khachay, V. G. Labunets, et al. – Communications in Computer and Information Science. – Springer. 661: 357–369.

4. Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. 2014. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. 12th International Symposium on Experimental Robotics Proceedings. Delhi, India. 79: 477-491.

5. Endres, F., Hess, J., Engelhard, N., and Sturm, J. 2012. An evaluation of the RGB-D SLAM system. IEEE International Conference on Robotics and Automation Proceedings. Taipei, Taiwan: 1691-1696.

6. Redmon, J., and Farhadi, A. 2017. YOLO9000: Better, Faster, Stronger. IEEE Conference on Computer Vision and Pattern Recognition-CVPR Proceedings. Honolulu, HI, USA.

7. Antonello, M., Wolf, D., Prankl, J., Ghidoni, S., Menegatti, E., and Vincze, M. 2018. Multi-view 3d entangled forest for semantic segmentation and mapping. IEEE International Conference in Robotics and Automation Proceedings. Brisbane, QLD, Australia: 1-8.

8. Besl, P., and McKay, N. 1992. A method for registration of 3D shapes. Journal of IEEE Transaction on Pattern Analysis and Machine Intelligence. 14(2): 239-256. doi: 10.1109/34.121791.

9. Tam, G.K., Cheng, Z.Q., Lai, Y.K, Langbein, F.C., Liu, Y., Marshall, D., Martin, R.R, Sun, X.F., and Rosin, P.L. 2013. Registration of 3D point clouds and meshes: A survey from rigid to nonrigid. Journal of IEEE Transactions on Visualization and Computer Graphics. 19 (7): 1199–1217. doi: 10.1109/TVCG.2012.310

10. Rusinkiewicz, S., and Levoy, M. Efficient variants of the Iterative closest point algorithm. 2001. 3-rd International Conference on 3D Digital Imaging and Modeling Proceedings. Quebec City, Canada: 145–152.

11. Khoshelham, K. 2016. Closed-form solutions for estimating a rigid motion from plane correspondences extracted from point clouds. ISPRS Journal of Photogrammetry and Remote Sensing. 114: 78–91. doi: 10.1016/j.isprsjprs.2016.01.010.

12. Du, S., Zheng, N., Ying, S., and Liu, J. 2010. Affine iterative closest point algorithm for point set registration. Journal of Pattern Recognition Letters. 31(9): 791–799. doi: 10.1016/j.patrec.2010.01.020.

13. Chen, Y., and Medioni, G. 1991. Object modeling by registration of multiple range images. IEEE International Conference on Robotics and Automation Proceedings. Sacramento, USA. 10(3): 2724-2729.

14. Horn, B. 1987. Closed-Form Solution of Absolute Orientation Using Unit Quaternions. Journal of the Optical Society of America A. 4(4): 629–642. doi: 10.1364/josaa.4.000629.

15. Horn, B., Hilden, H., and Negahdaripour, S. 1988. Closed-form Solution of Absolute Orientation Using Orthonormal Matrices. Journal of the Optical Society of America A. 5(7): 1127–1135. doi: 10.1364/JOSAA.5.001127.

16. Low, K. L. 2004. Linear least-squares optimization for point-to-plane ICP surface registration. Department of Computer Science, University of North Carolina at Chapel Hill. Technical Report TTR04-004. Available at: https://www.comp.nus.edu.sg/~lowkl/publications/lowk_point-to-plane_icp_techrep.pdf (accessed February 17, 2004).

17. Segal, A., Haehnel, D., and Thrun, S. 2009. Generalized-ICP. Science and Systems Proceedings of Robotics. Seattle, USA. 2(4): 645-653.

18. Vokhmintcev, A.V., Melnikov, A.V., Mironov, K.V., and Burlutskiy, V.V. 2019. Reconstruction of three-dimensional map based on closed form solution of variational problem of multisensor data registration. Journal of Doklady Mathematics. 99(1): 108–112. doi: 10.1134/S1064562419010290.

19. Vokhmintcev, A., Melnikov, A., Pachganov, S., and Burlutskii, V. 2019. The New Combined Closed-Solution for 3D Reconstruction of Environment Based on Iterative Closest Point Algorithm. 7th Scientific Conference on Information Technologies for Intelligent Decision Making Support Proceedings. Ufa, Russia: 23 –27.

20. Langis, C., Greenspan, M., and Godin, G. 2001. The parallel Iterative closest point algorithm. IEEE 3-rd International Conference 3D Digital Imaging and Modeling Proceedings. Quebec City, Canada: 195-204.

21. Thrun, S., Burgard, W., and Fox, D. 2000. A realtime algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. IEEE International Conference on Robotics and Automation Proceedings. San Francisco, USA: 321–328.

22. Vokhmintcev, A., Sochenkov, I., Kuznetsov, V., and Tikhonkikh, D. 2016. Face recognition based on matching algorithm with recursive calculation of local oriented gradient histogram. Journal of Doklady Mathematics. 93(1): 37-41. doi: 10.1134/S1064562416010178.

23. New York University depth dataset V2. Available at: https://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html (accessed December 28, 2012).