CorrI2P: Deep Image-to-Point Cloud Registration via Dense Correspondence

TCSVT

Siyu Ren      Yming Zeng      Junhui Hou*      Xiaodong Chen     

Overall Framework


Abstract

Motivated by the intuition that the critical step of localizing a 2D image in the corresponding 3D point cloud is establishing 2D-3D correspondence between them, we propose the first feature-based dense correspondence framework for addressing the challenging problem of 2D image-to-3D point cloud registration, dubbed CorrI2P. CorrI2P is mainly composed of three modules, i.e., feature embedding, symmetric overlapping region detection, and pose estimation through the established correspondence. Specifically, given a pair of a 2D image and a 3D point cloud, we first transform them into high-dimensional feature spaces and feed the resulting features into a symmetric overlapping region detector to determine the region where the image and point cloud overlap. Then we use the features of the overlapping regions to establish dense 2D-3D correspondence, on which EPnP within RANSAC is performed to estimate the camera pose, i.e., translation and rotation matrices. Experimental results on KITTI and NuScenes datasets show that our CorrI2P outperforms state-of-the-art image-to-point cloud registration methods significantly

Results

Description of the image
2D-3D Correspondence.

Citation

@article{ren2022corri2p,
title={Corri2p: Deep image-to-point cloud registration via dense correspondence},
author={Ren, Siyu and Zeng, Yiming and Hou, Junhui and Chen, Xiaodong},
journal={IEEE Transactions on Circuits and Systems for Video Technology},
volume={33},
number={3},
pages={1198--1208},
year={2022},
publisher={IEEE}}