Extending reliability of mmwave radar tracking and detection via fusion with camera

In this paper, a new radar-camera fusion system is presented. The fusion system takes into consideration the error bounds of the two different coordinate systems from the heterogeneous sensors, and further a new fusion-extended Kalman filter is utilized to adapt to the heterogeneous sensors. Real-world application considerations such as asynchronous sensors, multi-target tracking and association are also studied and illustrated in this paper. Experimental results demonstrated that the proposed fusion system can realize a range accuracy of 0.29m with an angular accuracy of 0.013rad in real-time. Therefore, the proposed fusion system is effective, reliable and computationally efficient for real-time kinematic fusion applications.