Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review

Haozhou Fei

doi:10.47297/taposatWSP2633-456915.20250608

PDF
导出
分享
收藏
专辑

Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review

6卷第8期

浏览量: 0 Downloads: 0

Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review
Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review
Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu, 210023, China
2025年6卷第8期页码：81-85
纸质出版：2025-08-30
DOI：10.47297/taposatWSP2633-456915.20250608
Accepted：

Scan QR Code

(2025). Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review. 科技理论与实践（英文版）, 6(8), 81-85.

Fei Haozhou.Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review[J].Theory and Practice of Science and Technology,2025,06(08):81-85.
(2025). Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review. 科技理论与实践（英文版）, 6(8), 81-85. DOI： 10.47297/taposatWSP2633-456915.20250608.

Fei Haozhou.Multi-Modal Fusion for 3D Object Detection in Autonomous Driving: A Review[J].Theory and Practice of Science and Technology,2025,06(08):81-85. DOI： 10.47297/taposatWSP2633-456915.20250608.

摘要

Abstract

Autonomous driving technologies have seen significant advancements due to improvements in artificial intelligence and sensor systems. A crucial element for safe and effective autonomous driving is the environmental perception system

with 3D object detection serving as a core task for vehicle awareness and decision-making. However

single-modality sensors

such as cameras

LiDAR

and radar

each have inherent limitations. Cameras provide rich semantic data but struggle under adverse conditions

LiDAR offers precise geometric data but is costly

and radar

while robust in all weather

lacks spatial resolution. Multi-modal sensor fusion addresses these challenges by combining complementary data from different sensor types

leading to a more comprehensive environmental understanding. This paper surveys the latest advancements in 3D object detection using multi-modal fusion for autonomous driving

focusing on feature-level fusion techniques

particularly the employment of Bird's-Eye-View (BEV) representation and Transformer architectures. The paper also explores the development of datasets

the evolution of fusion strategies

and the critical challenges faced

such as data alignment

performance efficiency

and robustness in harsh conditions. Finally

it outlines the future research directions aimed at optimizing the fusion process

including lightweight architectures and the integration of foundation models for improved performance and generalization.

关键词

Keywords

references

H. Li et al. , "The Effect of Rainfall and Illumination on Automotive Sensors Detection Performance," Sustainability , vol. 15 , no. 9.

A. Geiger, P. Lenz, and R. Urtasun, "Are we ready for autonomous driving? The KITTI vision benchmark suite," in 2012 IEEE Conference on Computer Vision and Pattern Recognition , 16–21 June 2012 2012, pp. 3354 – 3361 .

H. Caesar et al. , "nuscenes: A multimodal dataset for autonomous driving," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 11621 – 11631 .

P. Sun et al. , "Scalability in perception for autonomous driving: Waymo open dataset," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 2446 – 2454 .

M.-F. Chang et al. , "Argoverse: 3d tracking and forecasting w ith rich maps," 2019, pp. 8748 – 8757 .

X. Huang, P. Wang, X. Cheng, D. Zhou, Q. Geng, and R. Yang, "The ApolloScape Open Dataset for Autonomous Driving and Its Application," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42 , no. 10 , pp. 2702 – 2719 , 2020 .

P. Xiao et al. , "PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving," in 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) , 19–22 Sept. 2021 2021, pp. 3095 – 3101 .

A. Patil, S. Malla, H. Gang, and Y. T. Chen, "The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenes," in 2019 International Conference on Robotics and Automation (ICRA) , 20–24 May 2019 2019, pp. 9552 – 9557 .

J. Geyer et al. , "A2d2: Audi autonomous driving dataset," arXiv preprint arXiv: 2004.06320 , 2020.

M. Sheeny, E. D. Pellegrin, S. Mukherjee, A. Ahrabian, S. Wang, and A. Wallace, "RADIATE: A Radar Dataset for Automotive Perception in Bad Weather," in 2021 IEEE International Conference on Robotics and Automation (ICRA) , 30 May–5 June 2021 2021, pp. 1 – 7 .

C. R. Qi, W. Liu, C. Wu, H. Su, and L. J. Guibas, "Frustum pointnets for 3d object detection from rgb-d data," 2018, pp. 918 – 927 .

S. Vora, A. H. Lang, B. Helou, and O. Beijbom, "Pointpainting: Sequential fusion for 3d object detection," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 4604 – 4612 .

C. Wang, C. Ma, M. Zhu, and X. Yang, "Pointaugmenting: Cross-modal augmentation for 3d object detection," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 11794 – 11803 .

M. Liang, B. Yang, S. Wang, and R. Urtasun, "Deep continuous fusion for multi-sensor 3d object detection," in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 641 – 656 .

Z. Liu et al. , "Bevfusion: Multi-task multi-sensor fusion with unified bird's-eye view representation," arXiv preprint arXiv: 2205.13542 , 2022.

H. Cai, Z. Zhang, Z. Zhou, Z. Li, W. Ding, and J. Zhao, "BEVFusion4D: Learning LiDAR-Camera Fusion Under Bird's-Eye-View via Cross-Modality Guidan ce and Temporal Aggregation," arXiv e-prints, p. arXiv: 2303.17099 , 2023.

A. W. Harley, Z. Fang, J. Li, R. Ambrus, and K. Fragkiadaki, "Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?," 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 2759 – 2765 , 2022 .

M. Chen et al. , "Multi-Modal BEV Enhancement Fusion for 3D Object Detection in Autonomous Driving," IEEE Transactions on Intelligent Transportation Systems, pp. 1 – 13 , 2025 .

C. Hu et al. , "FusionFormer: A Multi-sensory Fusion in Bird's-Eye-View and Temporal Consistent Transformer for 3D Object Detection," 2023.

X. Bai et al. , "Transfusion: Robust lidar-camera fusion for 3d object detection with transformers," in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 1090 – 1099 .

Z. Wang, Z. Huang, Y. Gao, N. Wang, and S. Liu, "Mv2dfusion: Leveraging modality-specific object semantics for multi-modal 3d detection," arXiv preprint arXiv: 2408.05945 , 2024.

K. Liu, Y. Deng, J. Tong, and W. Li, "UniVoxel: A Novel Framework for 3D Object Detection in Autonomous Vehicles with Multi-modal Voxel Representation," IEEE Sensors Journal, pp. 1 – 1 , 2025 .

J. Yan et al. , "Cross modal transformer: Towards fast and robust 3d object detection," in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 18268 – 18278 .

S. Pang, D. Morris, and H. Radha, "CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection," in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 24 Oct.–24 Jan. 2021 2020, pp. 10386 – 10393 .

B. Shahian Jahromi, T. Tulabandhula, and S. Cetin, "Real-Time Hybrid Multi-Sensor Fusion Framework for Perception in Autonomous Vehicles," Key Laboratory of Universal Wireless Communications, Ministry of Education, School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China National Engineering Lab for Mobile Networ , vol. 19 , no. 20.

M. Ren, P. He, and J. Zhou, "Decision fusion of two sensors object classification based on the evidential reasoning rule," Expert Systems with Applications, vol. 210 , p. 118620 , 2022/12/30/ 2022.

X. Zhang, G. Liu, M. Li, Q. Ren, H. Tang, and D. P. Bavirisetti, "FusionNGFPE: An image fusion approach driven by non-global fuzzy pre-enhancement framework," Digital Signal Processing, vol. 156 , p. 104801 , 2025/01/01/ 2025.

A. B. Amjoud and M. Amrouch, "Object Detection Using Deep Learning, CNNs and Vision Transformers: A Review," IEEE Access, vol. 11 , pp. 35479 – 35516 , 2023 .

K. Bajbaa, M. Usman, S. Anwar, I. Radwan, and A. Bais, "Bird's-Eye View to Street-View: A Survey," City University of Hong Kong, Hong Kong;Delft University of Technology, Netherlands;Nanyang Technological University, Singapore, vol. abs/2405.08961, 2024.