Multisource heterogeneous data fusion: multimodal modeling approach for intelligent safety driving assistance system

Xiaochun Tong; Mary Jane C. Samonte

doi:10.1117/12.3038244

7 August 2024 Multisource heterogeneous data fusion: multimodal modeling approach for intelligent safety driving assistance system

Xiaochun Tong, Mary Jane C. Samonte

Author Affiliations +

Proceedings Volume 13229, Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024); 132292A (2024) https://doi.org/10.1117/12.3038244
Event: Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024), 2024, Nanchang, China

Abstract

Unsafe driver behavior is a significant cause of traffic accidents, and accurately identifying and warning of dangerous driving behaviors is crucial for improving road traffic safety. Single-modal data alone is insufficient to comprehensively describe complex driving behaviors. Multimodal data fusion technologies, by integrating heterogeneous information sources such as vision, speech, and motion, provide robust support for driving behavior analysis. This paper comprehensively reviews the current status, key technologies, application cases, challenges, and future development trends of safety driving behavior warning models based on multimodal fusion. It introduces typical architectures and algorithms of multimodal deep learning models, such as early/late/mixed fusion strategies, attention mechanisms, and temporal modeling, and analyzes application scenarios such as intelligent driving assistance systems and driving simulation training. Finally, it highlights challenges such as model performance, data quality and computational requirements, and discusses future directions. The development of multimodal safety driving behavior analysis technology will make important contributions to the construction of intelligent transportation systems.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Xiaochun Tong and Mary Jane C. Samonte "Multisource heterogeneous data fusion: multimodal modeling approach for intelligent safety driving assistance system", Proc. SPIE 13229, Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024), 132292A (7 August 2024); https://doi.org/10.1117/12.3038244

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Data modeling

Modeling

Data fusion

Safety

Systems modeling

Performance modeling

Intelligence systems

Show All Keywords

Keywords/Phrases

Search In:

Publication Years