Earticle

초록

영어

Effective fusion of heterogeneous modalities is a critical factor for improving performance in multimodal learning. However, existing Cross-Modal Transformer (CMT)-based fusion methods are constrained by fixed attention paths, which reduce adaptability to diverse inputs and restrict flexible exploration of optimal fusion strategies. Furthermore, as the number of modalities increases, the computational complexity grows exponentially, leading to scalability bottlenecks. To address these limitations, we propose a Multimodal Routing Optimization (MRO) framework that restructures cross-modal attention paths as a Supernet structure, drawing inspiration from the Once-for-All (OFA) paradigm. Leveraging Neural Architecture Search (NAS), Multimodal Routing Optimization (MRO) dynamically selects optimal routing paths that balance accuracy and computational cost (FLOPs), enabling scalable and efficient multimodal fusion.

목차

Abstract
1. Introduction
2. Related Works
2.1 Neural Architecture Search (NAS)
2.2 Lightweight Supernet-based NAS: Once-for-All (OFA)
2.3 Prior-guided Constrained Search
3. MRO : Multimodal Routing Optimization
3.1 MRO Overview
3.2 Supernet Architecture for Cross-Modal Attention Path Exploration
3.3 Prior-Guided Search Using a Modal Relation Graph
3.4 Prior-Guided Search with Rank Regularization for Consistency Alignment
4. Evaluation
4.1 Learning Dataset
4.2 Model Hyperparameter
4.3 Ablation Study on Rank Regularization Loss Functions
4.4 Performance Results
5. Conclusions
Acknowledgement
References

키워드

저자

• Jeong-Hun Kim [ Undergraduate Researcher, Division of Computer Engineering, Hansung University ]
• Mi-Hwa Song [ Associate Professor, Division of Computer Engeneering, Hansung University ] Corresponding Author

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  International Journal of Internet, Broadcasting and Communication

• 간기

  계간

• pISSN

  2288-4920

• eISSN

  2288-4939

• 수록기간

  2009~2025

• 십진분류

  KDC 326 DDC 380