Earticle

초록

영어

This paper proposed a Feature-level fusion technique that combines facial expression and audio modalities for multimodal emotion recognition. The learning model utilizes a hybrid approach combining CNN and LSTM to learn the spatiotemporal characteristics of video and audio modalities effectively. Compared to a unimodal approach, speech emotion recognition achieved 74% accuracy, and facial emotion recognition achieved 83% accuracy, while the proposed multimodal approach achieved 93% accuracy, demonstrated that multimodal emotion recognition is more accurate than unimodal emotion recognition. Furthermore, in tests using the RAVDESS dataset, the proposed model achieved higher emotion recognition rates compared to related studies. This study demonstrated the possibility of multimodal emotion recognition and designed a model capable of recognizing emotions in various environments and situations. Through this, we aim to contribute to the advancement of emotion recognition technology.

목차

Abstract
I. INTRODUCTION
II. RELATED WORK
A. Facial Emotion Recognition
B. Speech Emotion Recognition
C. Multimodal(Speech + Facial Emotion Recognition, Facial+ EEG Emotion Recognition)
III. PROPOSED METHOD
A. Preprocessing Process for Video and Audio Data
B. Structure of Proposed Model
IV. EXPERIMENTS
A. Used DATASET
B. EXPERIMENTS RESULTS
V. CONCLUSION
ACKNOWLEDGMENT
REFERENCES

키워드

저자

• Deoghwa KIM [ Department of Electrical and Computer Engineering Inha University Incheon, South Korea ]
• Han Wang [ Department of Electrical and Computer Engineering Inha University Incheon, South Korea ]
• Deok-Hwan Kim [ Department of Electrical and Computer Engineering Inha University Incheon, South Korea ] Corresponding Author

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  한국차세대컴퓨팅학회 학술대회

• 간기

  반년간

• 수록기간

  2021~2025

• 십진분류

  KDC 566 DDC 004