Earticle

초록

영어

This study presents an innovative approach to bidirectional sign language translation using an AI-based 3D interactive Unity character. We have developed a system that receives Korean text input, translates it into Korean Sign Language(KSL), and then uses Unity3D to enable a 3D character to perform sign language gestures in real-time. The system aims to deliver natural character movements while accurately reflecting the vocabulary and grammatical structure of KSL. Although the representation of non-manual signals(NMS) remains a significant challenge, the combination of Unity3D characters with AI-based translation plays a crucial role in addressing the issue of information alienation among KSL users.

목차

Abstract
I. INTRODUCTION
II. RELATED WORKS
III. METHODS
A. DATASET
B. Model for Bidirectional Sign Language Translation
C. Video Swin Transformer
D. Bidirectional Sign Language Translation Model
E. Unity3D Character Development
F. Experiment setup
IV. EXPERIMENT RESULT
A. Evaluation Criteria
B. Result
V. CONCLUSIONS
ACKNOWLEDGMENT
REFERENCES

저자

• Min Sung Jung [ dept. of Computer Science & Engineering Kyungnam university ]
• Byung-Joo Shin [ dept. of Computer Science & Engineering Kyungnam university ] Corresponding Author
• Hong Jo An [ dept. of Computer Science & Engineering Kyungnam university ]
• Beom Mo Kim [ dept. of Computer Science & Engineering Kyungnam university ]
• Dong En Kim [ dept. of Computer Science & Engineering Kyungnam university ]
• Yong Min Cho [ dept. of Computer Science & Engineering Kyungnam university ]
• Gyu Tae Park [ dept. of Computter Science & Engineering Kyungnam university ]

참고문헌