Earticle

초록

영어

This study proposes an interdisciplinary framework that integrates Visual Grammar with Artificial Intelligence (AI) to quantify multimodal semantics in sign language. Focusing on the lexical item “DANGER” in Korean Sign Language (KSL) and American Sign Language (ASL), we examine how representational, interactive, and compositional meanings are manifested in 3D kinematic patterns extracted from video. The analysis provides empirical evidence of contrasting representational strategies: KSL shows an internalized Z-axis “Impact” vector toward the chest, while ASL exhibits an externalized X-axis “Friction” vector associated with contact-based motion. In addition, AI-based quantification of non-manual markers reveals a consistent face–manual timing relationship, with facial tension preceding the manual sign by 185 ms (KSL) and 245 ms (ASL), supporting the theoretical view of facial expressions as grammatical anchors in multimodal communication. By moving beyond qualitative description toward measurement-based validation, this work positions AI as a methodological instrument for Digital Sign Humanities and demonstrates how micro-temporal and micro-spatial patterns—often difficult to detect through manual observation—can be systematically linked to linguistic interpretation. Future work will extend this framework to larger datasets and additional sign languages toward a global digital atlas of sign-language kinematics.

목차

Abstract
1. Introduction
1.1. Background: AI as a Precision Ruler for the Humanities
1.2. Research Objectives
1.3. Research Questions (RQ)
2. Literature Review: Visual Grammar and Multimodal AI
2.1 Visual Grammar: a humanities theory for multimodal meaning
2.2 Multimodal AI and quantitative sign-language analysis
2.3 Lexical resources supporting citation-form baselines
3. Data and Methodology
3.1. Data Selection
3.2. AI Methodology
3.3. Feature Set (aligned with RQs and Results)
4. Results and Discussion
4.1. Representational Meaning: Internalized vs. Externalized Hazards
4.2. Interactive Meaning: Facial Precedence (The &quat;Face-First&quat; Siren Effect)
4.3. Compositional Meaning: Spatial Salience
5. Conclusion
5.1. Summary of Findings
5.2. Future Work: Toward a Global Atlas
References

저자

• Lee, Seonhye [ 이선혜 | Pukyong National University ]
• Hahm, Hyunjong [ 함현종 | University of Guam ]
• Jhang, Se-Eun [ 장세은 | Korea Maritime & Ocean University ]

참고문헌