Earticle

초록

영어

As conversational AI systems such as ChatGPT have become more advanced, researchers are exploring ways to use them in education. However, we need effective ways to evaluate these systems before allowing them to help teach students. This study proposes a detailed framework for testing conversational AI across three important criteria as follow. First, specialized benchmarks that measure skills include giving clear explanations, adapting to context during long dialogues, and maintaining a consistent teaching personality. Second, adaptive standards check whether the systems meet the ethical requirements of privacy, fairness, and transparency. These standards are regularly updated to match societal expectations. Lastly, evaluations were conducted from three perspectives: technical accuracy on test datasets, performance during simulations with groups of virtual students, and feedback from real students and teachers using the system. This framework provides a robust methodology for identifying strengths and weaknesses of conversational AI before its deployment in schools. It emphasizes assessments tailored to the critical qualities of dialogic intelligence, user-centric metrics capturing real-world impact, and ethical alignment through participatory design. Responsible innovation by AI assistants requires evidence that they can enhance accessible, engaging, and personalized education without disrupting teaching effectiveness or student agency.

목차

Abstract
1. Introduction
2. Literature Review
3. Insights and Considerations for Evaluating ChatGPT
3.1 Personal Consistency
3.2 Contextual Adaptability
3.3 Simulation of Naturalness
3.4 Updated Ethical Benchmarks
4. Proposed Evaluation Framework
4.1 Conversational Intelligence Benchmarks
4.2 Simulations of Learner Engagement
4.3 Tracking Dialogue Effectiveness
4.4 Ethical Alignment Standards (EAS)
4.5 Learner-Centric Metrics
4.6 Iterative Evaluation Protocol
5. Evaluating Framework Efficacy
5.1 Feasibility Analysis
5.2 Adaptability Analysis
5.3 Limitations and Assumptions
6. Conclusion
References

키워드

저자

• Utkarch Mittal [ Manager, The Gap Inc. USA ] First Author
• Namjae Cho [ Professor, School of Business, Hanyang University ] Corresponding Author
• Giseob Yu [ Adjunct Professor, School of Business, Hanyang University ] Co-Author

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  JITAM [Journal of Information Technology Applications and Management]

• 간기

  격월간

• pISSN

  1598-6284

• eISSN

  2508-1209

• 수록기간

  1999~2026

• 십진분류

  KDC 005 DDC 005