현재 위치
Home
Shorter Articles
Prompt Design for GPT-4 Assessments of EFL Student Reports
-
- 발행기관
- 아시아영어교육학회 바로가기
-
- 간행물
- The Journal of AsiaTEFL SCOPUS KCI 등재 바로가기
-
- 통권
- Vol.23 No.2 (2026.06)바로가기
-
- 페이지
- pp.491-505
-
- 저자
-
- 언어
- 영어(ENG)
-
- URL
- https://www.earticle.net/Article/A486782
※ 기관로그인 시 무료 이용이 가능합니다.
4,800원
원문정보
초록
- 영어
- This study investigates the impact of different prompt design strategies on the performance of GPT-4 in assessing undergraduate reports within an English as a Foreign Language (EFL) context. As Large Language Models (LLMs) increasingly integrate into educational assessment, understanding how prompt engineering affects grading accuracy and alignment with human judgment is crucial. Three prompt design methods—TELeR Taxonomy, Six strategies framework, and Zero-shot prompting— were applied to evaluate 40 student business reports. The assessments were compared against human teacher scores using quantitative statistical analysis, specifically box-and-whisker plots and inter-rater agreement between teacher ratings and AI-generated scores using Cohen’s κ and quadratic weighted κ. Findings indicate that the choice of prompt design significantly influences the alignment between AI and human grading. The Six strategies method demonstrated strong alignment with teacher assessments, particularly in evaluating coherence and cohesion. The Zero-shot prompt demonstrated the strongest alignment with teacher ratings, yielding the most similar score distribution and the highest inter-rater agreement (quadratic weighted κ = 0.65). The results suggest that although this strategy most effectively approximates human scoring in AI-assisted assessment, human oversight remains essential for capturing the subjective and nuanced aspects of student writing.
목차
Abstract
Introduction
Literature Review
Factors Influencing Effectiveness of Prompt Design
Prompt Design Strategies
Factors Influencing Prompt Effectiveness in Automated EFL Grading
Materials and Methods
Research Design
Participants
Rater Profile and Scoring Procedure
Materials
Prompt Design
Instruments for Automated Assessment
Data Analysis
Results
TELeR Prompt Analysis
Six-Strategy Prompt Analysis
Zero-Shot Prompt Analysis
Conclusion
The Authors
References
Appendix
키워드
GPT-4 prompt engineering EFL assessment automated grading higher education저자
간행물 정보
발행기관
-
- 발행기관명
- 아시아영어교육학회 [Asia TEFL]
- 설립연도
- 2004
- 분야
- 사회과학>교육학
- 소개
- The goals of Asia TEFL are to promote scholarship, disseminate information, and facilitate cross-cultural understanding among persons concerned with the teaching and learning of English in Asia. In order to accomplish this, Asia TEFL will pursue the following goals: 1. To link ELT professionals in joint research on issues and concerns regarding English teaching and learning in the Asian context. 2. To publish an academic journal, The Asia TEFL Journal, as an internationally recognized journal in the field of English language teaching. 3. To host conferences and seminars addressing important issues concerning ELT in Asia. 4. To develop proficiency guidelines and assessment methods designed for the needs of the Asian context. 5. To develop programs for Asian learners and teachers of English to build their English language proficiency and cultural understanding and provide them with the skills required to be efficient English teaching professionals.
간행물
-
- 간행물명
- The Journal of AsiaTEFL
- 간기
- 계간
- pISSN
- 1738-3102
- eISSN
- 2466-1511
- 수록기간
- 2004~2026
- 등재여부
- SCOPUS,KCI 등재
- 십진분류
- KDC 740 DDC 420
이 권호 내 다른 논문 / The Journal of AsiaTEFL Vol.23 No.2
피인용수 : 0건 (자료제공 : 네이버학술정보)
함께 이용한 논문 이 논문을 다운로드한 분들이 이용한 다른 논문입니다.
출처 : 네이버학술정보
0개의 논문이 장바구니에 담겼습니다.