현재 위치
Home
Investigating the Effect of Training on Raters’ Bias toward Test Takers in Oral Proficiency Assessment : A FACETS Analysis
-
- 발행기관
- 아시아영어교육학회 바로가기
-
- 간행물
- The Journal of AsiaTEFL SCOPUS KCI 등재 바로가기
-
- 통권
- Vol.14 No.4 (2017.12)바로가기
-
- 페이지
- pp.687-702
-
- 저자
-
- 언어
- 영어(ENG)
-
- URL
- https://www.earticle.net/Article/A318839
※ 기관로그인 시 무료 이용이 가능합니다.
4,900원
원문정보
초록
- 영어
- Typically, variability among raters in scoring and their bias is mediated through rater training. However, questions still remain about whether training can affect raters’ severity or leniency. Furthermore, few studies have looked at the differences between trained and untrained raters in oral assessment. Oral test scores of 200 test takers rated by 20 raters and were analyzed before and after a training program using the multifaceted Rasch measurement (MFRM). The results demonstrated the constructive impact of training programs in reducing raters’ biases and increasing their consistency measures. This study indicated that inexperienced raters benefited more from a training program than experienced raters and thus achieved higher measures of consistency afterward. It also demonstrated a higher biased interaction for test takers on the extreme ends of the oral ability continuum. The findings demonstrated that it is almost impossible to completely eradicate rater variability even through rater training. Therefore, rater training should be viewed as a procedure to establish within-rater consistency rather than between-rater consistency. Since this study showed that inexperienced raters can rate even more reliably than experienced ones after training, there is no evidence whereby decision makers can exclude inexperienced raters solely because of their lack of adequate experience. Consequently, decision makers need to use their budgets for establishing rater training programs for inexperienced raters instead.
목차
Abstract
Introduction
Literature Review
Raters’ Background in Oral Performance Assessment
Rater Behavior in Oral Performance Assessment
Methodology
Participants
Instruments
Procedure
Data Analysis
Results
Discussion
Conclusion and Implications
The Authors
References
키워드
bias feedback interrater reliability intra-rater reliability multifaceted Rasch measurement (MFRM) rater severity/leniency rater training저자
간행물 정보
발행기관
-
- 발행기관명
- 아시아영어교육학회 [Asia TEFL]
- 설립연도
- 2004
- 분야
- 사회과학>교육학
- 소개
- The goals of Asia TEFL are to promote scholarship, disseminate information, and facilitate cross-cultural understanding among persons concerned with the teaching and learning of English in Asia. In order to accomplish this, Asia TEFL will pursue the following goals: 1. To link ELT professionals in joint research on issues and concerns regarding English teaching and learning in the Asian context. 2. To publish an academic journal, The Asia TEFL Journal, as an internationally recognized journal in the field of English language teaching. 3. To host conferences and seminars addressing important issues concerning ELT in Asia. 4. To develop proficiency guidelines and assessment methods designed for the needs of the Asian context. 5. To develop programs for Asian learners and teachers of English to build their English language proficiency and cultural understanding and provide them with the skills required to be efficient English teaching professionals.
간행물
-
- 간행물명
- The Journal of AsiaTEFL
- 간기
- 계간
- pISSN
- 1738-3102
- eISSN
- 2466-1511
- 수록기간
- 2004~2026
- 등재여부
- SCOPUS,KCI 등재
- 십진분류
- KDC 740 DDC 420
이 권호 내 다른 논문 / The Journal of AsiaTEFL Vol.14 No.4
피인용수 : 0건 (자료제공 : 네이버학술정보)
함께 이용한 논문 이 논문을 다운로드한 분들이 이용한 다른 논문입니다.
출처 : 네이버학술정보
0개의 논문이 장바구니에 담겼습니다.