Earticle

초록

영어

‘Appropriate’ evaluation of translation quality requires three criteria: validity, reliability and practicality. The human translation evaluation can be considered as ‘valid’, if it is conducted by a professional evaluator, but it leaves out the other two criteria of evaluating human translation. Automatic evaluation, on the other hand, fulfills the second and the third, but misses the first criteria. It would be therefore, desirable, if both methods could complement each other. This paper attempts to verify this possibility. Using an online tool (interactive BLEU score evaluator), 150 pairs of translations were automatically evaluated and compared to the ratings of a human evaluator (professional translator and evaluator). The results showed only a slight inter-rater agreement (Kappa = .054, .109), but made us some implicit suggestions about the improvement of Kappa score. This can be increased, if calculations are based on 1-gram matches(or even morphemes), instead of 1-4 gram matches, and if the reference translations contain a certain quantity of synonyms. With regard to the validity of BLEU, Brevity Penalty should be substituted by a Length Penalty (Han et al., 2012) accordingly to the characteristics of human translation.

목차

1. 들어가며
2. 인간평가
3. 자동평가
3.1. 자동평가의 종류
3.2. BLEU
4. 실험
4.1. 번역사와 텍스트
4.2. 분석방식
5. 결과 및 토론
5.1. 일치도 세부 결과
5.2. BLEU 평가의 타당성
6. 나가며
참고문헌

키워드

저자

• 정혜연 [ Chung, Hye-Yeon. | 한국외국어대학교 ]

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  통번역학연구 [Interpreting and Translation Studies]

• 간기

  계간

• pISSN

  1975-6321

• eISSN

  2713-8372

• 수록기간

  1997~2026

• 등재여부

  KCI 등재

• 십진분류

  KDC 717 DDC 400