Earticle

This study aims to investigate the text difficulty of the reading materials of Korean middle school English textbooks with Coh-Metrix, a software developed by the Institute for Intelligent Systems at the University of Memphis to analyze the linguistic and psycholinguistic features of English text and textbooks with a wide range of indices on cohesion and language. In this study, the textbook corpus consisted of the text files extracted from 13 English textbooks. These files were used for analyzing the text difficulty among grades with Coh-Metrix. The Coh-Metrix indices selected for this study contained basic counts, word frequency, word features, lexical diversity, pronouns, connectives, readability, syntax complexity, syntax similarity, reference cohesion, semantic cohesion, and situation model measures. The results showed that there were significant differences among grades for basic counts, word features, first pronouns, causal and temporal connectives, readability, reference and semantic cohesion, the number of words before main verbs, syntactic similarity, and situation model measures. The differences among grades, however, were not significant for word frequency, lexical diversity, second and third person pronouns, additive connectives, and NP density measures. The findings have educational implications for textbook design and language learning for English learners.

This study was motivated by the notorious difficulty of the English section of the 2019 College Scholastic Aptitude Test (CSAT). While there have been a large number of studies examining what factors determine the difficulty of ESL reading questions, text coherence has been paid little attention due to the highly linguistic orientation of the language testing field. It was generally agreed in many studies that text itself does not bear a direct relationship with its coherence. Instead, the coherence reflects the reader’s mental representation of the text elicited from it. During the process of drawing the mental representation, the reader both depends on the information explicitly presented in the text and make necessary inferences to fill in the missing parts that he or she failed to discover from the text. In order to explore the influence of text coherence on the difficulty of the reading questions, this study analyzed the most difficult five questions of the English section of the 2019 CSAT using the text analysis paradigm of Mann and Thompson’s Rhetorical Structure Theory and attempted to identify whether there are any elements to interfere with the drawing of the mental representation and whether the interrupting elements could lead to the low correct response rate. The analyses demonstrated that the texts for all five questions include some structural elements that might lead many test-takers to trouble in drawing the mental representation and, consequently, making correct answers.