Earticle

This study quantitatively analyzes the correlation between the Japanese response marker naruhodo and accompanying nonverbal behavior, specifically nodding, using a multimedia corpus. While previous research has mainly focused on other response markers such as soudesune, the interaction between naruhodo and nonverbal cues remains insufficiently explored. The main findings are as follows. First, naruhodo was primarily used by interviewers and moderators, indicating that the speaker’s role in the broadcast had a greater impact on its usage than factors such as age or gender. Second, in two-thirds of the naruhodo occurrences, it co-occurred with nodding behavior, especially when the preceding utterance reflected the speaker’s subjective opinion. Third, nodding rates and response types varied depending on the follow-up utterance, with two main patterns observed—continued attentive listening and confirmation requests—and nodding before a confirmation request tended to amplify explicit agreement. These findings suggest that linguistic and nonverbal elements function complementarily and highlight the need for theoretical and methodological expansion in multimodal interaction research.

TV programs are useful because they provide large amounts of diverse verbal and non-verbal language data (i.e., gaze, gesture, facial expression, and tone) from various speakers; researchers can collect data from these programs to conduct a quantitative study of Japanese language behavior. However, TV programs exhibit problems that make it difficult to keep observing speakers from specific angles. No research has identified the extent to which speakers can be observed from TV broadcasting images or whether this observation differs depending on specific factors. In an attempt to fill this gap in knowledge, this paper examined a TV interview program titled “Holiday Interview.” Focusing on the roles of speakers and types of speaking, we found out that an interviewer in the program appeared on screen about half as often as the interviewee. Particularly, as the interviewer primarily said hah- and ah- sodesune—thus backchanneling— the camera rarely focused on him. My future research will analyze speakers’ language behaviors and the image production process of the behaviors considering factors such as the number of speakers participating in interviews and interview styles, which are not examined in this paper. (Jeju National University)

Son, Young-Suk. 2013. Factors Influencing the Distribution of Japanese Plural Suffixes -tachi and -ra: A Study Employing the “Talk Show Multimedia Corpus”. Language Information. Volume 17. 47-72. This study examined which factors were significantly associated with Japanese speakers’ choice of the plural suffixes -tachi (たち) and -ra (ら), which resemble each other closely in meaning and are commonly seen in first-person plural pronouns such as watashi-tachi (わたし たち), watashi-ra (わたしら), boku-tachi (ぼくたち) and boku-ra (ぼくら), all meaning ‘we’. To do so, the study employed an intra- and extra-linguistic perspective based on the Talk Show Multimedia Corpus. The results revealed the following facts: The distribution of -tachi and -ra differed according to several intra-linguistic factors: the type of personal pronoun used with the suffix, the referents, and the content of the utterance, among others. Specifically, watashi and ore (おれ) ‘I’ are mainly used with -tachi, and boku ‘I’ with -ra. Further, -tachi is chosen when the referent of a personal pronouns includes all listeners, while -ra is frequently chosen when it does not. Finally, only -ra is used in honorific expressions, which in Japanese convey a meaning of downgrading of the speaker’s own status and that of people from their group or those with whom they are associated as compared to listeners. In the future, if the Multimedia Corpus can be improved to incorporate material other than that from television talk shows, it should be possible to comprehend the factors on the basis of which speakers select -tachi and -ra in colloquial Japanese, in more detail and from various perspectives, including speaker attributes that cannot be fully considered here because of the insufficient number of tokens.

Multimedia corpus enables researchers to observe both spoken sentence materials and the nonverbal expressions of speakers. To undertake a contrastive study of the language behaviors of Japanese and Korean speakers, I have established the multimedia corpora of Japanese and Korean talk shows. Improving upon the previous settings of the corpora (e.g.,these featured inconsistency in the number of transcribed spoken sentences materials in one unit cell, or the length of time of the video record capturing each speaker varied), I analyzed the ways the demonstrative pronouns of deixis, kore and yi-geot, were used in real dialogues. The comparative analysis of the two corpora revealed that kore is used to demonstrate the very things that are indicated, whereas yi-geot is used to demonstrate the gestures and the action movements of the speakers who use it. As pointed out, the scenes that included the speech acts of kore and yi-geot are not identical in the corpus of Japan or of Korea. In other words, the scenes that feature the two words in each corpus draw upon different situations and themes in the talk shows. Therefore, one should consider such differences in the corpora that were studied for this paper when researching the language behaviors of Japanese and Korean speakers. In order to reach a fuller understanding of the differences in the scenes of these languages as they were spoken, I further investigated the usages of words other than those studied in this paper.

인터넷 상에서 단어 등을 검색하면 해당 단어 발화시의 영상·음성을 참조할수 있도록 설계된 온라인 멀티미디어 코퍼스를 활용하여 일본어 진술부사에 관한 수업을 진행하였다. 그리고 수업에 참여한 학습자들로부터 피드백 받은 요구분석 결과를 바탕으로 향후 코퍼스 개선방안에 대해 고찰하였다. 코퍼스 자료수집, 언어정보 기록, 비언어정보 기록, 검색 및 검색결과 제시의 4가지 측면에서 고찰한 결과, 외국어 교육을 목적으로 한 본격적인 온라인 멀티미디어 코퍼스 구축에는 음성인식 및 얼굴인식을 비롯하여 다방면의 기술 도입이 불가피하다는 결론을 도출하였으며, 각각의 기술들을 기반으로 차세대 코퍼스 구축 방향성에 대해서도 제시하였다.

A class on Japanese modal adverbs was conducted using an online multimedia corpus which is designed to search for examination of the images and voices of an utterance. Based on the results of demand analysis of the feedback received from learners who participated in the class, improvement plans for the future corpus were thoroughly explored. On considering the results of four aspects such as the collection of corpus data, search and search results suggestions, language data records, and non-verbal data records, it was concluded that the introduction of various technologies including voice recognition and face recognition was inevitable for the comprehensive construction of the online multimedia corpus for the purpose of foreign language education. In addition, the direction for the overall corpus construction based on each technology was presented.

言語表現と非言語表現（視線、表情、身振りなど）とを用いて情報を伝える行為を言語行動と呼び、日本語話者と韓国語話者の言語行動を計量的観点から対照分析するために、既存の「日本語対談番組マルチメディア·コーパス」に加え、新たに「韓国語対談番組マルチメディア·コーパス」を作成した。それにより、韓国語をも対象とすることが可能となり、ある言語表現が、どの番組で、誰が誰に向かって話すときに、どのような非言語表現とともに発せられたかなどを、現実の大量のデータにもとづいて調べられるようになった。そして、実際に二種類のコーパスを使って、「一人称複数代名詞（私たち·僕らなど）」とそれに該当する韓国語の‘우리(uri)’の発話時における話し手の視線表現について調べた結果、代名詞の意味（指示対象）と視線表現は、言語の違いにかかわらず、強い相関関係にあるということが明らかになった。これは、日本語のみを対象としてきた孫(2014a)の結果を裏付けるものであると同時に、マルチメディア·コーパスを用いた対照言語行動研究の有用性を示すものでもある。ただし、今後、より効率的に対照分析を行うためには、二種類のコーパスが双方向に検索できるよう、並列コーパスを作ることが欠かせない。

Corpus, a collection of data on the language used in day-to-day life that can be processed through computers, is highly useful in linguistics and education. It can examine how a language is used during natural communication. In the case of spoken Japanese, however, online public corpora that can be utilized for research and education are quite rare. To address such an issue, based on “talk show corpus” data, the current study aims to build an “online multimedia corpus” where spoken Japanese can be found. The study used data from video files of 106 episodes of talk show programs, written text files of voice recordings from each episode, which is 40 hours recording in total, and Japanese-Korean subtitle files produced by Son, Young Suk (2017, 2018). As a result of building the online multimedia corpus, one can easily access the Internet to get information on spoken Japanese and share one’s findings without time or location constraints. A person does not even need advanced computer skills to access this. The contexts of front and rear are arranged in KWIC(Keyword in Context) format, so by using “language entry=keyword” to search, the relevant videos are automatically and continuously played. The implementation of this function, “video KWIC,” is significant, as it allows users to not only study spoken Japanese but also observe how it was spoken in videos.

The MMC (Multimedia Corpus) of talk shows that I have built drawing upon talk shows of Japanese TV programs can be a useful resource for learning Japanese, in that it provides users with an opportunity to observe the use of Japanese language and communication behaviors that are practiced in everyday life. However, because the MMC of talk shows has been established for research purposes specifically, Japanese learners at a beginner level have found it difficult to use. Reflecting upon this observance, I aimed to increase the possibility of utilizing the MMC as an educational resource by providing Japanese motion pictures with Korean subtitles. To accomplish this, I first translated entire Japanese sentences that are contained in the MMC into Korean, and then inserted Korean subtitles in the MMC. Further, I carried out a classroom activity for the Japanese expression, Shinkenda, using the Korean inserted MMC. For the classroom activity, both Japanese subtitles prepared for a previous paper (Son, forthcoming) and Korean subtitles for this paper were used. I did not have to translate the individual Japanese sentences into Korean while teaching, so I felt less pressured in preparing and carrying out the classroom instruction. Also, I observed that student learners understood the conversations of talk shows much better with Korean subtitles. However, it was pointed out by student learners (from an opinion survey) that the time between the subtitles appearing and disappearing should be improved. Further, they also stated that the size, amount, and location of the subtitles needed improvement. I intend to find methods to increase readability of the subtitles in further research.

I developed an MMC (Multimedia Corpus) of talk shows, drawing upon Japanese TV programs. The MMC can be useful for learning Japanese, as it provides users with an opportunity to observe the use of Japanese language and its communication behaviors practiced in everyday life. However, since the MMC has been established for research purpose in general, lower??level Japanese learners face difficulties in using it. Therefore, this paper aimed to increase the application capacity of the MMC by presenting motion pictures with subtitles that are created by adding yomigana tags to transcripts of spoken dialogues from the MMC. Further, using the MMC talk shows, I conducted a classroom activity with Japanese learners of kamoshirenai, which is a Japanese grammatical expression. Due to the yomigana tags, I did not have to explain to the students how to read Chinese characters and numeric numbers one by one; therefore, I was less stressed than before in preparing a class and undertaking classroom teaching. In addition, the overall satisfaction level among learners was high. I believe that my personal experiences can be extended to other teachers. However, the responses regarding students’ experiences of the subtitled motion pictures of the MMC for their learning, collected from the learners in my class activities, should be interpreted with care, as they are merely based on an opinion survey. A further investigation should be undertaken on which learning effects can be achieved using subtitled motion pictures of the MMC.

言語と非言語表現の運用面に注目し話者の配慮について考察する研究分野を「敬語行動研究」と呼び、敬語行動研究に「マルチメディア・コーパス」が有用な調査手段として活用できるということを、ことばと聞き手への指差し行為との関係についての事例調査を例に論じた。「マルチメディア・コーパス」とは、言語のみならず言語使用場面の映像をも参照できるコーパスのことであり、本稿では孫(2010)が作成した『対談番組のマルチメディア・コーパス』を用いている。本コーパスの大量のデータにもとづき、発話時に話し手が聞き手を指差すところを調べた結果、聞き手への指差し行為は主として〈確認要求〉〈質問〉を表す疑問表現で観察されるということ、観察される指差しは発話状況・対偶表現の仕方・発話内容によってその種類が顕著に異なるということ、とくに両手による指差し行為は話し手が尊敬語などを用いて聞き手の状態・心的状況・見解について尋ねる際に伴うということなどが、特定の傾向として抽出された。そしてこれらのことから、両手による聞き手への指差し行為には聞き手が誰であるかを明示させる機能のほかにも、聞き手の状態・心的状況などに対する話者の配慮を表す機能をも含まれていると考えられる。今後その機能をより具体的に解明するためにはさらなる調査・分析が必要であるが、本稿で導いた傾向は、従来の内省や意識調査などでは発見しにくいものであり、同時に、敬語行動研究におけるマルチメディア・コーパスの有用性を示すものである。

発話のみならず発話場面の映像․音声も参照できるコーパスを「マルチメディア․コーパス」と呼び、対談番組を資料として作成した『対談番組のマルチメディア․コーパス』(孫2010)に、新たにβ単位の見出し語․品詞․語種情報を搭載した。それによって、たとえば、「僕ら」「スチュワーデスら」「言ったら」「守っておられる」など「ら」を含む言語形式の中から、接尾辞「ら」․名詞․外来語など特定のもののみを抽出することが可能となった。そして搭載した言語情報をもとに、複数を表す接尾辞「たち」「ら」の使用例を検索し、その分布と、発話時における話し手の言語行動について調べた結果、「たち」と「ら」は組合わさる名詞․代名詞の種類によって使用分布が明確に異なるということ、同じ言語表現でも「僕ら=話し手と聞き手のみ」「僕ら=話し手と聞き手を含む人々」「僕ら=(聞き手を除く)人々と話し手」のように指示対象が異なる場合はそれによって発話の際に話し手が視線を向ける対象も変わるということなどが明らかになった。これらの調査․分析は、コーパスに言語情報を搭載することで可能となったことであり、同時に、言語情報を搭載したマルチメディア․コーパスの有用性を示すものでもある。ただし、今後「指示」という行為を、言語行動の観点からより正確に把握するためには、一人称複数代名詞と視線との関係以外に、指示と関わる多様な言語表現及び非言語表現をも対象に考察しなければならない。

L'objectif de cet article est de montrer comment on peut integrer les resultats des nouvelles analyses de corpus paralleles. Nous voulons illustrer d'autre part comment des types de corpus bilingues ouvrent de nouvelles perspectives aux apprenants et aux enseignants. Le corpus parallele devient un outil pedagogique multifonctionnel et flexible, beaucoup mieux adapte aux besoins d'utilisateurs tres divers. Nous exposons un usage pedagogique de corpus bilingues alignes dans la mise en oeuvre d'un outil de lecture de textes francais sur le Web. Nous presentons d'abord les principales fonctionnalites des corpus textuels. Nous decrivons ensuite le role specifique des corpus textuels dans un environnement multimedia. Nous montrons que les corpus paralleles peuvent aider les apprenants a traduire des textes litteraires. Nous mentionnons donc les principaux benefices de l'utilisation de corpus paralleles. On a amplement demontre, depuis plus de dix ans, l’interet que pouvait avoir l’utilisation d’un corpus dans le processus de traduction. Le corpus textuel est cependant souvent presente comme un outil permettant aux enseignants et aux apprenants de trouver des reponses a leurs questionnements. Du point de vue des concepteurs de l'outil (enseignants), l'acquisition systematique de tres grands corpus paralleles a ete facilitee par une serie d'outils d'extraction de textes bilingues sur le Web. Du point de vue des utilisateurs finaux (apprenants), les mots des lecons sont systematiquement illustres a l'aide d'exemples issus de ces corpus paralleles. L'apprenant peut ainsi observer le mot recherche sous plusieurs formes flechies et sa traduction dans plusieurs contextes minimaux et differencies, qui sont specifiques au contexte du mot candidat. Cette aide lexicale desambiguisee et contextualisee constitue une des plus fortes valeurs ajoutees du systeme.

In situations that include face-to-face utterances, the overall action of delivering information through verbal and non-verbal language (i.e., gaze, gesture, facial expression, and tone) is called “expressive behavior.” I compared the characteristics of that are commonly employed in expressive behavior research. Both types of corpus allow an examination of the images and voices of an utterance and include non-verbal language that is obtained from them as well as verbal language. However, the comparison revealed the following results. From the non-verbal language perspective, research that adopts multimodal corpora focuses on proving that various modes, including verbal language, gesture, and gaze, function dynamically in interactions between the speaker and listener as, for example, with turn-taking. In comparison, from the verbal language perspective, research that adopts multimedia corpora focuses on extracting patterned inclinations regarding the influences caused by the speaker’s verbal language in reciprocity to various elements (e.g., non-verbal language, the speaker’s generic characteristics, and information delivered by the utterance) that are related to expressive behavior. The difference in the focuses of the two types of corpus is distinctly reflected in the methods employed to collect the corpus data, and construct and utilize the corpus.