Earticle

초록

영어

This study explores the practical application of large language models (LLMs) in marketing, addressing the challenges posed by their stochastic and context-agnostic nature. By employing a human-in-the-loop approach with domain experts, the study aligns LLM-generated content with specific contexts and evaluates the impact of semantic inconsistency on user engagement. In collaboration with a major South Korean TV manufacturer, the researchers conducted a randomized field experiment with 39,588 smart TV devices, testing LLM-generated persuasive messages to recommend TV content. The results demonstrate that context-relevant LLM-generated messages significantly improve click-through rates. However, semantically inconsistent messages diminish this effect. These findings underscore the need to mitigate LLMs' stochastic nature through human oversight to ensure consistent and effective user engagement.

목차

Abstract
Introduction
Literature Review
Stochastic and Context-agnostic Nature of LLMs
Contextual Targeting and Priming Effect
Hypotheses Development
Method
Research Context: Smart TV Content Recommendation
Message Preparation: Human-in-the-loop Process
Experimental Design
Analysis and Results
Results of the Empirical Model
Empirical Extensions
Conclusions
Contributions
Limitations
References

저자

• Changhyun Lee [ Department of Management Information Systems, Hanyang University ]
• Jiyong Park [ Department of Management Information Systems, Terry College of Business, the University of Georgia ]
• Kyungjin Cha [ Department of Management Information Systems, Hanyang University ]

참고문헌