Earticle

초록

영어

Recently, Large Language Models (LLMs) have led to rapid performance improvements across natural language processing (NLP) and are expanding into various domains. However, pre-trained LLMs are based on general-purpose data, which can lead to domain bias and learning inefficiencies when applied to specific domains. Consequently, efficient fine-tuning techniques that can optimize specific domains while maintaining the model's generalizability are emerging as a key research topic. To overcome these limitations, this study proposed an Adaptive Layer-wise Fine-tuning (ALF) technique that performs selective parameter updates based on layer importance. ALF calculates the weight change rate of each layer during the learning process and dynamically updates only the layers with high importance, thereby improving efficiency without performance degradation while learning only about 20-30% of the total model parameters. The results of the attention weight analysis confirmed that ALF had improved focus on semantically central words, thereby improving contextual consistency and model interpretability (interpretable fine-tuning). These results suggest that ALF is an effective approach that simultaneously improves efficiency and generalization performance compared to existing fine-tuning techniques. Future research will develop a framework for optimizing generative AI models that is both sustainable and reliable, through expansion into multimodal data, reinforcement learning-based auto-tuning, and ethical fine-tuning.

목차

Abstract
1. Introduction
2. Related Work
2.1 Overview of Large Language Models (LLMs)
2.2 Basic concepts and classification of fine-tuning
2.3 Recent Research Trends in Adaptive Fine-tuning
3. Method
3.1 Overview of Research Framework
3.2 Experimental Models and Datasets
3.3 Proposed Fine-tuning Approach: Adaptive Layer-wise Fine-tuning (ALF)
3.4 Implementation Environment
4. Results and Discussion
4.1 Quantitative Performance Comparison
4.2 Attention Weight Distribution Analysis
4.3 Efficiency Evaluation
4.4 Discussion
5. Conclusion
References

키워드

저자

• Jee Young Lee [ Associate Professor, Department of Software, SeoKyeong University, Korea ] Corresponding Author

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  The International Journal of Advanced Smart Convergence

• 간기

  계간

• pISSN

  2288-2847

• eISSN

  2288-2855

• 수록기간

  2012~2025

• 십진분류

  KDC 326 DDC 380