Earticle

초록

영어

We provide a detailed analysis of the data processing and model training process for vulnerability classification using Transformer-based language models, especially sentence text-to-text transformers (ST5)-XXL and XLNet. The main purpose of this study is to compare the performance of the two models, identify the strengths and weaknesses of each, and determine the optimal learning rate to increase the efficiency and stability of model training. We performed data preprocessing, constructed and trained models, and evaluated performance based on data sets with various characteristics. We confirmed that the XLNet model showed excellent performance at learning rates of 1e-05 and 1e-04 and had a significantly lower loss value than the ST5-XXL model. This indicates that XLNet is more efficient for learning. Additionally, we confirmed in our study that learning rate has a significant impact on model performance. The results of the study highlight the usefulness of ST5-XXL and XLNet models in the task of classifying security vulnerabilities and highlight the importance of setting an appropriate learning rate. Future research should include more comprehensive analyzes using diverse data sets and additional models.

목차

Abstract
1. Introduction
2. Experimental Models: ST5-XXL and XLNet
2.1 Multi-Head Self-Attention Layer:
2.2 Feedforward Neural Network Layer:
3. Experiments and Results
3.1 Features of data sets used experiments
3.2 Workflow of XLNet & ST5-XXL model
3.3 Experimental Results
4. Conclusion
Reference

키워드

저자

• Chae-Rim Hong [ Graduate Student, Department of AI & Bigdata, aSSIST University, Korea ]
• Jin-Keun Hong [ Professor, Div. of Advanced IT, Baekseok University, Korea ] Corresponding Author

참고문헌

발행기관

간행물

표지보기 관심저널 등록

• 간행물명

  International Journal of Internet, Broadcasting and Communication

• 간기

  계간

• pISSN

  2288-4920

• eISSN

  2288-4939

• 수록기간

  2009~2025

• 십진분류

  KDC 326 DDC 380