Classifying Lexical Variations in Thai Social Media: A Machine Learning and LLMs Approach

Abstract

Social media platforms like Facebook, X (formerly Twitter), and TikTok produce large amounts of user content, offering valuable data for Social Media Analytics (SMA). However, analyzing Thai social media presents challenges due to lexical variations, such as spelling changes, semantic shifts, and the creation of new words (lexicogenesis). These features, along with the lack of spaces between words in Thai script, complicate traditional text analysis methods. This study addresses these challenges by using a combination of machine learning (ML) models and large language models (LLMs). We employ Bidirectional Long Short-Term Memory (Bi-LSTM) networks, pretrained models like BERT-base-Thai, and advanced LLMs such as LLaMA-3, Mistral, Gemma, and Phi to classify and analyze Thai social media content. Our research enhances the understanding of lexical variations and improves the processing of Thai language in social media, contributing to the development of more effective natural language processing tools.