Loading...
Assessing the Effects of Corrupted Parameters in a Large Language Model: A Case Study of LLAMA 3.2 1B
Author(s)
Thongsawaeng, Anon
Perdio, John Paul Layug
Zaw, Kaung Khant
Kraichoke, Phalat
Nwe, Hlaing Myat
Kertkeidkachorn, Natthawut
Date Issued
January 21, 2026
Type
Conference Paper
Abstract
This study explores the effects of parameter corruption in a large language model (LLM) by altering its weights and evaluating performance. Experiments involve corrupting different layers and matrix types, including Self-Attention and Feed-Forward components, with performance assessed using BERT and ROUGE scores. Testing the Llama-3.2-1B-Instruct model was performed on the GLUE-QNLI dataset. Results show that increased corruption leads to greater degradation, with Feed-Forward matrices having a stronger impact especially in the Down matrices. According to the study, later layers are more important for performance than those that come before them. These results shed light on possible future chip implementations of LLM, which may help guide the design of fault-tolerant systems by taking vulnerable parameter placement into account.
Citation
Asse 2025 2025 6th Asia Service Sciences and Software Engineering Conference, 64-69, 2026