Benchmarking Next-Generation Frontier Models for Document-Level Arabic-Thai Medical Translation: A Reliability Study of LLM-as-a-Judge

Simple item page
dc.contributor.authorLertsuksakda, Rathawut
dc.contributor.authorNetisopakul, Ponrudee
dc.contributor.authorBoonsang, Siridech
dc.date.accessioned2026-08-06T10:53:33Z
dc.date.available2026-08-06T10:53:33Z
dc.date.issued2026-01-01
dc.description.abstractThis research targets Arabic-to-Thai medical translation, a low-resource and linguistically distant pair critical for public health. We utilize the OpenWHO dataset, a resource comprising 293 documents. This dataset is shielded from webcrawling to minimize training data contamination and ensure rigorous zero-shot evaluation. We conduct a comparative analysis between 2026-era frontier models including GPT-5.2, Gemini 3 Pro, and Claude Opus 4.5 and industry-standard NMT represented by Google Translate. Using an LLM-as-a-Judge framework with an AI jury of efficient reasoning models such as GPT-5.1, Gemini 3 Flash, and Claude Sonnet 4.5, we performed 3,516 evaluations of fidelity, fluency, and cultural appropriateness. Results demonstrate that frontier models outperform traditional NMT across all dimensions. While high exact agreement suggests LLM-as-a-Judge frameworks are promising for scalable evaluation, human validation reveals persistent opportunities for improving AI-human alignment, necessitating targeted oversight for safety-critical applications.
dc.identifier.citation2026 8th International Conference on Natural Language Processing Icnlp 2026, 162-166, 2026
dc.identifier.doi10.1109/ICNLP69856.2026.11527976
dc.identifier.other2-s2.0-105041797231
dc.identifier.urihttps://dspace.kmitl.ac.th/handle/123456789/17593
dc.source2026 8th International Conference on Natural Language Processing Icnlp 2026
dc.subjectAI Safety
dc.subjectArabic-Thai
dc.subjectLarge Language Models
dc.subjectLLM-as-a-Judge
dc.subjectLow-Resource Machine Translation
dc.subjectMachine Translation
dc.subjectMedical Translation
dc.titleBenchmarking Next-Generation Frontier Models for Document-Level Arabic-Thai Medical Translation: A Reliability Study of LLM-as-a-Judge
dc.typeConference Paper

Files

Collections

Research