A Persona-based Automated Evaluation Framework for Intelligent AI Teaching Assistants

Simple item page
dc.contributor.authorLimcharoen, Chananyu
dc.contributor.authorSripinta, Keetaphat
dc.contributor.authorPasupa, Kitsuchart
dc.date.accessioned2026-08-06T10:55:53Z
dc.date.available2026-08-06T10:55:53Z
dc.date.issued2026-06-16
dc.description.abstractTo address global teacher shortages and the need for personalized learning, we develop an intelligent teaching assistant leveraging multimodal large language models, agentic retrieval-augmented generation, and function calling. Unlike standard models, our system provides verifiable pedagogical guidance by reasoning across heterogeneous resources, including lecture slides and instructional videos. To overcome the scarcity of real-world datasets and high human evaluation costs, we propose a scalable, human-annotation-free framework utilizing simulated learner agents grounded in the Big Five personality theory. This allows for systematic assessment across 18 distinct student personas. We introduce a novel process-level metric, the dialogue recovery rate, and a dynamic adaptive policy to mitigate conversational deadlocks. Experimental results across 1,800 simulated dialogues using the Qwen3 family (8B, 14B, 32B) and its Thai-variant (Typhoon 2.5) reveal that Qwen3-14B attains the highest robustness and aggregate tutor-performance score within the proposed framework (72.6%). Analysis demonstrates significant correlations between learner traits and performance: Conscientiousness positively correlates with success (r = 0.49), while Extraversion is negatively associated with structural adherence (r = -0.54). This work establishes a reproducible benchmarking protocol for persona-aware, adaptive intelligent tutoring systems.
dc.identifier.citationIait 2026 14th International Conference on Advances in Information Technology, 2026
dc.identifier.doi10.1145/3816713.3819067
dc.identifier.other2-s2.0-105045238740
dc.identifier.urihttps://dspace.kmitl.ac.th/handle/123456789/18201
dc.sourceIait 2026 14th International Conference on Advances in Information Technology
dc.subjectAgentic RAG
dc.subjectBig Five Personality.
dc.subjectGenerative AI
dc.subjectIntelligent Teaching Assistant
dc.subjectMultimodal LLM
dc.subjectSimulated Learners
dc.titleA Persona-based Automated Evaluation Framework for Intelligent AI Teaching Assistants
dc.typeConference Paper

Files

Collections

Research