Loading...
A Persona-based Automated Evaluation Framework for Intelligent AI Teaching Assistants
Author(s)
Date Issued
June 16, 2026
Type
Conference Paper
Abstract
To address global teacher shortages and the need for personalized learning, we develop an intelligent teaching assistant leveraging multimodal large language models, agentic retrieval-augmented generation, and function calling. Unlike standard models, our system provides verifiable pedagogical guidance by reasoning across heterogeneous resources, including lecture slides and instructional videos. To overcome the scarcity of real-world datasets and high human evaluation costs, we propose a scalable, human-annotation-free framework utilizing simulated learner agents grounded in the Big Five personality theory. This allows for systematic assessment across 18 distinct student personas. We introduce a novel process-level metric, the dialogue recovery rate, and a dynamic adaptive policy to mitigate conversational deadlocks. Experimental results across 1,800 simulated dialogues using the Qwen3 family (8B, 14B, 32B) and its Thai-variant (Typhoon 2.5) reveal that Qwen3-14B attains the highest robustness and aggregate tutor-performance score within the proposed framework (72.6%). Analysis demonstrates significant correlations between learner traits and performance: Conscientiousness positively correlates with success (r = 0.49), while Extraversion is negatively associated with structural adherence (r = -0.54). This work establishes a reproducible benchmarking protocol for persona-aware, adaptive intelligent tutoring systems.
Citation
Iait 2026 14th International Conference on Advances in Information Technology, 2026