TrafficInternVL: Spatially-Guided Fine-Tuning with Caption Refinement for Fine-Grained Traffic Safety Captioning and Visual Question Answering - Sasin Phimsiri; Sarut Sunpawatr; Riu Cherdchusakulchai; Pornprom Kiawjak; Teepakorn Tosawadi; Suchat Tungjitnob; Visarut Trairattanapa; Supawit Vatathanavaro; Wasu Kudisthalert; Chaitat Utintu; Worawit Saetan; Nathamon Kongsawat; et al. (My University, 2025)
Fine-grained traffic understanding requires both detailed visual descriptions and precise answers to safety-critical questions. We present TrafficInternVl, a framework for fine-grained traffic safety description and question answering, developed for AI City Challenge 2025 Track 2. Our approach is based on the InternVL3-38B vision-language model and integrates four key components: (1) spatially guided visual prompting via bounding-box-based cropping and rendering; (2) Adaptive view selection protocols; (3) low-rank adaptation (LoRA) fine-tuning, updating only 1% of model parameters; and (4) caption refinement for intra-scene consistency. Our model achieves a Caption Score of 32.75 (BLEU-4, METEOR, ROUGE-L, CIDEr averaged) and a VQA accuracy of 83.08 %. Code, prompts, and LoRA weights are released at https://github.com/ARV-MLCORE/TrafficInternVL
Keywords: Multimodal Machine Learning Applications; Topic Modeling; Video Analysis and Summarization