Let's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports

Simple item page
dc.contributor.authorSingh, Punit Kumar
dc.contributor.authorKumar, Nishant
dc.contributor.authorGhosh, Akash
dc.contributor.authorPasad, Kunal
dc.contributor.authorSoni, Khushi
dc.contributor.authorJaishwal, Manisha
dc.contributor.authorSaha, Sriparna
dc.contributor.authorAlfarozi, Syukron Abu Ishaq
dc.contributor.authorAbagissa, Asres Temam
dc.contributor.authorPasupa, Kitsuchart
dc.contributor.authorYang, Haiqin
dc.contributor.authorMoreno, Jose G.
dc.date.accessioned2026-08-06T10:48:27Z
dc.date.available2026-08-06T10:48:27Z
dc.date.issued2025-01-01
dc.description.abstractLanguage Models (LMs) are primarily evaluated on globally popular sports, often overlooking regional and indigenous sporting traditions. To address this gap, we introduce CultSportQA, a benchmark designed to assess LMs' understanding of traditional sports across 60 countries and 6 continents, encompassing four distinct cultural categories. The dataset features 33,000 multiple-choice questions (MCQs) across text and image modalities, each of which is categorized into three key types: history-based, rule-based, and scenario-based. To evaluate model performance, we employ zero-shot, few-shot, and chain-of-thought (CoT) prompting across a diverse set of Large Language Models (LLMs), Small Language Models (SLMs), and Multimodal Large Language Models (MLMs). By providing a comprehensive multilingual and multicultural sports benchmark, CultSportQA establishes a new standard for assessing AI's ability to understand and reason about traditional sports.
dc.identifier.citationEmnlp 2025 2025 Conference on Empirical Methods in Natural Language Processing Proceedings of the Conference, 15194-15241, 2025
dc.identifier.doi10.18653/v1/2025.emnlp-main.769
dc.identifier.other2-s2.0-105040230748
dc.identifier.urihttps://dspace.kmitl.ac.th/handle/123456789/16250
dc.sourceEmnlp 2025 2025 Conference on Empirical Methods in Natural Language Processing Proceedings of the Conference
dc.titleLet's Play Across Cultures: A Large Multilingual, Multicultural Benchmark for Assessing Language Models' Understanding of Sports
dc.typeConference Paper

Files

Collections

Research