Loading...
Quantitative Assessment of Retrieval Strategies in RAG Architectures: A Comparative Study Across Multiple Knowledge Domains Using Standardized Performance Metrics
Author(s)
Aromsuk, Tinnarat
Nootyaskool, Supakit
Netisopakul, Ponrudee
Date Issued
January 1, 2025
Type
Conference Paper
Abstract
This study conducts a systematic quantitative assessment of four distinct retrieval methodologies within Retrieval-Augmented Generation (RAG) frameworks: baseline implementation, hybrid dual-paradigm approach, hierarchical parent-child structure, and contextual compression. Through rigorous experimental evaluation spanning six distinct knowledge domains, we employ established metrics including ROUGE [1], BLEU [2], and computational timing measurements to characterize performance profiles. Our findings reveal that sophisticated retrieval approaches deliver substantial computational efficiency gains (4-5 × acceleration) alongside varied performance patterns across quality assessment dimensions. The hierarchical parent-child methodology demonstrates superior BLEU performance (0.1046 mean score) coupled with optimal retrieval speeds (0.0124 s), whereas hybrid approaches excel in ROUGE metrics (0.0317 mean score). Domain-specific analysis indicates pronounced performance disparities: medical/health domains (COVID19 pandemic) achieve highest aggregate scores (0.1198 mean ROUGE), while specialized technical and legal domains present distinct retrieval complexities. This research establishes empirical foundations for evidence-based retrieval method selection, identifying clear efficiency-quality relationships and domain-dependent optimization strategies for production RAG deployments.
Citation
Proceedings of the International Conference on Information Technology and Electrical Engineering Icitee, 2025