Advancing GAN Evaluation: The Advanced Mahalanobis Distance Learning Metric for Realistic Car Damage Image Assessment

Generative Adversarial Networks (GANs) have demonstrated remarkable capability in synthesizing high-quality images from limited data, addressing challenges of data scarcity and diversity in deep learning (DL) training. This is particularly valuable for car damage classification, where real-world datasets are often limited. To mitigate this, we created a custom damaged-undamaged car dataset for training GAN models and generating realistic car damage images. However, evaluating the per-image realism of GAN-generated images remains challenging. Standard GAN metrics, such as Fréchet Inception Distance (FID), Kernel Inception Distance (KID), and Inception Score (IS), provide dataset-level scores but do not assess individual image quality. Meanwhile, Image Quality Assessment (IQA) metrics require reference images, rendering them unsuitable for reference-free scenarios, particularly in unpaired GAN-generated data. To address these limitations—including the practical failure of standard Mahalanobis Distance Learning (MDL) on small or high-dimensional datasets due to non-invertible covariance matrices—we propose Advanced Mahalanobis Distance Learning (AMDL), which incorporates adaptive regularization and pseudo-inverse refinement on deep feature embeddings from pre-trained CNNs. AMDL enables stable and reliable per-image realism assessment under covariance matrix instability, without requiring large datasets or ground-truth references. Our comprehensive evaluation framework involves three procedures: (1) dataset-level evaluation of four GAN models using standard GAN metrics, (2) per-image realism assessment with AMDL, and (3) classifier-based validation with CNN and Vision Transformer (ViT) models (with vs. without AMDL). Experimental results show that AMDL provides precise per-image realism assessment, outperforms existing GAN metrics across datasets, and offers a practical solution for evaluating unpaired GAN-generated images in car damage classification.