Multi-agent deep learning on tensor fields for segmentation of ultrasound images

Abstract

Medical image analysis often relies on vector fields (VF), which are fundamental to deterministic models such as Active Contours, Level Set Methods, Phase Portrait Analysis, and artificial agent–based formulations. We experimentally demonstrate that a Deep Learning Neural Network (DLNN) capable of interpreting VF structures can substantially enhance the decision-making capabilities of artificial agents. We introduce a novel hybrid framework that integrates artificial life (AL) agents operating within a VF with a DLNN that guides their behavior. A key innovation of the model is the initialization of AL agents using streamlines derived from the VF orthogonal to the generalized gradient vector flow (GGVF) field. The VF is further transformed into a bi-directional Tensor Field (TF), where the spatial distribution and classification of degenerate points (DPs) serve as critical features. These DPs are leveraged to train AL agents through the DLNN, enabling them to follow meaningful anatomical structures. The framework employs DeepLabV3+ with ResNet50 as the backbone and is trained on 179 benign and 107 malignant breast ultrasound images collected at Thammasat University Hospital (TUH) and annotated by three leading radiologists, in addition to the BUSI and UDIAT datasets. Using 10-fold cross-validation, the proposed method achieves stable and robust performance across three datasets. Mean Dice scores of 94 . 84 ± 1 . 63 % (TUH), 94 . 16 ± 1 . 62 % (BUSI), and 93 . 67 ± 1 . 51 % (UDIAT) are obtained, with corresponding IoU values of 91 . 19 ± 1 . 76 % , 90 . 21 ± 1 . 83 % and 89 . 08 ± 1 . 70 % , demonstrating strong generalization across diverse imaging conditions. Comparative evaluations against state-of-the-art methods confirm the superiority of the proposed model. A video demonstration is available at: https://tinyurl.com/AL-DLNN .