Sign Language Sentence Recognition Using Hybrid Graph Embedding and Adaptive Convolutional Networks

Abstract

Sign language plays a crucial role in bridging communication barriers within the Deaf community. Recognizing sign language sentences remains a significant challenge due to their complex structure, variations in signing styles, and temporal dynamics. This study introduces an innovative sign language sentence recognition (SLSR) approach using Hybrid Graph Embedding and Adaptive Convolutional Networks (HGE-ACN) specifically developed for single-handed wearable glove devices. The system relies on sensor data from a glove with six-axis inertial sensors and five-finger curvature sensors. The proposed HGE-ACN framework integrates graph-based embeddings to capture dynamic spatial–temporal relationships in motion and curvature data. At the same time, the Adaptive Convolutional Networks extract robust glove-based features to handle variations in signing speed, transitions between gestures, and individual signer styles. The lightweight design enables real-time processing and enhances recognition accuracy, making it suitable for practical use. Extensive experiments demonstrate that HGE-ACN achieves superior accuracy and computational efficiency compared to existing glove-based recognition methods. The system maintains robustness under various conditions, including inconsistent signing speeds and environmental noise. This work has promising applications in real-time assistive tools, educational technologies, and human–computer interaction systems, facilitating more inclusive and accessible communication platforms for the deaf and hard-of-hearing communities. Future work will explore multi-lingual sign language recognition and real-world deployment across diverse environments.