Integrated Human-Machine Interaction System: ERP-SSVEP and Eye Tracking Based Technologies

Abstract

Human-Machine Interaction (HMI) requires a multidisciplinary research study mainly focused on interaction modalities between humans and machines. In this paper, we introduced an integrated HMI system using electrical brainwave signal (or called Electroencephalography: EEG) and eye tracking of pupil movement (or called ET), whose are used as an alternative channel to communicate with others for people with disabilities. In this experiment, the target and non-Target visual stimuli of EEG-based HMI system on the basis of event-related potential (ERP) and steady state visually evoked potential (SSVEP) signals have been performed. For the ET based framework, we proposed the user-friendly virtual keyboard typing for Thai language, i.e., free-form and automatic typing modes. The results showed that the integrated HMI using ERP-SSVEP yielded an average accuracy of 97.4% and reaction time approximately was 724.2 millisecond for control commands. The automatic typing mode performed an average accuracy of 97%, with an average printing time of 6.17 seconds per word for ET based virtual Thai keyboard.