Playing Rubik's cube in mixed reality

Abstract

This paper describes a prototype system for virtual objects manipulation in mixed reality environment. SPIDAR-8, the multi-fingers string-based haptic interface device, is used to display force feedback at four fingertips on each hand of the user, i.e., thumb, index, middle, and ring finger. Fingertip positions measured by the haptic interface device are also used to model the 3D virtual hands. Then image sequences of real hands of the user taken by video camera are superimposed on the silhouette of the virtual hands. By employing the technique of sensors fusion and the successive adjustment algorithm, correct geometric registration of image of real hands and the virtual objects can be achieved. The user can see his or her real hands manipulating the virtual objects in the simulated virtual world and perceive force feedback in the same way as manipulating real objects. The system is aimed to enhance the sensation of immerse into the virtual world. © 2003 by Springer Science+Business Media New York.