Abstract:

In view of the shortcomings of low accuracy, large size, and poor wearability of current virtual reality human-machine interaction devices, this study proposes a new type of assisted VR device for gesture recognition and virtual reality human-machine interaction. The system includes a glove consisting of a set of flex sensors and a nine-axis motion attitude sensor, and a wrist strap based on functional electrical stimulation. The data glove system transmits the data through wireless communication and these data is calculated in the attitude program algorithm on the computer. The results are used to control the hand posture constructed in VR scene in real time. The collision algorithm in the VR scene is adaopted to predict the force haptic value generated by the hand colliding with the object in the scene. Finally, the wireless transmission to the FES stimulation wristband generates different levels of stimulation pulses to assist in completing the force and tactile control of the hand, thereby achieving the effect of human-computer interaction. The measured experimental results show that the average recognition accuracy of a single finger of 5 healthy volunteers is 94.7%±1.3%. The FES wristband can provide a current output of 0~30 mA and a wrist joint strength control with a maximum torque of 1.8 N·m, and the action delay time is 20 ms.

Key words: gesture recognition, virtual reality, functional electrical stimulation, data glove, rehabilitation, tactile enhancement feedback