Human Gesture Recognition using Hidden Markov Models and Sensor Fusion


  • Emmanuel Domínguez Ramón National Institute of Astrophysical, Optical and Electronic
  • Raquel Diaz Hernandez
  • Leopoldo Altamirano Robles



Gesture Recognition, Collaborative Robotics, Vision-based Sensors, Myography Sensors, Sensor Fusion.


Considering the continued drive of human needs besides the constant improvement of technology, it is convenient to develop techniques that enhance the communication between computers and humans in the most intuitive ways as possible. The possibility to automatically recognize human gestures using artificial vision among other kind sensors
allows to explore a whole range of interaction applications to control and interact with environments. Nowadays, most of approaches for gesture recognition using sensors agree in the use of vision, myography and movement devices applied to robotic, medical and industrial applications. In the context of this work, we study the principles of using both vision and
body contact sensing applied to automatic classification of a human gesture set. For this, two different approaches are evaluated: Feed-forward Neural Networks and Hidden Markov Models. These models are studied and implemented for the recognition up to eight different human hand gestures commonly applied in collaborative robotics tasks. In our tests,
we conclude the effectiveness of combining the information of two different sort of devices for human gesture recognition reaching accuracy rates up to 95.05% for a whole proposed Hand-gesture set.


Download data is not yet available.

Author Biography

Emmanuel Domínguez Ramón, National Institute of Astrophysical, Optical and Electronic

Emmanuel Domínguez is a researcher in the autonomus perception laboratory, wich belong of department of computational vision in the National Institute of Astrophysics, Optical and Electronic. He studied Bachelor of Mechatronic Engineering in the Institute Tecnologic of Veracruz. In the actually he lives in Cholula, Puebla, developing research in the field of robotics, gesture recognition and artificial intelligence.


S.Mitra and t. Acharya "Gesture recognition: A survey"

P. Molchanov, S. Gupta, K. Kim, and K. Pulli, "Multisensor system for driver's hand-gesture recognition"

H. Junker, O. Amft, P. Lukowicz, and G. Tröster, “Gesture spotting with body-

worn inertial sensors to detect user activities,”

L. Keun-Cheol, J.-h. Kweon, K. Kyung-Jae, and J.-w. Choi, “Method and appa-

ratus for controlling a home device remotely in a home network system,”

W. Wang, R. Li, Z. M. Diekel, Y. Chen, Z. Zhang, and Y. Jia, “Controlling ob-

ject hand-over in human–robot collaboration via natural wearable sensing,”

H. Jin, Q. Chen, Z. Chen, Y. Hu, and J. Zhang, “Multi-leapmotion sensor based

demonstration for robotic refine tabletop object manipulation task,”

M. E. Benalcázar, C. E. Anchundia, J. A. Zea, P. Zambrano, A. G. Jaramillo, and

M. Segura, “Real-time hand gesture recognition based on artificial feed-forward

neural networks and emg,”

S. Athavale and M. Deshmukh, “Dynamic hand gesture recognition for human

computer interaction; a comparative study,”

X. Zabulis, H. Baltzakis, and A. A. Argyros, “Vision-based hand gesture recog-

nition for human-computer interaction.,”

T. T. Rautiainen, P. Hui, R. H. S. Kaunisto, I. A. Teikari, and J. P. J. Ollikainen,

“Gesture control,”

N. Ç. Kılıboz and U. Güdükbay, “A hand gesture recognition technique for human–

computer interaction,”

Y. Sun, C. Li, G. Li, G. Jiang, D. Jiang, H. Liu, Z. Zheng, and W. Shu, “Ges-

ture recognition based on kinect and semg signal fusion,”

S. J. M. Jorgensen et al., Human detection, gesture recognition, and policy gener-

ation for human-aware robots.

S. S. Rautaray and A. Agrawal, “Vision based hand gesture recognition for human

computer interaction: a survey,”

K. Sinha, R. Kumari, A. Priya, and P. Paul, “A computer vision-based gesture

recognition using hidden markov model,”

P. Premaratne, S. Yang, P. Vial, and Z. Ifthikar, “Centroid tracking based dynamic

hand gesture recognition using discrete hidden markov models,”

Y. S. FIGUEROA, “Reconocimiento anticipado de gestos,”

O. Pink, J. Becker, and S. Kammel, “Automated driving on public roads: Ex-

periences in real traffic,”

L. R. Rabiner and B.-H. Juang, “An introduction to hidden markov models,”

F. Lv and R. Nevatia, “Recognition and segmentation of 3-d human action us-

ing hmm and multi-class adaboost,”

M. R. Ahsan, M. I. Ibrahimy, and O. O. Khalifa, “Electromygraphy (emg) signal

based hand gesture recognition using artificial neural network (ann),”

M. Ahmad and S.-W. Lee, “Hmm-based human action recognition using multi-

view image sequences,”

T. Yu, C. Finn, A. Xie, S. Dasari, T. Zhang, P. Abbeel, and S. Levine, “One-

shot imitation from observing humans via domain-adaptive meta-learning,”




How to Cite

Domínguez Ramón, E., Diaz Hernandez, R., & Altamirano Robles , L. (2022). Human Gesture Recognition using Hidden Markov Models and Sensor Fusion. Computer Science, 23(2).