Lifelogging system based on Averaged Hidden Markov Models: dangerous activities recognition for caregivers support

Aleksandra Postawka, Jarosław Rudy


In this paper a prototype lifelogging system for monitoring persons with cognitive disabilities and elderly people, as well as a method for automatic detection of dangerous activities are presented. The system allows remote monitoring of observed persons via Internet website and respects the privacy of the persons by displaying their silhouettes instead of actual images. Application allows viewing of both real-time and historic data. Lifelogging data (skeleton coordinates) needed for posture and activity recognition are acquired using Microsoft Kinect 2.0. Several activities are marked as potentially dangerous and generate alarms sent to the caregivers upon detection. Recognition models are developed using Averaged Hidden Markov Models with multiple learning sequences. Action recognition includes methods for differentiation between normal and potentially dangerous activities e.g. self-aggressive autistic behavior) using the same motion trajectory. Some activity recognition examples and results are presented.


Lifelogging; Abnormal Human Activity Recognition; Hidden Markov Models; Machine Vision; Microsoft Kinect

Full Text:



Ann O.C., Theng L.B.: Human activity recognition: A review. In: 2014 IEEE International Conference on Control System, Computing and Engineering (ICCSCE 2014), pp. 389–393. 2014. URL

Gurrin C., Smeaton A.F., Doherty A.R.: LifeLogging: Personal Big Data. In: Foundations and Trends in Information Retrieval, vol. 8(1), pp. 1–125, 2014. ISSN 1554-0669. URL

Jalal A., Kamal S., Kim D.: A Depth Video Sensor-Based Life-Logging Human Activity Recognition System for Elderly Care in Smart Indoor Environments. In: Sensors, vol. 14(7), pp. 11735–11759, 2014. ISSN 1424-8220. URL http: //

van Kasteren T.L.M., Englebienne G., Kröse B.J.A.: An activity monitoring system for elderly care using generative and discriminative models. In: Personal and Ubiquitous Computing, vol. 14(6), pp. 489–498, 2010. ISSN 1617-4917. URL

Khan Z.A., Sohn W.: Abnormal human activity recognition system based on R-transform and kernel discriminant technique for elderly home care. In: IEEE Transactions on Consumer Electronics, vol. 57(4), pp. 1843–1850, 2011. ISSN 0098-3063. URL

Mann S.: Wearable Wireless Webcam. URL Accessed on 11th of December 2017.

Miaou S.G., Sung P.H., Huang C.Y.: A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information. In: 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, 2006. D2H2., pp. 39–42. 2006. URL

Nana P., Min D., Yue Z., Xin C., Sheng B.: The Elderly’s Falling Motion Recognition Based on Kinect and Wearable Sensors, pp. 1129–1141. Springer International Publishing, Cham, 2017. ISBN 978-3-319-48036-7. URL

O’Hara K., Tuffield M.M., Shadbolt N.: Lifelogging: Privacy and empowerment with memories for life. In: Identity in the Information Society, vol. 1(1), pp. 155–172, 2008. ISSN 1876-0678. URL

Pal M., Saha S., Konar A.: Distance matching based gesture recognition for healthcare using Microsoft’s Kinect sensor. In: 2016 International Conference on Microelectronics, Computing and Communications (MicroCom), pp. 1–6. 2016. URL

Popoola O.P., Wang K.: Video-Based Abnormal Human Behavior Recognition—A Review. In: IEEE Transactions on Systems, Man, and Cybernetics, Part C(Applications and Reviews), vol. 42(6), pp. 865–878, 2012. ISSN 1094-6977. URL

a.B. Poritz: Hidden Markov models: a guided tour. In: ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing, pp. 7–13, 1988. ISSN 1520-6149. URL

Postawka A.: Exercise Recognition Using Averaged Hidden Markov Models, pp. 137–147. Springer International Publishing, Cham, 2017. ISBN 978-3-319-59060-8. URL

Postawka A.: Real-time monitoring system for potentially dangerous activities detection. In: 2017 22nd International Conference on Methods and Models in Automation and Robotics (MMAR), pp. 1005–1008. 2017. URL

Postawka A., Śliwiński P.: A Kinect-Based Support System for Children with Autism Spectrum Disorder, pp. 189–199. Springer International Publishing, Cham, 2016. ISBN 978-3-319-39384-1. URL

Rabiner L., Juang B.: An introduction to hidden Markov models. In: IEEE ASSP Magazine, vol. 3(January), p. Appendix 3A, 1986. ISSN 0740-7467. URL MASSP.1986.1165342.

Rowe M., Lane S., Phipps C.: CareWatch: A Home Monitoring System for Use in Homes of Persons With Cognitive Impairment. In: Topics in Geriatric Rehabilitation, vol. 23(1), pp. 3–8, 2007.

Siddiqui S.A., Snober Y., Raza S., Khan F.M., Syed T.Q.: Arm gesture recognition on Microsoft Kinect using a Hidden Markov Model-based representations of poses. In: 2015 International Conference on Information and Communication Technologies (ICICT), pp. 1–6. 2015. URL

Wu P., Peng H.K., Zhu J., Zhang Y.: SensCare: Semi-automatic Activity Summarization System for Elderly Care, pp. 1–19. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. ISBN 978-3-642-32320-1. URL

Yin J., Yang Q., Pan J.J.: Sensor-Based Abnormal Human-Activity Detection. In: IEEE Trans. on Knowl. and Data Eng., vol. 20(8), pp. 1082–1090, 2008. ISSN 1041-4347. URL

Yoshihara Y., Tang D., Kubota N.: Life Log Visualization System Based on Informationally Structured Space for Supporting Elderly People. In: 2013 Second International Conference on Robot, Vision and Signal Processing, pp. 78–83. 2013. ISSN 2376-9793. URL

Yu M., Rhuma A., Naqvi S.M., Wang L., Chambers J.: A Posture Recognition Based Fall Detection System for Monitoring an Elderly Person in a Smart Home Environment. In: IEEE Transactions on Information Technology in Biomedicine, vol. 16(6), pp. 1274–1286, 2012. ISSN 1089-7771. URL

Yu X., Wu L., Liu Q., Zhou H.: Children tantrum behaviour analysis based on Kinect sensor. In: 2011 Third Chinese Conference on Intelligent Visual Surveillance, pp. 49–52. 2011. URL



  • There are currently no refbacks.