Lifelogging system based on Averaged Hidden Markov Models: dangerous activities recognition for caregivers support

Aleksandra Postawka; Jarosław Rudy

doi:10.7494/csci.2018.19.3.2855

Authors

Aleksandra Postawka Department of Computer Engineering, Wrocław University of Science and Technology http://orcid.org/0000-0001-8439-6951
Jarosław Rudy Department of Computer Engineering, Wrocław University of Science and Technology http://orcid.org/0000-0003-1095-6041

DOI:

https://doi.org/10.7494/csci.2018.19.3.2855

Keywords:

Lifelogging, Abnormal Human Activity Recognition, Hidden Markov Models, Machine Vision, Microsoft Kinect

Abstract

In this paper a prototype lifelogging system for monitoring persons with cognitive disabilities and elderly people, as well as a method for automatic detection of dangerous activities are presented. The system allows remote monitoring of observed persons via Internet website and respects the privacy of the persons by displaying their silhouettes instead of actual images. Application allows viewing of both real-time and historic data. Lifelogging data (skeleton coordinates) needed for posture and activity recognition are acquired using Microsoft Kinect 2.0. Several activities are marked as potentially dangerous and generate alarms sent to the caregivers upon detection. Recognition models are developed using Averaged Hidden Markov Models with multiple learning sequences. Action recognition includes methods for differentiation between normal and potentially dangerous activities e.g. self-aggressive autistic behavior) using the same motion trajectory. Some activity recognition examples and results are presented.

Downloads

Download data is not yet available.

References

Ann O.C., Theng L.B.: Human activity recognition: A review. In: 2014 IEEE International Conference on Control System, Computing and Engineering (ICCSCE 2014), pp. 389–393. 2014. URL http://dx.doi.org/10.1109/ICCSCE.2014.7072750.

Gurrin C., Smeaton A.F., Doherty A.R.: LifeLogging: Personal Big Data. In: Foundations and Trends in Information Retrieval, vol. 8(1), pp. 1–125, 2014. ISSN 1554-0669. URL http://dx.doi.org/10.1561/1500000033.

Jalal A., Kamal S., Kim D.: A Depth Video Sensor-Based Life-Logging Human Activity Recognition System for Elderly Care in Smart Indoor Environments. In: Sensors, vol. 14(7), pp. 11735–11759, 2014. ISSN 1424-8220. URL http: //dx.doi.org/10.3390/s140711735.

van Kasteren T.L.M., Englebienne G., Kröse B.J.A.: An activity monitoring system for elderly care using generative and discriminative models. In: Personal and Ubiquitous Computing, vol. 14(6), pp. 489–498, 2010. ISSN 1617-4917. URL http://dx.doi.org/10.1007/s00779-009-0277-9.

Khan Z.A., Sohn W.: Abnormal human activity recognition system based on R-transform and kernel discriminant technique for elderly home care. In: IEEE Transactions on Consumer Electronics, vol. 57(4), pp. 1843–1850, 2011. ISSN 0098-3063. URL http://dx.doi.org/10.1109/TCE.2011.6131162.

Mann S.: Wearable Wireless Webcam. URL http://wearcam.org/netcam.html. Accessed on 11th of December 2017.

Miaou S.G., Sung P.H., Huang C.Y.: A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information. In: 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, 2006. D2H2., pp. 39–42. 2006. URL http://dx.doi.org/10.1109/DDHH.2006.1624792.

Nana P., Min D., Yue Z., Xin C., Sheng B.: The Elderly’s Falling Motion Recognition Based on Kinect and Wearable Sensors, pp. 1129–1141. Springer International Publishing, Cham, 2017. ISBN 978-3-319-48036-7. URL http://dx.doi.org/10.1007/978-3-319-48036-7_83.

O’Hara K., Tuffield M.M., Shadbolt N.: Lifelogging: Privacy and empowerment with memories for life. In: Identity in the Information Society, vol. 1(1), pp. 155–172, 2008. ISSN 1876-0678. URL http://dx.doi.org/10.1007/s12394-009-0008-4.

Pal M., Saha S., Konar A.: Distance matching based gesture recognition for healthcare using Microsoft’s Kinect sensor. In: 2016 International Conference on Microelectronics, Computing and Communications (MicroCom), pp. 1–6. 2016. URL http://dx.doi.org/10.1109/MicroCom.2016.7522586.

Popoola O.P., Wang K.: Video-Based Abnormal Human Behavior Recognition—A Review. In: IEEE Transactions on Systems, Man, and Cybernetics, Part C(Applications and Reviews), vol. 42(6), pp. 865–878, 2012. ISSN 1094-6977. URL http://dx.doi.org/10.1109/TSMCC.2011.2178594.

a.B. Poritz: Hidden Markov models: a guided tour. In: ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing, pp. 7–13, 1988. ISSN 1520-6149. URL http://dx.doi.org/10.1109/ICASSP.1988.196495.

Postawka A.: Exercise Recognition Using Averaged Hidden Markov Models, pp. 137–147. Springer International Publishing, Cham, 2017. ISBN 978-3-319-59060-8. URL http://dx.doi.org/10.1007/978-3-319-59060-8_14.

Postawka A.: Real-time monitoring system for potentially dangerous activities detection. In: 2017 22nd International Conference on Methods and Models in Automation and Robotics (MMAR), pp. 1005–1008. 2017. URL http://dx.doi.org/10.1109/MMAR.2017.8046967.

Postawka A., Śliwiński P.: A Kinect-Based Support System for Children with Autism Spectrum Disorder, pp. 189–199. Springer International Publishing, Cham, 2016. ISBN 978-3-319-39384-1. URL http://dx.doi.org/10.1007/978-3-319-39384-1_17.

Rabiner L., Juang B.: An introduction to hidden Markov models. In: IEEE ASSP Magazine, vol. 3(January), p. Appendix 3A, 1986. ISSN 0740-7467. URL http://dx.doi.org/10.1109 MASSP.1986.1165342.

Rowe M., Lane S., Phipps C.: CareWatch: A Home Monitoring System for Use in Homes of Persons With Cognitive Impairment. In: Topics in Geriatric Rehabilitation, vol. 23(1), pp. 3–8, 2007.

Siddiqui S.A., Snober Y., Raza S., Khan F.M., Syed T.Q.: Arm gesture recognition on Microsoft Kinect using a Hidden Markov Model-based representations of poses. In: 2015 International Conference on Information and Communication Technologies (ICICT), pp. 1–6. 2015. URL http://dx.doi.org/10.1109/ICICT.2015.7469478.

Wu P., Peng H.K., Zhu J., Zhang Y.: SensCare: Semi-automatic Activity Summarization System for Elderly Care, pp. 1–19. Springer Berlin Heidelberg, Berlin, Heidelberg, 2012. ISBN 978-3-642-32320-1. URL http://dx.doi.org/10.1007/978-3-642-32320-1_1.

Yin J., Yang Q., Pan J.J.: Sensor-Based Abnormal Human-Activity Detection. In: IEEE Trans. on Knowl. and Data Eng., vol. 20(8), pp. 1082–1090, 2008. ISSN 1041-4347. URL http://dx.doi.org/10.1109/TKDE.2007.1042.

Yoshihara Y., Tang D., Kubota N.: Life Log Visualization System Based on Informationally Structured Space for Supporting Elderly People. In: 2013 Second International Conference on Robot, Vision and Signal Processing, pp. 78–83. 2013. ISSN 2376-9793. URL http://dx.doi.org/10.1109/RVSP.2013.25.

Yu M., Rhuma A., Naqvi S.M., Wang L., Chambers J.: A Posture Recognition Based Fall Detection System for Monitoring an Elderly Person in a Smart Home Environment. In: IEEE Transactions on Information Technology in Biomedicine, vol. 16(6), pp. 1274–1286, 2012. ISSN 1089-7771. URL http://dx.doi.org/10.1109/TITB.2012.2214786.

Yu X., Wu L., Liu Q., Zhou H.: Children tantrum behaviour analysis based on Kinect sensor. In: 2011 Third Chinese Conference on Intelligent Visual Surveillance, pp. 49–52. 2011. URL http://dx.doi.org/10.1109/IVSurv.2011.6157022.