TinyML based Deep Learning Model for Activity Detection
Article Sidebar
Main Article Content
Abstract
Our physical and emotional well-being are directly impacted by our body positions. In addition to promoting a confident, upright image, maintaining good body posture during various activities also ensures that our musculoskeletal system is properly aligned. On the other side, bad posture can result in a number of musculoskeletal conditions, discomfort, and reduced productivity. Accurate systems that can detect posture in real time, activity detection, are required due to the rising use of wearable technology and the growing interest in health and fitness tracking. The goal of this project is to create a TinyML model for wearable activity detection that will allow users to assess their posture and make necessary corrections in order to improve their health and general well-being. The project intends to contribute to the creation of useful posture detection technologies that can be quickly implemented on wearable devices for widespread usage by leveraging machine learning algorithms and wearable sensor data. For reliable posture categorization, the model architecture combines deep neural networks (DNN) and LSTM layers. With the development and implementation of the TinyML model, a significant decrease in the model's power consumption, memory, and latency was achieved without any compromise in the accuracy. This work can be used in the fields of health, wellness, rehabilitation, corporate life, sports and fitness to keep track of calories burned, activity duration, distance traveled, posture analysis, and real-time tracking.
Article Details
References
Surantha, Nico, Prabadinata Atmaja, and Maulana Wicaksono. "A review of wearable internet-of-things device for healthcare." Procedia Computer Science 179 (2021): 936-943.
Ihoume, Ilham, Rachid Tadili, Nora Arbaoui, Mohamed Benchrifa, Ahmed Idrissi, and Mohamed Daoudi. "Developing a multi-label tinyML machine learning model for an active and optimized greenhouse microclimate control from multivariate sensed data." Artificial Intelligence in Agriculture 6 (2022): 129-137.
Awad, Ahmed, Mostafa M. Fouda, Marwa M. Khashaba, Ehab R. Mohamed, and Khalid M. Hosny. "Utilization of mobile edge computing on the Internet of Medical Things: A survey." ICT Express (2022).
Rüb, Marcus, and Axel Sikora. "A Practical View on Training Neural Networks in the Edge." IFAC-PapersOnLine 55, no. 4 (2022): 272-279.
Ray, Partha Pratim. "A review on TinyML: State-of-the-art and prospects." Journal of King Saud University-Computer and Information Sciences (2021).
Altayeb, Moez, Marco Zennaro, and Ermanno Pietrosemoli. "TinyML Gamma Radiation Classifier." Nuclear Engineering and Technology (2022).
Signoretti, Gabriel, Marianne Silva, Pedro Andrade, Ivanovitch Silva, Emiliano Sisinni, and Paolo Ferrari. "An evolving tinyml compression algorithm for iot environments based on data eccentricity." Sensors 21, no. 12 (2021): 4153.
de Prado, Miguel, Manuele Rusci, Alessandro Capotondi, Romain Donze, Luca Benini, and Nuria Pazos. "Robustifying the deployment of tinyml models for autonomous mini-vehicles." Sensors 21, no. 4 (2021): 1339.
David, Robert, Jared Duke, Advait Jain, Vijay Janapa Reddi, Nat Jeffries, Jian Li, Nick Kreeger et al. "Tensorflow lite micro: Embedded machine learning for tinyml systems." Proceedings of Machine Learning and Systems 3 (2021): 800-811.
Bringmann, Oliver, Wolfgang Ecker, Ingo Feldner, Adrian Frischknecht, Christoph Gerum, Timo Hämäläinen, Muhammad Abdullah Hanif et al. "Automated HW/SW co-design for edge AI: state, challenges and steps ahead." In Proceedings of the 2021 International Conference on Hardware/Software Codesign and System Synthesis, pp. 11-20. 2021.
Banbury, Colby, Chuteng Zhou, Igor Fedorov, Ramon Matas, Urmish Thakker, Dibakar Gope, Vijay Janapa Reddi, Matthew Mattina, and Paul Whatmough. "Micronets: Neural network architectures for deploying tinyml applications on commodity microcontrollers." Proceedings of Machine Learning and Systems 3 (2021): 517-532.
Reddi, Vijay Janapa, Brian Plancher, Susan Kennedy, Laurence Moroney, Pete Warden, Anant Agarwal, Colby Banbury et al. "Widening access to applied machine learning with tinyml." arXiv preprint arXiv:2106.04008 (2021).
Banbury, Colby R., Vijay Janapa Reddi, Max Lam, William Fu, Amin Fazel, Jeremy Holleman, Xinyuan Huang et al. "Benchmarking tinyml systems: Challenges and direction." arXiv preprint arXiv:2003.04821 (2020).
Surantha, Nico, Prabadinata Atmaja, and Maulana Wicaksono. "A review of wearable internet-of-things device for healthcare." Procedia Computer Science 179 (2021): 936-943.
Nuvvula, Sri, Eric Y. Ding, Connor Saleeba, Qiming Shi, Ziyue Wang, Alok Kapoor, Jane S. Saczynski et al. "NExUS-Heart: Novel examinations using smart technologies for heart health—Data sharing from commercial wearable devices and telehealth engagement in participants with or at risk of atrial fibrillation." Cardiovascular Digital Health Journal 2, no. 5 (2021): 256-263.
Sarivan, I-M., Johannes N. Greiner, D. Díez Álvarez, Felix Euteneuer, Matthias Reichenbach, Ole Madsen, and Simon Bøgh. "Enabling real-time quality inspection in smart manufacturing through wearable smart devices and deep learning." Procedia Manufacturing 51 (2020): 373-380.
S., E. ., & Kasturi, K. . (2023). Evaluation of Linkage Between the Corporate Economic Value-Added Analysis and Information Technology Using Big Data. International Journal of Intelligent Systems and Applications in Engineering, 11(4s), 109–117. Retrieved from https://ijisae.org/index.php/IJISAE/article/view/2577
Gopinath, Sridhar, Nikhil Ghanathe, Vivek Seshadri, and Rahul Sharma. "Compiling KB-sized machine learning models to tiny IoT devices." In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 79-95. 2019.
Pardamean, Bens, Haryono Soeparno, Bharuno Mahesworo, Arif Budiarto, and James Baurley. "Comparing the accuracy of multiple commercial wearable devices: a method." Procedia Computer Science 157 (2019): 567-572.
Godfrey, Alan, Victoria Hetherington, Hubert Shum, Paolo Bonato, N. H. Lovell, and Sam Stuart. "From A to Z: Wearable technology explained." Maturitas 113 (2018): 40-47.
Tan, Song Lim. "Posture detection of smartphone users using deep learning." PhD diss., UTAR, 2018.
Huynh-The, Thien, Cam-Hao Hua, Nguyen Anh Tu, and Dong-Seong Kim. "Physical activity recognition with statistical-deep fusion model using multiple sensory data for smart health." IEEE Internet of Things Journal 8, no. 3 (2020): 1533-1543.
Coffen, Brian, and Md Shaad Mahmud. "TinyDL: edge computing and deep learning based real-time hand gesture recognition using wearable sensor." In 2020 IEEE International Conference on E-health Networking, Application & Services (HEALTHCOM), pp. 1-6. IEEE, 2021.
Mekruksavanich, Sakorn, and Anuchit Jitpattanakul. "Sport-related activity recognition from wearable sensors using bidirectional gru network." Intelligent Automation & Soft Computing 34, no. 3 (2022): 1907-1925.
Bian, Sizhen, and Paul Lukowicz. "Capacitive sensing based on-board hand gesture recognition with TinyML." In Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers, pp. 4-5. 2021.
Islam, Md Milon, Sheikh Nooruddin, Fakhri Karray, and Ghulam Muhammad. "Human activity recognition using tools of convolutional neural networks: A state of the art review, data sets, challenges, and future prospects." Computers in Biology and Medicine (2022): 106060.
Ambati, Loknath Sai, and Omar El-Gayar. "Human activity recognition: a comparison of machine learning approaches." Journal of the Midwest Association for Information Systems (JMWAIS) 2021, no. 1 (2021): 4.
Jain, Rahul, Vijay Bhaskar Semwal, and Praveen Kaushik. "Deep ensemble learning approach for lower extremity activities recognition using wearable sensors." Expert Systems 39, no. 6 (2022): e12743.
Burq, Maximilien, and Niranjan Sridhar. "Human Activity Recognition Using Self-Supervised Representations of Wearable Data." arXiv preprint arXiv:2304.14912 (2023).
Ricotti, Valeria, Balasundaram Kadirvelu, Victoria Selby, Richard Festenstein, Eugenio Mercuri, Thomas Voit, and A. Aldo Faisal. "Wearable full-body motion tracking of activities of daily living predicts disease trajectory in Duchenne muscular dystrophy." Nature Medicine (2023): 1-9.
Issa, Mohamed E., Ahmed M. Helmi, Mohammed AA Al-Qaness, Abdelghani Dahou, Mohamed Abd Elaziz, and Robertas Damaševi?ius. "Human activity recognition based on embedded sensor data fusion for the internet of healthcare things." In Healthcare, vol. 10, no. 6, p. 1084. MDPI, 2022.
Alsareii, Saeed Ali, Mohsin Raza, Abdulrahman Manaa Alamri, Mansour Yousef AlAsmari, Muhammad Irfan, Hasan Raza, and Muhammad Awais. "Artificial Intelligence and Internet of Things Enabled Intelligent Framework for Active and Healthy Living." Computers, Materials and Continua 75, no. 2 (2023): 3833-3848.
Dowd, Kieran P., Deirdre M. Harrington, Alan K. Bourke, John Nelson, and Alan E. Donnelly. "The measurement of sedentary patterns and behaviors using the activPAL™ professional physical activity monitor." Physiological measurement 33, no. 11 (2012): 1887.
Ms. Mohini Dadhe, Ms. Sneha Miskin. (2015). Optimized Wireless Stethoscope Using Butterworth Filter. International Journal of New Practices in Management and Engineering, 4(03), 01 - 05. Retrieved from http://ijnpme.org/index.php/IJNPME/article/view/37
Schrader, Lisa, Agustín Vargas Toro, Sebastian Konietzny, Stefan Rüping, Barbara Schäpers, Martina Steinböck, Carmen Krewer, Friedemann Müller, Jörg Güttler, and Thomas Bock. "Advanced sensing and human activity recognition in early intervention and rehabilitation of elderly people." Journal of Population Ageing 13 (2020): 139-165.
Granat, Malcolm, Andreas Holtermann, and Kate Lyden. "Sensors for Human Physical Behaviour Monitoring." Sensors 23, no. 8 (2023): 4091.
Manimaran, M., A. Sasi Kumar, N. V. S. Natteshan, K. Baranitharan, R. Mahaveerakannan, and K. Sudhakar. "Detecting the Human Activities of Aging People using Restricted Boltzmann Machines with Deep Learning Technique in IoT." In 2023 Third International Conference on Artificial Intelligence and Smart Energy (ICAIS), pp. 105-110. IEEE, 2023.
Thu, Nguyen Thi, To-Hieu Dao, Bao Bo Quoc, Duc-Nghia Tran, Pham Van Thanh, and Duc-Tan Tran. "Real-time wearable-device based activity recognition using machine learning methods." International Journal of Computing and Digital Systems 11, no. 1 (2022): XXXX-XXXX.
Paraschiakos, Stylianos, Ricardo Cachucho, Matthijs Moed, Diana van Heemst, Simon Mooijaart, Eline P. Slagboom, Arno Knobbe, and Marian Beekman. "Activity recognition using wearable sensors for tracking the elderly." User Modeling and User-Adapted Interaction 30, no. 3 (2020): 567-605.
Jayasinghe, Udeni, Balazs Janko, Faustina Hwang, and William S. Harwin. "Classification of static postures with wearable sensors mounted on loose clothing." Scientific Reports 13, no. 1 (2023): 131.
Bonnet, Cédrick T., and Boris Cheval. "Sitting vs. standing: an urgent need to rebalance our world." Health Psychology Review (2022): 1-22.
Natalia Volkova, Machine Learning Approaches for Stock Market Prediction , Machine Learning Applications Conference Proceedings, Vol 2 2022.
Balli, Serkan, Ensar Arif Sa?ba?, and Musa Peker. "Human activity recognition from smart watch sensor data using a hybrid of principal component analysis and random forest algorithm." Measurement and Control 52, no. 1-2 (2019): 37-45.
Hung, Wei-Chih, Fan Shen, Yi-Leh Wu, Maw-Kae Hor, and Cheng-Yuan Tang. "Activity recognition with sensors on mobile devices." In 2014 International Conference on Machine Learning and Cybernetics, vol. 2, pp. 449-454. IEEE, 2014.
Ferreira, José M., Ivan Miguel Pires, Gonçalo Marques, Nuno M. Garcia, Eftim Zdravevski, Petre Lameski, Francisco Flórez-Revuelta, Susanna Spinsante, and Lina Xu. "Activities of daily living and environment recognition using mobile devices: a comparative study." Electronics 9, no. 1 (2020): 180.
Pires, Ivan Miguel, Nuno M. Garcia, Eftim Zdravevski, and Petre Lameski. "Activities of daily living with motion: A dataset with accelerometer, magnetometer and gyroscope data from mobile devices." Data in brief 33 (2020): 106628.
Vähä?Ypyä, H., P. Husu, J. Suni, T. Vasankari, and H. Sievänen. "Reliable recognition of lying, sitting, and standing with a hip?worn accelerometer." Scandinavian journal of medicine & science in sports 28, no. 3 (2018): 1092-1102.
Mekruksavanich, Sakorn, and Anuchit Jitpattanakul. "Deep convolutional neural network with rnns for complex activity recognition using wrist-worn wearable sensor data." Electronics 10, no. 14 (2021): 1685.
Gupta, Saurabh. "Deep learning based human activity recognition (HAR) using wearable sensor data." International Journal of Information Management Data Insights 1, no. 2 (2021): 100046.
Oluwalade, Bolu, Sunil Neela, Judy Wawira, Tobiloba Adejumo, and Saptarshi Purkayastha. "Human activity recognition using deep learning models on smartphones and smartwatches sensor data." arXiv preprint arXiv:2103.03836 (2021).
Mekruksavanich, Sakorn, Anuchit Jitpattanakul, Phichai Youplao, and Preecha Yupapin. "Enhanced hand-oriented activity recognition based on smartwatch sensor data using lstms." Symmetry 12, no. 9 (2020): 1570.
Tensorflow lite conversion, URL: https://www.tensorflow.org/lite/models/convert
Extrasensory dataset, URL: https://www.kaggle.com/datasets/yvaizman/the-extrasensory-dataset
Wisdm dataset - Weiss,Gary. (2019). WISDM Smartphone and Smartwatch Activity and Biometrics Dataset . UCI Machine Learning Repository. https://doi.org/10.24432/C5HK59