SmartFall
About Our SmartFall
Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long term care facilities.
News
11-2021: Funded Ph.D. Positions available
01-2022: One Crossmodal KD paper has been accepted to ICASSP 2022.
06-2022: Texas State professor enhancing safety for older adults through technology.
07-2022: Progressive Cross-Modal Knowledge Distillation for Skeleton-to-sensor Human Action Recognition accepted to ACM MM 2022
06-2023: One Transformer-based GAN paper has been accepted to ICAI 2023.
01-2024: One Cross-architecture KD paper has been accepted to FG 2024.
02-2024: Dr. Ngu presented Fall Detection Technologies to over 50 seniors at Price Center, San Marcos
03-2024: Pole balancing on the fingertip: Model-motivated machine learning forecasting of falls accepted to Frontier in Physiology
03-2024: The Impact of Synthetic Data on Fall Detection Application accepted to AIME 2024 Conference at Utah
03-2024: An Empirical Study on AI-Powered Edge Computing Architectures for Real-Time IoT Applications accepted to COMPSAC 2024
04-2024: LightHART: A Ligght-weight Transformer for Human Activity Recognition has been accepted in ICPR in September 2024.
05:2024: SmartFallMM: An inter-generational multimodal Human Activity Dataset for Cross-Modal Learning submitted to IEEE perComp 2024
07-2024: Dr. Ngu presented AI-Powered Edge Computing Architecture at COMPSAC Conference in Japan in July 2024
08-2024: Dr. Ngu presented Fall Detection Technologies at the Aging in Texas Conference in Austin in August 2024
09-2024: Experimental Study of Long Short-Term Memory and Transformer Models for Fall Detection on Smartwatches published at sensors 24(19)
10-2024: SSDL:Sensor-to-Skeleton Diffusion Model with Lipschitz Regularization for Human Activity accepted by MMM 2025
Dataset
09-2021: Dataset for Fall Detection is avaliable here.
09-2021: Dataset for ADL from nine seniors participants is available here.
Project Members
Professor, TXST
Assisstant Professor, IIT
Assisstant Professor, UT Austin
Professor, CMC
Ph.D. student, TXST
Ph.D. student, TXST
Ph.D. student, TXST
M.S. student, TXST
M.S. student, TXST
B.S. student, TXST
Executive Director, LiveOak
Professor, TXST
Research Works
Adaptive Cross-architecture Mutual Knowledge Distillation
Ni J., Tang H., Shang Y., Duan B, Yan Y.
The 18th IEEE International Conference on Automatic Face and Gesture Recognition, Istanbul, Turkey, 27-31 May 2024.
Multimodal ConvTransformer for Human Activity Recognition
Haque S.T. and Ngu A. H.
IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies Proceedings (CHASE 2024), June 19-21, Wilmington, DE, USA.
The Impact of Synthetic Data on Fall Detection Application
Debnath M., Kabir Md. S., Ni J., Ngu A. H
22nd International Conference on Artificial Intelligence in Medicine (AIME 2024), July 9-12, Salt Lake City, Utah, USA
An Empirical Study on AI-Powered Edge Computing Architectures for Real-Time IoT Applications
Yasmin A., Mahmud T., Debnath M., Ngu A. H.
IEEE Computers, Software, and Applications Conference (COMPSAC 2024), July2-4, Osaka, Japan
Pole balancing on the fingertip: model-motivated machine learning forecasting of falls
Debnath, M., Chang, J., Bhandari, K., Nagy, D. J., Insperger, T., Milton, J. G., & Ngu, A. H.
Frontiers in physiology (2024)
P-Fall: Personalization Pipeline for Fall Detection on Wearable
Ngu A.H., Yasmin A., Mahmud T.,Mahmood A., Sheng Q.Z.
IEEE/ACM CHASE 2023.
Generating Realistic Multi-class Biosignals with BioSGAN: A Transformer-based Label-guided Generative Adversarial Network
Li X., Ngu. A.H., Metsis V.,
ICAI 2023.
Transfer Learning On Small Datasets for Improved Fall Detection
Maray, N., Ngu, A. H., Ni, J., Debnath, M., Wang, L.
Sensors 2022.
Personalized Watch-based Fall Detection Using a Collaborative Edge-Cloud Framework
Ngu, AH, Metsis V., Coyne S., Srinivas P., Mahmud T., Chee KH.,
IJNS 2022.
Progressive Cross-modal Knowledge Distillation for Human Action Recognition
Ni, J., Ngu, A. H., & Yan, Y.
ACM MM 2022.
TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network
Li, X., Metsis, V., Wang, H., & Ngu, A. H. H.
AIME 2022, Canada.
Cross-modal knowledge distillation for Vision-to-Sensor action recognition
Ni, J., Sarbajna, R., Liu, Y., Ngu, A. H., & Yan, Y.
ICASSP 2022,pp. 4448-4452
Collaborative Edge-Cloud Computing for Personalized Fall Detection
Ngu, A. H., Coyne, S., Srinivas, P., & Metsis, V.
AIAI 2021,pp. 323-336
SmartFall: A Smartwatch-Based Fall Detection System Using Deep Learning
Mauldin, T. R., Canby, M. E., Metsis, V., Ngu, A. H., & Rivera, C. C.
Sensors, 18(10), 3363.
Ensemble Deep Learning on Wearables Using Small Datasets
Mauldin, T., Ngu, A. H., Metsis, V., & Canby, M. E.
ACM Transactions on Computing for Healthcare, 2(1), 1-30.