SmartFall

About Our SmartFall

Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long term care facilities.

News

09-2021: Dataset for Fall Detection is avaliable now.

09-2021: Dataset for ADL from nine seniors participants is available now.

10-2021: One Crossmodal KD paper has been submitted to ICASSP 2022.

11-2021: Funded Ph.D. Positions available

01-2022: UPDATE: Crossmodal KD paper has been accepted to ICASSP 2022.

04-2022: One Progressive KD paper has been submitted to ACM MM 2022.

06-2022: Texas State professor enhancing safety for older adults through technology.

07-2022: UPDATE: Progressive KD paper has been accepted to ACM MM 2022.

03-2023: One Cross-architecture Mutual KD paper has been submitted to ICCV 2023.

03-2023: One Self-KD paper has been submitted to ICCV 2023.

04-2023: One Cross-modal GANs for HAR paper has been submitted to ACM MM 2023.

06-2023: One Synthetic Data Generation for Fall Anticipation paper has been submitted to PLOS-ONE.

Project Members

Dr.Anne H. H. Ngu
Professor, TXST
Dr.Yan Yan
Assisstant Professor, IIT
Dr.Joshua Chang
Assisstant Professor, UT Austin
Dr.John G. Milton
Professor, CMC
Jianyuan Ni
Ph.D. student, TXST
Minakshi Debnath
Ph.D. student, TXST
Syed Tousiful Haque
Ph.D. student, TXST
Awatif Yasmin
M.S. student, TXST
Dhadkan Shrestha
M.S. student, TXST
Rachel Medina
Executive Director, LiveOak
Dr.Kyong Hee Chee
Professor, TXST

Keshav Bhandari, Ph.D. 2018-2022, First Employment: Senior Scientist at Tesla.

Nader Maray, M.S. 2021-2023.

Research Works

P-Fall: Personalization Pipeline for Fall Detection on Wearable

Ngu A.H., Yasmin A., Mahmud T.,Mahmood A., Sheng Q.Z.

IEEE/ACM CHASE 2023.

Generating Realistic Multi-class Biosignals with BioSGAN: A Transformer-based Label-guided Generative Adversarial Network

Li X., Ngu. A.H., Metsis V.,

ICAI 2023.

Transfer Learning On Small Datasets for Improved Fall Detection

Maray, N., Ngu, A. H., Ni, J., Debnath, M., Wang, L.

Sensors 2022.

Personalized Watch-based Fall Detection Using a Collaborative Edge-Cloud Framework

Ngu, AH, Metsis V., Coyne S., Srinivas P., Mahmud T., Chee KH.,

IJNS 2022.

Progressive Cross-modal Knowledge Distillation for Human Action Recognition

Ni, J., Ngu, A. H., & Yan, Y.

ACM MM 2022.

TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network

Li, X., Metsis, V., Wang, H., & Ngu, A. H. H.

AIME 2022, Canada.

Cross-modal knowledge distillation for Vision-to-Sensor action recognition

Ni, J., Sarbajna, R., Liu, Y., Ngu, A. H., & Yan, Y.

ICASSP 2022,pp. 4448-4452

Collaborative Edge-Cloud Computing for Personalized Fall Detection

Ngu, A. H., Coyne, S., Srinivas, P., & Metsis, V.

AIAI 2021,pp. 323-336

SmartFall: A Smartwatch-Based Fall Detection System Using Deep Learning

Mauldin, T. R., Canby, M. E., Metsis, V., Ngu, A. H., & Rivera, C. C.

Sensors, 18(10), 3363.

Ensemble Deep Learning on Wearables Using Small Datasets

Mauldin, T., Ngu, A. H., Metsis, V., & Canby, M. E.

ACM Transactions on Computing for Healthcare, 2(1), 1-30.