About Our SmartFall

Falls are a significant cause of morbidity and mortality in the elderly. A robust and low-cost solution for the estimation of fall risk and detection of falls will allow seniors to live independently and reduce medical costs due to falls. Wearable devices have been developed to detect “hard falls”, namely falls that cause injury. However, many falls in the elderly do not cause physical injury (“soft falls”). These occur in association with weight transfer activities such as turning and sit-stand transitions. Indeed, the ability to control the position and movements of the trunk (“core”) is essential for coordinating the movements of the limbs during weight transfer. The goal of this project is to combine real-world limb-core dynamics of an individual with data collected by accelerometer via a commodity wristwatch and a cell phone on the opposite hip to improve the detection of hard and soft falls. A personalized fall risk analysis and detection model will be created for each user via real-time learning of the limb-core dynamics using state of the art machine learning algorithm. We will also assess the perceptions and preferences of elderly patients using this technology and evaluate their attitudes towards continuous data collection and sharing of health data for improved health. The software system, the real-world gait and weight transfer movement and the associated accelerometer data will be made freely available to any institution, investigator or research student interested in the study of machine learning on health conditions as well as on fall risk and analysis. This project will train graduate and undergraduate students in technical skills (machine learning, wearable technologies and data analysis skills) as well as in people skills for working with the elderly who live in long term care facilities.


09-2021: Dataset for Fall Detection is avaliable now.

09-2021: Dataset for ADL from nine seniors participants is avaliable now.

10-2021: One Crossmodal KD paper has been submitted to ICASSP 2022.

11-2021: Funded Ph.D. Positions available

01-2022: UPDATE: Crossmodal KD paper has been accepted to ICASSP 2022.

04-2022: One Progressive KD paper has been submitted to ACM MM 2022.

06-2022: Texas State professor enhancing safety for older adults through technology.

07-2022: UPDATE: Progressive KD paper has been accepted to ACM MM 2022.

Project Members

Dr.Anne H. H. Ngu
Professor, TXST
Dr.Yan Yan
Assisstant Professor, IIT
Dr.Joshua Chang
Assisstant Professor, UT Austin
Dr.John G. Milton
Professor, CMC
Keshav Bhandari
Ph.D. student, TXST
Jianyuan Ni
Ph.D. student, TXST
Minakshi Debnath
Ph.D. student, TXST
Rachel Medina
Executive Director, LiveOak
Dr.Kyong Hee Chee
Professor, TXST

Gaowen Liu, visiting during 09/2018-09/2020. Data Scientist, Cisco.

Xianjing Han, visiting during 09/2019-08/2020. Ph.D. Student, Shandong University.

Research Works

Progressive Cross-modal Knowledge Distillation for Human Action Recognition

Ni, J., Ngu, A. H., & Yan, Y.

To appear in ACM MM 2022.

TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network

Li, X., Metsis, V., Wang, H., & Ngu, A. H. H.

To appear in AIME 2022, Canada.

Cross-modal knowledge distillation for Vision-to-Sensor action recognition

Ni, J., Sarbajna, R., Liu, Y., Ngu, A. H., & Yan, Y.

ICASSP 2022,pp. 4448-4452

Collaborative Edge-Cloud Computing for Personalized Fall Detection

Ngu, A. H., Coyne, S., Srinivas, P., & Metsis, V.

AIAI 2021,pp. 323-336

SmartFall: A Smartwatch-Based Fall Detection System Using Deep Learning

Mauldin, T. R., Canby, M. E., Metsis, V., Ngu, A. H., & Rivera, C. C.

Sensors, 18(10), 3363.

Ensemble Deep Learning on Wearables Using Small Datasets

Mauldin, T., Ngu, A. H., Metsis, V., & Canby, M. E.

ACM Transactions on Computing for Healthcare, 2(1), 1-30.