Automatic Patient Functionality Assessment from Multimodal Data using Deep Learning Techniques - Development and Feasibility Evaluation

Wearable devices and mobile sensors enable the real-time collection of an
abundant source of physiological and behavioural data unobtrusively. Unlike
traditional in-person evaluation or ecological momentary assessment (EMA)
questionnaire-based approaches, these data sources open many possibilities in
remote patient monitoring. However, defining robust models is challenging due to
the data's noisy and frequently missing observations. This work proposes an
attention-based Long Short-Term Memory (LSTM) neural network-based pipeline for
predicting mobility impairment based on WHODAS 2.0 evaluation from such digital
biomarkers. Furthermore, we addressed the missing observation problem by
utilising hidden Markov models and the possibility of including information from
unlabelled samples via transfer learning. We validated our approach using two
wearable/mobile sensor data sets collected in the wild and socio-demographic
information about the patients. Our results showed that in the WHODAS 2.0
mobility impairment prediction task, the proposed pipeline outperformed a prior
baseline while additionally providing interpretability with attention heatmaps.
Moreover, using a much smaller cohort via task transfer learning, the same model
could learn to predict generalised anxiety severity accurately based on GAD-7
scores.