Multimodal analysis for assessment of mental health

Problem Statement: The importance of mental health has increased significantly over the past decade. However, the methods for the assessment of mental health issues at early stages are still in their infancy compared to the availability of corresponding methods for early assessment of physical health issues. Hence, it is required that due research is done to develop methods for early assessment of abnormalities leading to mental health problems.

Issues with Current Solutions: Unlike physical health parameters, the mental health is assessed through a number of subjective parameters. Hence, there is lack of objective and quantitative methods for mental health assessments. In addition, the patients seek help when their mental health problem is at advanced stage. So, there is lack of continuous monitoring for mental health issues.

Challenges: Many of the abnormalities related to mental health issues are subtle in nature and are related to behavior and other changes in facial expressions, speech and handwriting. In addition, there are changes in cortisol levels, skin conductance, heart rate variability and breathing rate. Hence, there are multiple modalities that should be included for measuring and quantifying any abnormalities related to mental health.

Solution: Every modality has its pros and cons. For example, in neuroimaging, functional magnetic resonance imaging has high spatial resolution (in mm) and low temporal resolution (in seconds) while electroencephalogram has low spatial resolution (in cm) and high temporal resolution (in milliseconds). Combining both of them will result in high spatial as well as high temporal resolution. This research deals with the assessment of abnormalities leading to mental health problems by utilizing multimodal approach. The various modalities may include, but not limited to, electroencephalogram (EEG) brain signals, facial videos, speech audios, handwriting and text from social media. The physiological parameters from various modalities include, but not limited to, the heart rate, breathing rate, dominant emotion, fatigue and stress. Dominant emotion can be classified as positive or negative and then sub-classified as sad, happy, angry etc. Data mining and data fusion techniques will be developed for this multimodal analysis. The corresponding multimodal data is available for this project.

Few Words About Supervision: I have extensive experience of working in the field of neuro-signal and neuroimage processing and I am currently head of a research group in this area. This is a multidisciplinary project and it will involve working with clinicians. However, the core of the project is related to IT in terms of development of a new method. Please feel free to contact me at malik@fit.vutbr.cz