Computer Science Department , MS Thesis Presentation, Reza Saadati Faard "Multimodal Neural Networks for Chronic Wound Decision Support" Faard

Reza Saadati Fard 

PhD 90 student

WPI – Computer Science Department 

Tuesday, August 20, 2024

Time: 10:00 AM – 11:00 AM 

Location: Fuller Labs 141 

Advisor: Prof. Emmanuel Agu 

Reader: Prof. Daniel Reichman 

Abstract:

Chronic wounds, which affect up to 10.5 people in the US, and cost $28 to $32 billion annually, often require an extended healing period.  Regular assessment and effective management are necessary to promote healing process and minimize expenses and adverse outcomes like limb amputations. However, many patients are treated in their homes and may receive inconsistent and non-standardized care. 

Whether a patient should be referred to see experts is a major decision made by visiting nurses, which could result in unnecessary amputations if they get it wrong. In this master’s thesis, we propose a Deep Multimodal Wound Assessment Tool (DM-WAT), a novel machine learning algorithm that analyzes multimodal data (smartphone wound images and clinical notes in the patient’s Electronic Health Record (EHR)), to generate recommendations on whether the patient should be referred to the clinic to see a wound expert. BERT-based models, which are state-of-the-art neural networks models for analyzing clinical notes, are employed to extract textual features from clinical notes.

 Pre-trained image classification models are utilized to extract visual features from corresponding wound images. These textual and visual features are then fused using the intermediate fusion algorithm. To overcome the scarcity of labeled data, image augmentation algorithms are employed to achieve high performance even on relatively small datasets. Rigorous, systematic evaluation is utilized including comparing DM-WAT to state-of-the-art baselines, using a dataset of 205 wound images and clinical notes.