Skip to yearly menu bar Skip to main content


Spotlight Poster

Flex-MoE: Addressing Missing Modalities in Alzheimer’s Disease with Flexible Mixture-of-Experts

Sukwon Yun · Inyoung Choi · Jie Peng · Yangfan Wu · Jingxuan Bao · Qiyiwen Zhang · Jiayi Xin · Qi Long · Tianlong Chen

[ ]
Fri 13 Dec 11 a.m. PST — 2 p.m. PST

Abstract:

Recent advancements in Alzheimer’s Disease (AD) research, propelled by machine learning innovations, have significantly pushed the frontier of the field. Critically, while multimodal approaches are essential in the AD domain, existing works primarily focus on single modalities such as imaging, genomics, or clinical data. Few recent studies attempt to integrate multiple modalities, and the lack of robust methods to handle missing modalities often limits the usability of such integrative approaches, forcing reliance on samples where the target modalities are available. Addressing this gap, we propose a novel multimodal learning framework, Flex-MoE (Flexible Mixture-of-Experts), which is flexible in incorporating diverse modalities in AD research while being resilient to missing modality scenarios through an advanced Sparse MoE design. The core idea of Flex-MoE is to handle samples with diverse modality combinations by sorting them based on the number of available modalities and passing them through modality-specific encoders. For missing embeddings, a missing modality bank with learnable embeddings is introduced. These embeddings, along with observed modality combinations, are processed through a Transformer with a Sparse MoE layer. Flex-MoE first trains experts with full modality samples to inject generalized knowledge and uses a newly designed S-Router to specialize expert knowledge for fewer modality combinations, fixing the top-1 gate as the corresponding observed modality combination expert. By evaluating Flex-MoE on the gold-standard ADNI dataset in the AD stage prediction task, we demonstrate its effectiveness in handling diverse missing modality scenarios. The source code of this study can be found here: https: //anonymous.4open.science/r/flex-mixture-of-experts-E074/.

Live content is unavailable. Log in and register to view live content