Skip to yearly menu bar Skip to main content


Poster
in
Workshop: AI for Accelerated Materials Design (AI4Mat-2023)

Multi-modal Foundation Model for Material Design

Seiji Takeda · Indra Priyadarsini S · Akihiro Kishimoto · Hajime Shinohara · Lisa Hamada · Hirose Masataka · Junta Fuchiwaki · Daiju Nakano

Keywords: [ Deep Learning ] [ muti modality ] [ generative model ] [ Foundation Model ] [ property prediction ] [ deep learning ] [ foundation model ]


Abstract:

We propose a multi-modal foundation model for small molecules, a shift from traditional AI models that are tailored for individual tasks and modalities. This model uses a late fusion strategy to align and fuse three distinct modalities: SELFIES, DFT properties, and optical spectrum. The model is pre-trained with over 6 billion samples to provide two primary functions, generating fused feature representations across the three modalities, and cross-modal predictions and genrations. As preliminary experiments, we demonstrate that the fused representation successfully improves the performance of property predictions for chromophore molecules, and showcase 6 distinct cross-modal inferences.

Chat is not available.