Modality-Agnostic Style Transfer for Holistic Feature Imputation

Seunghun Baek1*, Jaeyoon Sim1*, Mustafa Dere2, Minjeong Kim3, Guorong Wu2, Won Hwa Kim1

(* : Equal Contribution)

1POSTECH, 2UNC-Chapel Hill 3UNC-Greensboroz

ISBI 2024 (Oral)

:greece: Athens, Greece

Paper

An overview of our framework. In Content Extraction, Modality-agnostic embedding is extracted from any type of feature for the subject. In Style Injection, modality-specific generators can generate missing features not present in the original subject. Shape: imaging scan (i.e., domain), Color: AD-stage label (i.e., class)

Illustraion of our framework.

Abstract

Characterizing a preclinical stage of Alzheimer’s Disease (AD), via single imaging is difficult as its early symptoms are quite subtle. Therefore, many neuroimaging studies are curated with various imaging modalities, e.g., MRI and PET, however, it is often challenging to acquire all of them from all subjects and missing data become inevitable. In this regards, in this paper, we propose a framework that generates unobserved imaging measures for specific subjects using their existing measures, thereby reducing the need for additional examinations. Our framework transfers modality-specific style while preserving AD-specific content. This is done by domain adversarial training that preserves modality-agnostic but AD-specific information, while a generative adversarial network adds an indistinguishable modality-specific style. Our proposed framework is evaluated on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) study and compared with other imputation methods in terms of generated data quality. Small average Cohen’s d < 0.19 between our generated measures and real ones suggests that the synthetic data are practically usable regardless of their modality type.

Quantative Results

Averaged Cohen's d

Table: The average absolute Cohen's d across all the ROIs between actual and generated distributions. Lower values are better, and the values ≥ 0.5 are in bold.

Classification Performance

Table: Classification performances (CN / EMCI / LMCI) on ADNI data with all modalities. The number of generators required are given in ().

Visualization of the averaged absolute Cohend's d

Figure: Visualization of the averaged absolute Cohen's d between actual distribution and generated distribution on the inner left cortical regions of EMCI subjects. AD-specific regions show better imputation results as lower Cohen's d implices higher correspondence. (Row: Source, Column: Target.)

Conclusion

In this paper, we propose a novel framework that generates unobserved imaging measures for specific subjects using their existing measures. To reduce the needs for taking several imaging scans, our framework addresses the imputation for missing measures by transferring modality-specific style while preserving AD-specific content. Experimental results on the ADNI study show that our model provides a probable estimation of target modality for individual subjects, which yields similar distributions of generated measurements to those from observed data and helps downstream analyses. Since our work is applicable regardless of modality type, our approach has potential to be adopted by other neuroimaging studies that are limited by missing measures.

BibTeX

@article{baek2024modality,
  author    = {Seunghun Baek and Jaeyoon Sim and Mustafa Dere and Minjeong Kim and Guorong Wu and Won Hwa Kim},
  title     = {Modality-agnostic Style Transfer for Holistic Feature Imputation},
  journal   = {ISBI},
  year      = {2024},
}