eeg2vision-multimodal-eeg-framework-2d-visual

name: eeg2vision-multimodal-eeg-framework-2d-visual description: > EEG2Vision — Modular end-to-end EEG-to-image reconstruction framework using diffusion models with MLLM-guided boosting. Evaluates performance across EEG resolutions (128/64/32/24 channels). Enables real-time brain-to-image applications with low-density EEG. Use when: EEG visual reconstruction, brain-to-image, diffusion models for EEG, multimodal LLM for neuroscience, low-density EEG decoding. Trigger: EEG to image, brain reconstruction, visual decoding EEG, diffusion EEG, EEG2Vision, 脑电图像重建, EEG视觉重建. version: 1.0.0 author: Research Synthesis (arXiv:2604.08063) license: MIT metadata: hermes: tags: [EEG, visual-reconstruction, diffusion, multimodal-LLM, brain-to-image, low-density-EEG] source_paper: "EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neuroscience (arXiv:2604.08063)"

EEG2Vision: EEG-to-Image Reconstruction with MLLM Boosting

Overview

EEG2Vision reconstructs 2D visual stimuli from non-invasive EEG signals using a two-stage pipeline:

1. EEG-conditioned diffusion model for initial reconstruction
2. MLLM-guided boosting for semantic refinement

Key innovation: Works with low-density EEG (as few as 24 channels), enabling real-world BCI applications.

Architecture

┌──────────────────────────────────────────────────────────┐
│  Stage 1: EEG-Conditioned Diffusion Reconstruction        │
│                                                          │
│  EEG (N channels) → Feature Extractor → Latent Condition │
│                                        ↓                  │
│                        ┌───────────────────────────┐     │
│                        │ Diffusion Model            │     │
│                        │ (EEG-conditioned generation)│    │
│                        └─────────────┬─────────────┘     │
│                                      ↓                    │
│                        Initial Reconstructed Image       │
└──────────────────────────────────────────────────────────┘
                          ↓
┌──────────────────────────────────────────────────────────┐
│  Stage 2: MLLM-Guided Boosting                           │
│                                                          │
│  Initial Image → MLLM → Semantic Description (prompt)   │
│                          ↓                                │
│  Initial Image + Semantic Prompt → I2I Diffusion        │
│                          ↓                                │
│                  Refined Image                           │
│  (improved geometry, perceptual coherence,               │
│   EEG-grounded structure preserved)                      │
└──────────────────────────────────────────────────────────┘

Channel Resolution Results

Key insight: Semantic accuracy drops sharply with fewer channels, but the boosting mechanism provides greater relative improvement in low-channel settings.

Implementation Pattern

class EEG2Vision:
    def __init__(self, eeg_encoder, diffusion_model, mllm, i2i_diffusion):
        self.eeg_encoder = eeg_encoder
        self.diffusion = diffusion_model
        self.mllm = mllm
        self.i2i_diffusion = i2i_diffusion
    
    def reconstruct(self, eeg_signal):
        # Stage 1: EEG-conditioned diffusion
        eeg_features = self.eeg_encoder(eeg_signal)
        initial_image = self.diffusion.generate(condition=eeg_features)
        
        # Stage 2: MLLM-guided boosting
        semantic_prompt = self.mllm.describe(initial_image)
        refined_image = self.i2i_diffusion.refine(
            initial_image, 
            prompt=semantic_prompt
        )
        return refined_image

Applications

• Real-time brain-to-image BCI
• Cognitive neuroscience research
• Low-cost EEG visualization
• Clinical neuroimaging applications
• Consumer-grade EEG device applications

Activation Keywords

• EEG to image, brain reconstruction, visual decoding
• diffusion models for EEG, MLLM boosting
• low-density EEG, brain-to-image, EEG2Vision
• EEG图像重建, 脑电视觉重建, 扩散模型

References

• Emanuele Balloni, Emanuele Frontoni, et al. "EEG2Vision: A Multimodal EEG-Based Framework for 2D Visual Reconstruction in Cognitive Neuroscience." arXiv:2604.08063