Physiology and contextual interpretation

MultiEEG-GPT

Explores EEG with facial or speech-derived context in language-model classification benchmarks, without claiming clinical validation.

MultiEEG-GPT (UbiComp/ISWC ’24)

UbiComp/ISWC Companion · Exploratory study

EEG
large language models
mental health

Read the paper

MultiEEG-GPT system overview — MultiEEG-GPT overview. Study and evaluation in the linked paper.

The question

What can physiological measurements tell us when considered alongside other cues?

The approach

How the system works

This exploratory study evaluates zero-shot and few-shot language-model classification using EEG representations with available facial or speech-derived information. Audio-related inputs include transcripts and extracted features, not direct native audio processing.

Evidence & scope

What the study supports

The formal paper title is Exploring Large-Scale Language Models to Evaluate EEG-Based Multimodal Data for Mental Health, published in UbiComp/ISWC Companion 2024. MultiEEG-GPT is the project short name.

On MODMA with one-shot prompting, mean classification accuracy across five runs was 62.71% for EEG alone and 79.00% for EEG plus speech-derived context (Table 2). These are dataset-specific benchmark results, not clinical diagnostic accuracy.

The Home illustration is conceptual. It does not display recorded EEG, reproduce the benchmark, or infer a visitor’s emotions.

The broader direction.

The work motivates studying complementary evidence rather than treating one physiological measurement as a complete description of a person. Acknowledging uncertainty is part of the broader research agenda, not a claim that this prototype provides calibrated or clinically validated judgments.

The related IntervEEG-LLM framework explores how multimodal interpretations can inform supportive dialogue. Its example cases are not evidence of clinical effectiveness. Explore IntervEEG-LLM

Explore this research direction