This repository contains all relevant codes and materials prepared for our work, "PhysioBind: Task-Aware EEG-BVP Cross-Modality Representation Alignment for Multidimensional Cognitive Workload Assessment".
PhysioBind is a task-aware cross-modality representation alignment framework for aligning less privileged modalities (in this case, blood volume pulse or BVP) to more privileged supervisory modalities (in this case, electroencephalography or EEG), while ensuring more accurate estimation of cognitive workload. A trainable BVP encoder is aligned to the latent space of a frozen pretrained LaBraM EEG foundation model, enabling inference from BVP alone.
This framework was evaluated with the WAUC Dataset, an open multi-modal dataset collecting live EEG and BVP data from 48 participants while participating in the NASA MATB-II Task under three different physical activity-level conditions. The NASA Task Load Index questionnaire was administered after each experimental section, which we used as a baseline for cognitive workload estimation using subject-independent regression and classification.
Aligned BVP representations consistently outperform traditional BVP feature-based models and frequently surpass EEG-based approaches, particularly for Temporal, Performance, Effort, and Frustration demand. These results demonstrate that cross-modality alignment can yield robust and deployable workload representations without reliance on EEG at inference time.