All science

Perspective

Reading the mind with EEG and fNIRS

Context

Non-invasive neural decoding has seen various approaches over the years, each to varying degrees of success but none achieving true viability for a consumer product. fMRI and MEG approaches are provenly good, yet impractical due to large machines and magnetic shielding requirements. EEG approaches often display flashes of brilliance, but inconsistencies plague model outputs due to the weak biological nature of the electrical signal.

Teal’s thesis revolves around a less-used data modality called fNIRS to supercharge our biological context, which uses infrared light to image blood flow and metabolic activity at much higher spatial resolutions.

Light is a good listener

Certain wavelengths of near-infrared light are primarily absorbed by hemoglobin, passing through most tissues and miscellaneous matter in the brain and head. Because intentional brain activity demands energy, tracing the movement of the blood’s oxygen carrier enables us to peek into the mind comfortably—without making any incisions.

scalpcortexsourcedetector
Near-infrared light enters at a source, curves through the cortex, and is read at a detector — sampling activity centimeters deep, with no incision.

Timing from electricity, place from light

EEG samples at an incredibly high rate, but smears across space because axonal electrical fields require mass coupling to reach the scalp’s surface. Optical fNIRS signals are slower, but image a more reliable byproduct of neural activity through a more consistent physical medium.

EEG — electrical
fNIRS — blood oxygen
Real recordings, not an illustration: both panels are one subject’s averaged response to seeing a picture of an animal, from OpenNeuro ds004514 — EEG and fNIRS recorded simultaneously from the same person. Brightness is response magnitude at each sensor. (fNIRS optodes here cover the left hemisphere only.)

It should be intuitive from here that neither signal, independently, provides enough information. Unfortunately, reading minds is pretty hard. Fused, however, EEG and fNIRS seem to cover each other’s blind spots. One tells us when, the other tells us where, and structure begins to emerge.

EEG
fNIRS weights
Fused
The same moment (animal imagery) for two different subjects. Processing fNIRS data into a trust score for each electrode (middle), we preserve every bit of the EEG’s temporal resolution (left), but constrain it to biological truths (right).

With our multimodal data pipeline, the random noise that plagues raw EEG data is replaced in favor of visibly clearer signals. Improving signal-to-noise ratio in this manner is a significant step forward, and it has propelled our efforts to develop a powerful, consumer-viable decoding model.

Looking forward

This is the first of many notes from the lab. We’ll share what we’re learning, what we’re getting wrong, and what we’re building towards as we bring Teal to life.


All science