This chapter is part of the book from high school on.
Chapter 7 · High school
The IntoMind One carries an AI model that sums up four seconds of your EEG as 76 numbers. What can small models read from those numbers, and how do you know?
The IntoMind One carries a small AI model on its chip, called an encoder. Every four seconds, it reads the EEG from all four channels and sums it up as 76 numbers, called an embedding.
Think of an embedding as a description. Two stretches of EEG that are alike get descriptions that are alike: lists of numbers close to each other. The encoder learned to describe EEG this way from the recordings of many people, in many studies.
Try it
Your group picks five short songs, and describes each with the same five numbers, such as its speed, loudness and number of instruments, without naming it. Swap descriptions with another group. Can they match each list of numbers to its song?
| Song | Number 1 | Number 2 | Number 3 | Number 4 | Number 5 |
|---|---|---|---|---|---|
| A | |||||
| B | |||||
| C | |||||
| D | |||||
| E |
You should seeSome songs matched easily, and some confused, because their numbers were alike: an embedding, and its limits.
A head is a much smaller model that reads the 76 numbers and gives an answer, such as an age, or how much a reading looks like your On state. Heads run on the IntoMind One's chip too, so the EEG doesn't have to leave the device for a head to answer.
The IntoMind One comes with two open heads, for age and for biological sex. The custom AI head you teach in the Mind-Controlled Light Switch is a third.
The open heads were tested on whole studies they had never seen. Averaging a person's readings, the age head was off by 9.16 years on average. Two much simpler methods were off by 10.02 years, using alpha and the other bands' power, and by 9.77, always guessing the average age.
Their published results say that the heads do about as well as much simpler methods, and that neither is a useful predictor of a person's age or sex. They are published to show what the embedding carries.
Try it
The biological sex head's published score, its AUC, was 0.623, where 0.5 is guessing and 1 is perfect. Band power's was 0.614. From all of these numbers, write what each head can do, and what it can't.
Work it outThe age head can , and can't . The sex head can , and can't .
What is an embedding, in your own words?
Where do the encoder and the heads run, and why does that matter for privacy?
The age head does about as well as always guessing the average age of the studies it learned from. Why would anyone publish it?
IntoMind (2026). Heads for encoder e2f9b60f411f0e73. github.com/intomind/eeg-foundation-model-heads.
Chapter 7 · College
What does the network on the IntoMind One's chip keep of four seconds of EEG, what can a 156-byte head read from it, and how do you read a published evaluation as a specification?
The IntoMind One runs a neural network on its Bluetooth chip, an nRF52840 at 64 megahertz with 256 kilobytes of memory. This encoder has 170,772 parameters, 76 wide and 3 layers deep. It was distilled, with no labels, from a larger encoder pretrained on public EEG.
It reads four-second windows at 500 samples a second, without needing to know where the electrodes are, and describes each with 76 numbers: its embedding. It also describes each fifth of a second with a token. Its weights stay on the device and aren't published. Everything it sends is open.
A head is linear: weights and a bias from the 76 numbers to each output. For the chip's integer arithmetic its weights are quantized: scaled so the largest becomes 127, and rounded to signed bytes. A one-output head is 156 bytes.
Each head names the encoder it was fitted beside, by its ID, since its weights mean something only on that encoder's 76 numbers. A head is weights, not code: a device never runs code it is sent. Age is built into the model, sex ships in slot 1, and the Mind-Controlled Light Switch puts custom AI heads in slot 2.
Try it
From the top of the demos' source, fit a head on the trainer's test data with the trainer's λ, round it to bytes, and measure how far rounding moves the scores. Then do the same with λ = 1.
import json
import numpy as np
X = np.array(json.load(open("tests/fixtures/train/embeddings.json")), float)
y = np.array(json.load(open("tests/fixtures/train/targets.json")), float)
A = np.c_[X, np.ones(len(X))]
def head(lam):
"""Ridge on the weights, none on the intercept, then rounded to signed bytes."""
P = lam * np.eye(A.shape[1])
P[-1, -1] = 0
w = np.linalg.solve(A.T @ A + P, A.T @ y)
scale = np.abs(w[:-1]).max() / 127 # the largest weight becomes 127
q = np.round(w[:-1] / scale)
b = np.round(w[-1] / scale)
return w, q, b, scale
for lam in (json.load(open("tests/fixtures/train/ridge.json"))["ridge"], 1.0):
w, q, b, scale = head(lam)
exact = A @ w
rounded = (X @ q + b) * scale
print(f"λ {lam:.0f}: rounding moves a score by up to {np.abs(exact - rounded).max():.3f},",
f"and changes {np.sum((exact > 0.5) != (rounded > 0.5))} of 150 decisions")You should seeWith the trainer's λ, rounding moves a score by less than 0.01. With λ = 1, by about 0.17, though here no decision flips.
An open reconstruction head turns each token back into the fifth of a second it came from. On held-out windows from studies used neither to train the encoder nor to fit the head, it recovers about two thirds of the variance, R² = 0.660, but not evenly: most of the slow rhythms, less from 13 to 30 hertz, and almost nothing above 30.
Windows with gross artifacts, 3.9 percent of them, can't be recovered in amplitude and aren't in these numbers. So a head on the embedding sees alpha well, and a 15 hertz flicker response poorly: the BCI Flight Simulator reads the signal itself.
The age head was fitted on 183,642 windows from 3,742 people in 93 studies, and scored on each study left out in turn, with the electrode positions hidden. Averaged over a person's windows, it misses by 9.16 years, against 10.02 for band power fitted the same way and 9.77 for always guessing the training studies' mean age.
The sex head's AUC is 0.623, against 0.614 for band power. The evaluation says it plainly: neither head is a useful predictor of a person's age or sex, and neither is offered as one. They show what the embedding carries.
Try it
From the published evaluation, fill in what each is, so someone could decide whether to use it.
| Takes | Gives | Tested on | Against what | For | |
|---|---|---|---|---|---|
| The encoder | |||||
| The age head | |||||
| The sex head | |||||
| The reconstruction head |
Why must a head name the encoder it was fitted beside?
Could a head on the embedding detect a 15 hertz flicker response well? Use the reconstruction table.
Where do the encoder and the heads run, and why does that matter for privacy?
The age head is 0.61 years better than the mean age, with a 95 percent interval from 0.03 worse to 1.25 better. What can you conclude, and why would anyone publish it?
IntoMind (2026). Heads for encoder e2f9b60f411f0e73. github.com/intomind/eeg-foundation-model-heads.
Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv.
Jacob, B., and others (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Chapter 7 · Graduate
What does a small pretrained encoder on the device keep of four seconds of EEG, how much data does a linear head on it need, and what does running it in integers cost?
Labeled EEG is scarce and unlabeled EEG plentiful, so encoders are pretrained by self-supervision: predicting masked parts of the signal, or telling a window's own context from others', as BENDR did with a contrastive task on a large archive of clinical recordings (Kostas, Aroca-Ouellette and Rudzicz, 2021). Larger models followed, turning EEG into tokens and predicting masked ones (Jiang, Zhao and Lu, 2024). A model too large for a device can be distilled into a small one trained to reproduce its outputs (Hinton, Vinyals and Dean, 2015).
The IntoMind One's encoder has 170,772 parameters, 76 wide and 3 layers deep, distilled with no labels from a larger encoder pretrained on public EEG. It reads four-second windows at 500 samples a second, from its own copy of the stream, and doesn't need to know where the electrodes are. It describes each channel's every fifth of a second with a token, 80 tokens for a window of four channels, and the embedding is their mean: 76 numbers for each window. It runs on an nRF52840 at 64 megahertz with 256 kilobytes of memory, by default on every window, a new one every four seconds, without ever costing the stream a sample. Its weights stay on the device and aren't published.
A head is linear, up to 32 outputs, and runs in integers. The embedding reaches it as 16-bit integers, each value times 4,096, rounded and saturated. Each output's weights are signed bytes, scaled so the largest is 127, with a 32-bit bias in the same units, and the output is
y = s (b + Σi wi ei)
exact in the integers, then scaled. A one-output head is 156 bytes: a 40-byte header with its shape, a name and the encoder's identity, 76 weights, the bias, the scale, and a 32-byte hash. The device checks the hash and the shape, and never who made the head. Each head names its encoder, since its weights mean something only on that encoder's embedding.
A linear head trained to predict a property is a probe: its held-out accuracy measures how much of the property is linearly decodable from the representation, with that much data (Alain and Bengio, 2016). A probe that fails on 88 windows says as much about 88 windows as about the embedding. A probe that succeeds still needs a baseline that skips the encoder, such as band power fitted the same way, before the encoder gets the credit.
The published evaluation does exactly that. On studies the heads never saw, neither the age head nor the sex head beats band power on one window, and the age head is not clearly better than the training studies' mean age. An open reconstruction head shows what the tokens keep: on held-out windows without gross artifacts, R² of 0.871 from 1 to 4 hertz, 0.665 from 4 to 8, 0.728 from 8 to 13, 0.421 from 13 to 30 and 0.122 from 30 to 45. A head on the embedding sees slow rhythms and alpha well, and a 15 hertz flicker or anything faster poorly.
Analysis
On the trainer's fixture, train heads on random sets of 77 to 120 windows and score them on 30 held out, with the demo's penalty and with a heavier one that also uses no labels. Then round both heads to bytes, as the device runs them, and measure what rounding changes.
import json
import numpy as np
F = "tests/fixtures/train/" # in the demos' source
X = np.array(json.load(open(F + "embeddings.json")), float) # 150 windows of 76, as integers
y = np.array(json.load(open(F + "targets.json")), float)
rng = np.random.default_rng(7)
def penalty(Xt, rule):
"""Label-free penalties: the demo's gap ridge, or the Gram matrix's mean eigenvalue."""
ev = np.linalg.eigvalsh((Xt - Xt.mean(0)).T @ (Xt - Xt.mean(0)))
return np.sqrt(max(ev[0], 1) * ev[1]) if rule == "gap" else ev.mean()
def fit(Xt, yt, lam):
m = Xt.mean(0)
xc = Xt - m
w = np.linalg.solve(xc.T @ xc + lam * np.eye(X.shape[1]), xc.T @ (yt - yt.mean()))
return w, yt.mean() - m @ w
def quantized(w, b):
"""The head as the device runs it: weights to signed bytes, the bias in the same units."""
scale = np.abs(w).max() / 127
return np.clip(np.rint(w / scale), -127, 127), np.rint(b / scale), scale
# 1. A learning curve: held-out accuracy against the number of training windows.
for n in (77, 80, 88, 100, 120): # more windows than weights, always
acc = {"gap": [], "mean": []}
for _ in range(200):
order = rng.permutation(len(y))
train, test = order[:n], order[120:] # the last 30 of each shuffle are held out
for rule in acc:
w, b = fit(X[train], y[train], penalty(X[train], rule))
acc[rule].append(np.mean((X[test] @ w + b > 0.5) == (y[test] == 1)))
print(f"{n:3d} windows: gap ridge {np.mean(acc['gap']):.2f}, "
f"mean eigenvalue {np.mean(acc['mean']):.2f}")
# 2. What rounding to bytes costs, at each penalty, on all 150 windows.
for rule in ("gap", "mean"):
w, b = fit(X, y, penalty(X, rule))
q, qb, scale = quantized(w, b)
exact, device = X @ w + b, scale * (X @ q + qb)
print(f"{rule}: rounding moves scores by up to {np.abs(exact - device).max():.4f}, "
f"and flips {np.sum((exact > 0.5) != (device > 0.5))} of 150 decisions")You should seeWith the demo's penalty, held-out accuracy of 0.55 at 77 windows, 0.65 at 88 and 0.75 at 120. With the heavier penalty, 0.70 to 0.73 throughout. Rounding moves a score by less than 0.01 at either penalty, and flips no decision.
Then report: why the demo's penalty is worst just above 76 windows, and what you would tell someone choosing how long to teach a head.
Account for every byte of a one-output head on a 76-number embedding.
Bound the error that rounding the weights adds to one output, and say why the actual error is far smaller.
The embedding is the mean of a window's tokens. What can a head on it not tell apart?
Why is the demo's penalty at its worst just above 76 windows, and fine at 120?
A colleague trains a head on 80 windows of two states and reports 95 percent accuracy on them. What do you say?
Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv.
Alain, G., and Bengio, Y. (2016). Understanding intermediate layers using linear classifier probes. arXiv.
Jacob, B., and others (2018). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
Kostas, D., Aroca-Ouellette, S., and Rudzicz, F. (2021). BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data. Frontiers in Human Neuroscience.
Jiang, W.-B., Zhao, L.-M., and Lu, B.-L. (2024). Large brain model for learning generic representations with tremendous EEG data in BCI. International Conference on Learning Representations.
IntoMind (2026). Heads for encoder e2f9b60f411f0e73. github.com/intomind/eeg-foundation-model-heads.
Part 2 · Bits · Chapter 7
This topic starts at high school. In middle school, the Age Detector lab's background covers what students need.