Analyses
First, reproduce a head from its training data with the demos' Python program, and check it matches the one saved, byte for byte. Then, for each wearer: each session's held-out score against its 70 relabelings with the demo's penalty and with a heavier one that also uses no labels, session 1's head on session 2 with and without each session's mean removed, and how far the sessions moved compared with the states.
cd python
python3 teach_it.py --from w1-session1.json --out reproduced.imh
cmp reproduced.imh w1-session1.imh && echo identical
import json
from itertools import combinations
import numpy as np
def load(path):
"""A training file the demo saved: embeddings as integers, labels, and rounds."""
w = json.load(open(path))["windows"]
X = np.array([v["embedding"] for v in w], float)
return X, np.array([v["label"] for v in w], float), np.array([v["round"] for v in w])
def penalty(X, rule):
"""Both use no labels: "gap" is the demo's, "mean" the Gram matrix's mean eigenvalue."""
ev = np.linalg.eigvalsh((X - X.mean(0)).T @ (X - X.mean(0)))
return np.sqrt(max(ev[0], 1) * ev[1]) if rule == "gap" else ev.mean()
def fit(X, y, rule):
"""Ridge with the intercept unpenalized: the weights and the intercept."""
m = X.mean(0)
xc = X - m
lam = penalty(X, rule)
w = np.linalg.solve(xc.T @ xc + lam * np.eye(X.shape[1]), xc.T @ (y - y.mean()))
return w, y.mean() - m @ w
def held_out(X, y, r, rule):
"""Readings right with each round held out in turn."""
right = 0
for k in np.unique(r):
w, b = fit(X[r != k], y[r != k], rule)
right += np.sum((X[r == k] @ w + b > 0.5) == (y[r == k] == 1))
return right
def relabelings(y, r):
"""Every labeling of the rounds with as many On as were taught: 70 for 4 of 8."""
rounds = np.unique(r)
on = sum(y[r == k][0] for k in rounds)
return [np.isin(r, c).astype(float) for c in combinations(rounds, int(on))]
s1, s2 = load("w1-session1.json"), load("w1-session2.json") # the files you saved
for rule in ("gap", "mean"):
for name, (X, y, r) in (("session 1", s1), ("session 2", s2)):
real = held_out(X, y, r, rule)
p = np.mean([held_out(X, z, r, rule) >= real for z in relabelings(y, r)])
print(f"{rule:4s} {name}: {real} of {len(y)} on held-out rounds, p {p:.3f}")
# Transfer: the head from all of session 1, on all of session 2, against its relabelings.
for recenter in (False, True):
(X1, y1, _), (X2, y2, r2) = s1, s2
if recenter: # each session's own mean, which uses no labels
X1, X2 = X1 - X1.mean(0), X2 - X2.mean(0)
w, b = fit(X1, y1, rule)
hit = lambda z: np.sum((X2 @ w + b > 0.5) == (z == 1))
p = np.mean([hit(z) >= hit(y2) for z in relabelings(y2, r2)])
print(f"{rule:4s} session 1's head on session 2{', recentered' if recenter else ''}: "
f"{hit(y2)} of {len(y2)}, p {p:.3f}")
(X1, y1, _), (X2, y2, _) = s1, s2
shift = np.linalg.norm(X2.mean(0) - X1.mean(0))
states = np.linalg.norm(X1[y1 == 1].mean(0) - X1[y1 == 0].mean(0))
print(f"the sessions' means are {shift:.0f} apart, "
f"the two states' means within session 1 {states:.0f}")
Then report:
- Each session's score and p with both penalties, and why a score below 44 of 88 can still have a small p.
- The transfer, raw and recentered, against each session's own held-out score.
- The probes and the mirror site: what the head responds to besides the state.
- For the class: which pairs and sites held up on a second day.
Sources
Pfurtscheller, G., and Lopes da Silva, F. H. (1999). Event-related EEG/MEG synchronization and desynchronization: basic principles. Clinical Neurophysiology.
Shenoy, P., Krauledat, M., Blankertz, B., Rao, R. P. N., and Müller, K.-R. (2006). Towards adaptive classification for BCI. Journal of Neural Engineering.
Vidaurre, C., and Blankertz, B. (2010). Towards a cure for BCI illiteracy. Brain Topography.
Zanini, P., Congedo, M., Jutten, C., Said, S., and Berthoumieu, Y. (2018). Transfer learning: a Riemannian geometry framework with applications to brain-computer interfaces. IEEE Transactions on Biomedical Engineering.
Thompson, M. C. (2019). Critiquing the concept of BCI illiteracy. Science and Engineering Ethics.
Panachakel, J. T., and Ramakrishnan, A. G. (2021). Decoding covert speech from EEG: a comprehensive review. Frontiers in Neuroscience.
