The demo, exactly

Background

Get ready

Procedure

1. Decide privately

   Reading your own score is a choice, and nobody else will see it. If you would rather nobody saw you wear the IntoMind One, use the station out of the room's view, or a time outside class. The rest of the lab works without anyone's score.

2. Read it alone, if you choose

   Put the IntoMind One on at any site, connect to the Biological Sex Detector demo, and sit still for a couple of minutes. Save image saves a picture of your score, yours to keep or delete. Then close the tab.

   You should seeTwo clouds that overlap almost entirely, spaced as the published AUC implies, and your average score among them.

Analyses

import numpy as np
from scipy.stats import norm, rankdata

def auc(score, group):
    """The chance that a random member of group 1 outscores a random member of group 0."""
    r = rankdata(score)
    n1 = group.sum()
    return (r[group == 1].sum() - n1 * (n1 + 1) / 2) / (n1 * (len(group) - n1))

# 1. What the published AUCs imply, with two normal groups of equal spread.
d1, dm = (np.sqrt(2) * norm.ppf(a) for a in (0.588, 0.623))
m = 40                                   # the most windows averaged for one person
stable = (1 / (dm / d1) ** 2 - 1 / m) / (1 - 1 / m)
ceiling = d1 / np.sqrt(stable)
print(f"d' {d1:.2f} on one window, {dm:.2f} averaged")
print(f"the share of a window's score that is stable within a person: about {stable:.2f}")
print(f"averaging without limit: d' {ceiling:.2f}, AUC {norm.cdf(ceiling / np.sqrt(2)):.3f}")

# 2. A score that reads contact and nothing of the brain, when contact differs by group.
rng = np.random.default_rng(4)
n = 4000
group = rng.integers(0, 2, n)                     # the label the studies recorded
hair = rng.normal(0, 1, n) + 1.0 * group          # hair under the electrodes, more in group 1
contact = -0.8 * hair + rng.normal(0, 1, n)       # thicker hair, poorer contact
score = -0.6 * contact + rng.normal(0, 1, n)      # poorer contact, higher score: no brain term
proxy = contact + rng.normal(0, 0.5, n)           # a measured index of contact, with error
adjusted = score - np.polyval(np.polyfit(proxy, score, 1), proxy)
print(f"contact only: AUC {auc(score, group):.3f}, "
      f"after adjusting for the index {auc(adjusted, group):.3f}")

# 3. The same, with a real difference and contact that doesn't differ by group.
contact = rng.normal(0, 1, n)
score = 0.3 * group - 0.5 * contact + rng.normal(0, 1, n)
proxy = contact + rng.normal(0, 0.5, n)
adjusted = score - np.polyval(np.polyfit(proxy, score, 1), proxy)
print(f"a real difference: AUC {auc(score, group):.3f}, "
      f"after adjusting {auc(adjusted, group):.3f}")

Open problems

1. Brain, or body? How much of any EEG classifier's separation of the sexes comes from the brain rather than anatomy, hair, contact or muscle is not known, for this head or the deep network that found more.
2. Mosaics and classifiers. Whether brains are better described as two types or as mosaics depends on the question, and the debate Joel and others (2015) opened continues.
3. What the labels mean. Most datasets record two categories without saying how they were assigned. What a classifier learns from such labels, and for whom it fails, is open.

Written analysis