Tonotopy

See the sound of the mind.

fMRI spatial evidence / EEG scalp + temporal response

HRA structure map × conservative auditory/sensorimotor audio-evidence prior · no learned regional localization · signal paths are an abstract surface encoding · HIGH
Playback
00:00.0NEURAL ESTIMATE
fMRI-informed spatial evidence DISPLAY APPROX · FUSION LOCKED
Audio-conditioned scalp proxy FAST TIMING
render — ms · inference timeline · — FPS · High · YouTube telemetry/inactive · analysis loading
Methods / evidence boundary

From open recordings to a browser brain

Tonotopy is a population-level music-response experiment, not a listener scan. We joined open anatomy, stimulus-aligned fMRI and EEG research, leakage-controlled modeling, and a real-time 3D renderer. The visual system distinguishes measured evidence, learned estimates, and presentation-only interpolation so that detail never silently becomes a scientific claim.

open dataalignment + QCheld-out modelsnull gatesbrowser display
01

A real anatomical coordinate system

The ghost brain begins with the open Human Reference Atlas Brain, Male v1.3, licensed CC BY 4.0 and derived from the Allen Human Reference Atlas and Visible Human references. Its 283 named structures were preserved as the anatomical vocabulary. For responsive WebGL rendering, the geometry was centered, uniformly scaled, merged into exterior and deep layers, simplified into multiple levels of detail, and Meshopt-compressed. The browser asset retains 72,839 triangles, 23,364 exterior vertices, 48 bounded activation zones, 1,400 curvature-aware surface points, and 2,030 mesh-following paths.

Those points and paths are an abstract surface encoding for motion and continuity. They are not neurons, white-matter tractography, or measured axons. Original structure names remain available for selection and explanation.

02

Music-aligned fMRI evidence

The first proof of concept used OpenNeuro ds003720 with its author-preprocessed derivative: five participants hearing 540 fifteen-second GTZAN excerpts. A deliberately small fast path fit eight latent response components on one participant, with entire tracks and runs held out. These latent channels prove the end-to-end timing and export path, but they do not have anatomical names.

For regional work we audited ds006583 and the joint EEG-fMRI dataset ds002725. BOLD series were motion-checked, censored, nuisance-regressed, baseline-centered, delayed for the hemodynamic response, and summarized into 32 bilateral atlas targets. Registration and population-reliability gates did not pass, so the current colored anatomy remains an evidence-constrained regional prior—not promoted learned voxel localization.

03

EEG timing and scalp topology

EEG was evaluated independently. Joint scanner EEG from ds002725 was aligned through music markers and scanner triggers, with gradient/pulse-artifact correction and participant-held-out correlated-component tests. Clean non-scanner ds004356 supplied exact clip-level 32-channel evaluation. MUSIN-G / ds003774 supplied a separate 20-participant, 12-song test of 16 scalp neighborhoods across theta, alpha, beta, and low-gamma power.

One continuous-EEG estimator passed its frozen scientific development and untouched-participant confirmation gates, but unclear model-weight and redistribution rights keep that result license-blocked and absent from the browser. The rights-clear EEG routes evaluated so far failed their scientific gates. The final DAAMEE Stage 10 development test was also RED: across 16 participants and 384 task trials, held population correlation was 0.006 while the strongest conservative null was 0.047. Learned EEG therefore remains gated off. The current outer field is an audio-conditioned scalp display proxy—not measured or predicted EEG power or voltage. Sensor positions use the DAAMEE v1.0.0 32-channel CapTrak geometry found byte-identically in all 16 development archives. It is therefore a dataset scalp template—not individualized placement, HRA MRI coregistration, cortical source localization, or a person-specific prediction.

04

One audio clock, two biological clocks

Every training example retains its dataset, participant, run, stimulus identity, exact time range, and preprocessing hashes. Audio becomes log-mel energy plus onset, flux, tonal-context, rhythm, and broad-band descriptors. fMRI stays on its slow native BOLD clock; EEG stays on its faster temporal/spectral clock. They meet only on a labeled 10 Hz display timebase. Interpolation smooths presentation but does not manufacture measurements or increase biological resolution.

Built-in playlist tracks use frozen, exact-hash audio analyses at 20 Hz for audio evidence only; they have no recording-bound model prediction or ground-truth response. A YouTube iframe does not expose audio samples, so an arbitrary pasted link is analyzed by a capped same-origin service before playback: audio is acquired transiently, converted into evidence and compact-model feature timelines, then deleted. The browser verifies the derived artifact and can run the unvalidated compact fMRI POC through ONNX before starting the iframe. That output remains a model rehearsal, not a validated regional prediction. No screen, tab, or microphone permission is requested.

05

Compact models with adversarial controls

Regularized temporal-response-function/ridge baselines were tested before compact causal temporal-convolution models. Splits were frozen before fitting; adjacent frames from the same clip or run never crossed train and test; scalers were fit on training data only. Evaluation used held-out participants, songs, clips, or runs plus time shifts, mismatched music, shuffled identities, mean and silence controls, and clustered permutation tests.

A modality can drive learned visuals only when matching music reliably beats its nulls. Fusion cannot rescue a failed fMRI or EEG head. The small fMRI POC was exported to ONNX and checked for numerical parity, but it failed its scientific validation gate; no regional fMRI head has passed promotion. The rights-clear DAAMEE EEG development head also failed its frozen held-participant, held-melody, task-transfer, and null gates, so confirmation stayed unopened and learned fusion remains LOCKED. That fail-closed boundary is why the display labels an approximation instead of overstating model accuracy.

06

Scientific constraints become visual grammar

fMRI/evidence chooses anatomical location; EEG supplies only external scalp topology and fast timing. Inactive anatomy stays near-transparent. Learned fMRI values can enter only through explicit named HRA crosswalk anchors, with smooth falloff confined inside each anchored structure; the browser does not invent cross-structure regional predictions. Pulses, fading paths, and surface particles are allowed only where the current regional field provides support. EEG-only mode keeps the brain ghosted specifically to avoid implying cortical source localization.

The browser runs inference in a Worker through ONNX Runtime Web/WASM and renders independently at the screen refresh rate. A 35 ms attack and 220 ms release make the EEG display fluid; these envelopes are choreography, not extra neural samples. Structure clicks expose the atlas name, functional context, display strength, evidence support, and the relevant limitation.

Compute

Practical independent implementation budget

$50–$80practical independent implementation allowance

A practical independent implementation budget on commodity marketplace compute is $50–$80: enough for several preprocessing attempts, data transfer and temporary storage, failed starts, compact model selection, ONNX export, and browser QA. This is a planning estimate—not money claimed as spent—and it excludes a future 10–12-hour multi-participant training run.