See the music: a notebook on realtime interactive audio visualization
2026
In the early 2000s, I was fascinated by WinAmp visualizers that made music something to see as well as hear. They could lock me in for hours. The idea that you could connect two senses like that single-handedly drove me into my own explorations. This notebook is what those explorations grew into.
A poster or a performer
When you start thinking about music visualization, you quickly realize there are two types: static and audio-reactive. The first can capture the shape of sound in a given timeframe; the second lives in the present. Both can be functional and an artform.
A waveform lets you navigate a sound-space using your eye; an equalizer lets you tune the balance to your liking. As art, a static visual can hang on a wall — a whole song as one shape. A reactive one is closer to a performer: it exists only while the music plays.
The rest of this notebook builds the reactive kind, piece by piece — the composition in Fig. 0 is nothing but the techniques below. One question runs under all of them: can a visual feel composed with the music instead of merely shaken by it? Every section answers one part of that, and hands its answer back to Fig. 0.
The size of sound
Let’s start simple: what’s the most basic parameter of any audio we can react to? That would probably be presence — but take something less boolean: volume. The most direct way to picture volume is size, something that grows when the music does. One number doesn’t sound like much, but it covers a surprising share of real visualizers.
Scene setup
I won’t walk through scene setup in detail — this notebook assumes some three.js and react-three-fiber. We need a simple scene with a sphere we can animate later, lit well enough to read as a body: a key light for shape and a soft fill so the front face doesn’t drown in shadow. One extra element is declared and then ignored — the post-processing pass that prints this notebook’s figures in dots:
export default function App() {
return (
<Canvas camera={{ position: [0, 0, 4.4], fov: 45 }}>
<ambientLight intensity={0.7} />
<directionalLight position={[2, 2, 3]} intensity={2.2} />
<directionalLight position={[-2, 1.5, 4]} intensity={0.6} />
<Sphere />
</Canvas>
);
}
const Sphere = () => {
const meshRef = useRef();
useFrame(() => {
// Animation logic will be added here
});
return (
<mesh ref={meshRef}>
<sphereGeometry args={[1, 64, 64]} />
<meshStandardMaterial color="white" />
</mesh>
);
};
Here is that scene, rendered exactly as this page prints it — no audio yet, nothing to react to, just geometry under light:
The useAudio hook
Before anything can react, the scene needs two things: a sound source, and a number it can read every frame. Two hooks lay that pipeline down, and almost everything later in this notebook is a more selective way of producing the number.
useAudio is the source — loading, playing, pausing — wrapped in react and three.js methods:
const useAudio = (url) => {
const audioRef = useRef();
if (!audioRef.current) {
audioRef.current = new THREE.Audio(new THREE.AudioListener());
}
const audio = audioRef.current;
const buffer = useLoader(THREE.AudioLoader, url);
const { camera } = useThree();
useEffect(() => {
audio.setBuffer(buffer);
audio.setLoop(true);
camera.add(audio.listener);
return () => {
camera.remove(audio.listener);
if (audio.isPlaying) audio.stop();
};
}, [audio, buffer, camera]);
const togglePlay = () => {
if (audio.context.state === "suspended") ;
audio.isPlaying ? audio.pause() : audio.play();
};
return { audio, togglePlay };
};
audioRef holds one THREE.Audio, created lazily so React’s re-renders can’t stack up duplicates; useLoader decodes the file and suspends the component until it’s ready. The effect hands the buffer over and attaches the listener to the camera, and its cleanup stops playback, so an unmounted visualizer can’t leak a running track. The one line to keep is in togglePlay: a fresh AudioContext starts suspended, and only a user gesture can wake it.
The useAnalyser hook
And useAnalyser is the number. It wraps three.js’s AudioAnalyser, which reads the spectrum of whatever is playing — again a three.js convenience over the raw Web Audio API.
const useAnalyser = (audio) => {
const fftSize = 512;
const analyserRef = useRef();
if (!analyserRef.current) {
analyserRef.current = new THREE.AudioAnalyser(audio, fftSize);
}
const analyser = analyserRef.current;
const freqAvg = () => analyser.getAverageFrequency() / 256 || 0;
return { freqAvg };
};
The FFT size sets how many samples go into each analysis window — here, 512. A larger fftSize gives finer frequency detail but a heavier, slower-moving picture. That trade-off comes back more than once in this notebook.
* A 512-sample window is enough for average volume. It is too coarse for the band work later — we’ll grow it when we get there.
To picture volume we ask the analyser for a single value: getAverageFrequency(). Magnitudes come back as a Uint8Array, so values run 0–255 — dividing by 256 normalizes them to a friendlier 0–1. Before wiring it to any geometry, just look at it: the figure below runs the hook against the selected track and logs nothing else. Press play — one number, already following the music. Every visualizer in this notebook is, underneath, a decision about what to do with numbers like this one.
// press play — the hook reads zeros until the track sounds
Wiring it together
With both hooks ready, the rest is short — and it settles the pipeline every later example reuses: audio in, analysis per frame, visuals out. The sphere grows and shrinks with the loudness of the track, and a simple onClick toggles playback:
const Sphere = () => {
const { audio, togglePlay } = useAudio("/audio/black-pulse-drive.mp3");
const { freqAvg } = useAnalyser(audio);
const meshRef = useRef();
useFrame(() => {
meshRef.current.scale.set(freqAvg() + 1, freqAvg() + 1, freqAvg() + 1);
});
return (
<mesh ref={meshRef} onClick={togglePlay}>
<sphereGeometry args={[1, 64, 64]} />
<meshStandardMaterial color="white" />
</mesh>
);
};
It works. It also knows almost nothing about the music. A kick, a vocal and a hi-hat all arrive as the same instruction — grow. One number can say how much and never what, so the sphere can be loud but it can’t be intentional.
This signal survives all the way into Fig. 0, but with its job cut down to one: it grains the blob’s surface, an octave finer than anything else moving. Everything else up there needed a listener that can tell instruments apart.
Where is the bass?
So we can see how loud the music is — can we see something else? The next property of sound everyone knows is pitch: sounds are high or low, bass at the bottom, hi-hats on top. Before we go looking for the bass, though, let’s see the whole spectrum and what’s actually in there — the picture the classic WinAmp bars drew.
Getting the frequency data
The getFrequencyData() method fetches an array of amplitude values for consecutive frequency bins — the output of the Fourier transform behind the analyser. Two added lines expose it from our hook:
const useAnalyser = (audio) => {
const fftSize = 512;
const analyserRef = useRef();
if (!analyserRef.current) {
analyserRef.current = new THREE.AudioAnalyser(audio, fftSize);
}
const analyser = analyserRef.current;
const freqAvg = () => analyser.getAverageFrequency() / 256 || 0;
const freqDataArr = () => analyser.getFrequencyData();
return { freqAvg, freqDataArr, fftSize };
};
Mapping the data to bars
To visualize the data we map it onto a Bars component. The array length is half of the fftSize: the Fourier transform’s output is symmetric around the Nyquist frequency, so only the first half carries usable information — a detail we’ll return to in a moment.
const Bars = () => {
const { audio, togglePlay } = useAudio("/audio/black-pulse-drive.mp3");
const { freqDataArr, fftSize } = useAnalyser(audio);
const barsArr = new Array().fill(null);
return (
<group onClick={togglePlay}>
{((bar, i) => (
<Bar
key={i}
position={[i / 24, 0, 0]}
freqDataArr={freqDataArr}
id={i}
/>
))}
</group>
);
};
Each Bar represents one frequency bin: a thin plane whose height follows the amplitude of its slice of the spectrum, re-read on every frame:
const Bar = ({ position, freqDataArr, id }) => {
const meshRef = useRef();
useFrame(() => {
const data = freqDataArr();
const barData = data[id] / 256 || 0;
meshRef.current.scale.y = barData + 0.1;
});
return (
<mesh ref={meshRef} position={position}>
<planeGeometry args={[0.02, 1]} />
<meshBasicMaterial color="white" />
</mesh>
);
};
The empty half
Look closely at Fig. 12: only about half of the bars ever react to the music at all. The data we receive from getFrequencyData() isn’t broken — it’s just not shaped the way intuition expects.
Wishful thinking says the spectrum should split into neat, equally-busy thirds:
The analyser reports a different picture. Two definitions explain it:
FFT size sets how many bins divide the spectrum. With fftSize 512 we get 256 bins; a higher size gives more detail for more computation. Sample rate is how many audio samples arrive per second. The highest frequency a digital signal can represent is half its sample rate — the Nyquist frequency. At the typical 48,000 Hz, that’s 24,000 Hz.
Here is the catch: those 256 bins divide the range linearly. Each bin covers sampleRate / fftSize ≈ 94 Hz, marching in equal steps from 0 Hz all the way to 24 kHz. Music doesn’t live there. Pitch is logarithmic — every octave doubles the frequency — so the eight audible octaves below 5 kHz squeeze into the first fifty-three bins, while the entire top half of the display covers 12–24 kHz: a single octave, and a nearly empty one. No common instrument’s fundamental reaches past the piano’s top C at 4.2 kHz (bin forty-five), and above ~16 kHz — where lossy encoders low-pass and adult hearing fades — the bytes are often literal digital silence.
getFrequencyData() returns: 256 bins in index order. Hover a bar for the slice it covers — every note in music fits in the first fifth.Play the figure and the middle of the array visibly moves — no contradiction. That’s the brilliance band: harmonics of the notes below, plus the hiss of hi-hats, snares, and sibilant vocals. Real motion, little power — produced music rolls off at roughly 4.5–6 dB per octave up there. Fundamentals set the pitch; this region adds the sparkle.
The other end has the opposite squeeze: at 94 Hz per bin, an entire kick drum — roughly 40–100 Hz — fits inside one bin. The grid is too coarse for the bass we care most about and absurdly generous to air nobody hears. To react to the kick, snare, or melody separately, we have to define our own slices of the spectrum.
First it helps to know where the instruments actually sit — in the track you’re listening to, not in a chart of typical ranges. Each row below is one stem of the selected mix, looping on its own clock and always playing alone: press one to hear it, another to switch, again to stop (the notebook’s track keeps its place meanwhile). Listen to each part and watch where its energy lands on the axis; the next section turns those locations into band edges. To make a visual react to one instrument, we’ll keep its slice and ignore the rest.
Filtering frequencies
Everything so far reacted to the whole mix at once. It gets interesting when a visual reacts to parts of the music: the floor shakes with the kick, a sparkle answers the hi-hats. So let’s slice the spectrum into bands measured in hertz — the unit instruments live in.
First, resolution. Remember the asterisk under Fig. 5? This is where the window grows. Separating a kick from a bass line needs bins finer than 94 Hz, so fftSize becomes a parameter — from here on I pass 2048, which brings bins down to ~23 Hz.
Finer bins turned out not to be enough. My first band listener averaged the bytes across the kick’s slice, and the number came out wrong in a specific way: the kick was obviously several times louder than the groove around it, and the reading barely moved — a lazy drift where the ear heard a slam. The bytes aren’t amplitudes. They’re decibels in disguise.
The analyser maps the range from minDecibels to maxDecibels onto 0–255 logarithmically, and that costs twice. Anything above the default −30 dB ceiling clips flat at 255 — and a healthy kick lives well above it, so the loudest part of the signal is exactly the part that stops moving. Averaging raw bytes then averages logarithms, so what survives comes out compressed: a hit 4× louder than the groove reads as barely 1.2×.
So the hook grows in three ways at once. It takes an fftSize, exposes the raw analyser and the real sample rate for the band math, and widens the decibel window so a loud kick has somewhere to go:
const useAnalyser = (audio, fftSize = 512) => {
const analyserRef = useRef();
if (!analyserRef.current) {
analyserRef.current = new THREE.AudioAnalyser(audio, fftSize);
analyserRef.current.analyser.;
analyserRef.current.analyser.maxDecibels = -6;
}
const analyser = analyserRef.current;
const freqAvg = () => analyser.getAverageFrequency() / 256 || 0;
const freqDataArr = () => analyser.getFrequencyData();
return {
analyser,
freqAvg,
freqDataArr,
fftSize,
sampleRate: audio.context.sampleRate,
};
};
That widened window fixes the clipping. The compression is useBand’s job: it undoes the mapping first, turning every byte back into a linear amplitude, and only then averages — because that is the space where ratios behave.
const useBand = (analyser, fftSize, sampleRate, lowHz, highHz) => {
const binHz = sampleRate / fftSize;
return () => {
const node = analyser.analyser;
const data = analyser.getFrequencyData();
const lo = Math.max(0, Math.floor(lowHz / binHz));
const hi = Math.min(data.length - 1, Math.ceil(highHz / binHz));
let sum = 0;
for (let i = lo; i <= hi; i++) {
const db = node.minDecibels +
(data[i] / 255) * (node.maxDecibels - node.minDecibels);
sum += ;
}
const amp = sum / (hi - lo + 1) || 0;
return ;
};
};
// Two instruments, two listeners:
const kick = useBand(analyser, fftSize, sampleRate, 30, 100);
const snare = useBand(analyser, fftSize, sampleRate, 1500, 8000);
Before pinning the arguments down, try them by hand: drag the amber posts to move the band’s edges, drag the wash between them to slide the whole window, pull it up or down to amplify what comes through. Find the kick down low; slide up into the hi-hat hiss; park in the mids where the melody crowds. The sphere hears only what you select:
Why 30–100 Hz for the kick and 1.5–8 kHz for the snare? That’s where these instruments live — you heard it in the stems, and maybe just found both bands yourself in Fig. 18. A kick is a pitched thump, almost all of it below 100 Hz. A snare is mostly a burst of noise smeared across the upper mids, far above its own modest fundamental. They barely overlap — which is why something as blunt as an average can separate them:
Watch the sphere between kicks: it never quite settles, because the bass line shares its band. Instruments crowd the same neighborhoods, and a band listener hears the whole neighborhood. The beat detector in the next section has to work despite that.
Still, this is the first real movement in Fig. 0. The blob up there swells on a 25–90 Hz band rather than on the whole mix — it breathes with the kick instead of with the song, and the terrain and the fields lean on bands of their own. Different objects, different ears.
Catching the beat
Every visualizer so far is continuous: each frame maps energy to size. But the strongest moments in a music visual are discrete — a flash, a spawned particle, a camera cut, exactly on the beat. So how do we turn a wiggling signal into an event?
The obvious version — “fire when bass energy crosses 0.5” — fails twice, and both failures show up inside the first bar. A fixed threshold breaks the moment the track gets quieter or louder. And one kick stays loud for a dozen frames, so a dozen events fire where the ear heard a single hit. The fix is a short memory and a cooldown:
- Adaptive threshold. A beat is energy well above its own recent normal (the last ~0.75 seconds) — loudness-proof by construction.
- Rising edge. Fire only while energy is climbing, so the event lands on the attack.
- Refractory period. After firing, hold for ~0.28 s: one hit, one event.
const useBeatInFrame = (analyser, fftSize, sampleRate, onBeat) => {
const ;
const ;
const ;
useEffect(() => {
analyser.analyser.;
}, [analyser]);
const bass = useBand(analyser, fftSize, sampleRate, 25, 140);
const history = useRef([]);
const lastBeat = useRef(-Infinity);
const prev = useRef(0);
useFrame(({ clock }) => {
const t = clock.elapsedTime;
const energy = ;
const h = history.current;
h.push({ t, e: energy });
while (h.length && t - h[0].t > 0.75) h.shift();
const mean = h.reduce((s, x) => s + x.e, 0) / h.length || 0;
const rising = energy > prev.current;
prev.current = energy;
if (
energy > floor &&
rising &&
energy > mean * threshold &&
t - lastBeat.current > minInterval
) {
lastBeat.current = t;
onBeat({ time: t, energy });
}
});
};
I tuned this for kick-forward music with a steady pulse, and three rules in thirty-five lines gets you remarkably far. It is still naive. Play something with a sparse, syncopated kick — half of trap and hip-hop — and watch the readout: the kick drops out for two bars, the threshold adapts down, stray hits sneak past, and the “BPM” slides from 140 to a half-time 60 without the song ever changing tempo. A detector that forgets everything older than 0.75 seconds cannot hold a groove.
That costs more than a wobbling number, because beats don’t decorate a composition — they decide it. In Fig. 0 a single eight-step counter advances once per beat, and the corner indicator, the spark bursts and the blob’s quarter turn all read that one counter. A detector that miscounts doesn’t make the piece wobble; it puts every decision in it in the wrong place. So before we assemble anything, the detector has to grow up.
Keeping the beat
One constraint up front: the best beat tracking is offline — scan the whole track, try beat grids against everything, pick the best fit. A live visualizer can’t do that; the future hasn’t played yet. And the naive detector’s deeper flaw is that it treats tempo as a measurement, re-taken every frame from 0.75 seconds of memory. The tracker treats it as a belief: built from evidence, defended against noise. It has three parts — better onsets, an election, and a flywheel.
Onsets first. Instead of one band’s energy, we watch the whole spectrum for instants where energy suddenly appears anywhere. The standard tool is spectral flux: sum every bin’s rise since the last frame, ignore the falls. A hi-hat, a snare, a piano chord — they all count now, not just the kick. The threshold adapts (mean plus 2.5 deviations of recent flux) and a one-frame lookback confirms each peak:
const useOnsets = (analyser, onOnset) => {
const prev = useRef(new Uint8Array(analyser.frequencyBinCount));
const history = ;
const lastFlux = useRef(0);
const lastOnset = useRef(-Infinity);
useFrame(({ clock }) => {
const t = clock.elapsedTime;
const spec = analyser.getFrequencyData();
let flux = 0;
for (let i = 1; i < spec.length; i++) {
const rise = ;
if (rise > 0) flux += rise;
}
prev.current.set(spec);
const h = history.current;
h.push({ t, flux });
while (t - h[0].t > 1.5) h.shift();
const mean = h.reduce((s, x) => s + x.flux, 0) / h.length;
const dev =
Math.sqrt(h.reduce((s, x) => s + (x.flux - mean) ** 2, 0) / h.length) || 1;
const prevFlux = lastFlux.current;
if (
prevFlux > &&
&&
t - lastOnset.current > 0.1
) {
lastOnset.current = t;
onOnset({ time: t, strength: (prevFlux - mean) / dev });
}
lastFlux.current = flux;
});
};
Then the election. Every onset pairs with the recent ones before it, and every gap votes for the periods it implies: 923 ms votes for 923 ms — and for its half, third, and quarter, because if beats land every 462 ms, 923 ms gaps keep happening too. Votes pile into a histogram; old votes decay. After a few bars, one period towers over the rest.
Finally the flywheel. The tracker doesn’t act on the first winner: it waits until the leading period beats the runner-up decisively (octave twins don’t count) and holds that confidence for a second and a half. Then the tempo locks, and the beat becomes a free-running clock — firing whether or not the kick shows up, like a drummer keeping time through a breakdown. Onsets near a predicted beat nudge the clock: phase a lot, tempo a little. Onsets far from it only matter when they are strong, repeated, and the histogram has genuinely moved — silence is not a tempo change; only sustained contradiction is.
// 1 - the election
const vote = (onset) => {
for (let i = 0; i < hist.length; i++) ;
for (const past of onsets) {
const gap = onset.time - past.time;
for (let k = 1; k <= 4; k++) {
const period = ;
if (period < 0.25 || period > 1.45) continue;
hist[Math.round((period - 0.25) / 0.01)] +=
(onset.strength * past.strength) / k;
}
}
onsets.push(onset);
};
// 2 - confidence
const confidence = () => {
const peak = strongestPeriod(hist);
const runner = ;
return peak.score / (peak.score + runner.score);
};
// 3 - the commit
if (!locked && ) {
locked = true;
period = peak.period;
nextBeat = lastOnset.time + period;
}
// 4 - the flywheel
useFrame(({ clock }) => {
if (locked && clock.elapsedTime >= nextBeat) {
onBeat({ time: nextBeat, });
nextBeat += period;
}
});
// 5 - the nudge
const onGridOnset = (onset) => {
const err = onset.time - nearestGridTime(onset.time);
if (Math.abs(err) < 0.15 * period) {
;
period += 0.05 * err;
} else if (onset.strength > 4) {
contradictions += 1;
if () locked = false;
}
};
That’s why it holds where the naive detector wandered. Breakdown? The flywheel keeps predicting, and the kick’s return snaps phase back within a beat or two. Syncopation? Off-grid hits are just onsets, outvoted by the histogram. A real tempo change? Contradictions pile up, the histogram migrates, the lock releases, the election re-runs. The naive detector answered all three the same way: panic.
And this is the clock Fig. 0 runs on. The eight-count at the top of this page is the tracker’s, which is why the composition keeps striding through a breakdown instead of stalling when the kick does — the flywheel is what makes the piece look like it knows the song.
A note on performance
One detour before the assembly. Every example so far had a frame to itself; the composition at the top runs six visualizers inside one, and it only affords that because of two habits. Here is the first.
Remember the map() from Fig. 10? It mounts 256 React components, runs 256 frame callbacks, copies the frequency data 256 times per frame, and asks the GPU for 256 draw calls. Fine for learning; wasteful for the real thing. InstancedMesh collapses all of it: one component, one data read, one draw call, any number of bars.
const InstancedBars = () => {
const { audio, togglePlay } = useAudio("/audio/black-pulse-drive.mp3");
const { freqDataArr, fftSize } = useAnalyser(audio);
const count = fftSize / 2;
const meshRef = useRef();
const dummy = useMemo(() => new THREE.Object3D(), []);
useFrame(() => {
const data = freqDataArr();
for (let i = 0; i < count; i++) {
dummy.position.set((i - count / 2) / 24, 0, 0);
dummy.scale.set(1, data[i] / 256 + 0.1, 1);
dummy.updateMatrix();
meshRef.current.setMatrixAt(i, dummy.matrix);
}
meshRef.current.instanceMatrix.needsUpdate = true;
});
return (
<instancedMesh ref={meshRef} args={[null, null, count]} onClick={togglePlay}>
<planeGeometry args={[0.02, 1]} />
<meshBasicMaterial color="white" />
</instancedMesh>
);
};
The second habit is on the React side: nothing in a useFrame should ever call setState. Mutate refs, materials, and uniforms directly — React orchestrates the scene; the frame loop just moves it.
Putting it together
We started with a sphere that knew one thing: loud or quiet. The composition at the top listens three ways at once. The overall level sets texture, the kick band sets shape, and the locked beat grid makes decisions. The wiring is short — the interesting part is which signal gets which job.
The split is between signals that describe and signals that decide. A band always has a value, so it belongs on properties that always have one: the size of a swell, the depth of a grain, the shiver of a terrain. A beat either arrives or it doesn’t, so it belongs on things that happen — a spark burst, a quarter turn, a cut. Swap them and both break. Sparks driven by continuous energy smear into a fog that never quite starts; a swell driven by beats is a shape with eight positions and no way between them.
The decisions share one counter. Every beat advances a single eight-step index, and the corner indicator, the sparks on steps four and eight, and the blob’s quarter turn on eight all read that index instead of counting for themselves. Three objects, one clock: they can’t drift apart, so the piece reads as one performer rather than three musicians who almost agree. That counter runs on the tracker, not the naive detector — an eight-count has to survive a breakdown, or the composition loses the plot exactly where the music gets interesting.
And one thing is deliberately kept off the grid. The blob’s radius pops on the kick’s own transient — two followers over the same band, one fast, one slow, their gap is the attack — because a pop on the predicted beat is punctual, while a pop on the drum itself is played. The grid is for structure; the audio is for feel:
const follow = (v, target, dt, up, down) =>
v + (target - v) * (1 - Math.exp(-() * dt));
useFrame((_, dt) => {
const raw = kickBand();
// Describing signals: eased toward the music, never wired to it raw.
kick.current = ;
level.current = follow(level.current, freqAvg(), dt, 10, 4);
// The kick's own attack: two followers over one band, fast minus slow.
fast.current = follow(fast.current, raw, dt, 60, 60);
slow.current = follow(slow.current, raw, dt, 8, 8);
const gap = ;
const attack = Math.min(1, Math.max(0, gap * 6.5));
thump.current = Math.max(attack, );
= kick.current;
= level.current;
= thump.current;
});
// Deciding signals: one locked grid, one counter, every visual reading it.
(analyser, fftSize, sampleRate, () => {
step.current = ;
if () burst();
if () quarterTurn();
});
If you take one habit from this notebook, take the one holding that loop together: never wire raw FFT data straight to geometry. Raw values are what make a visual look like a meter — jumpy, literal, obedient. The follower and the envelope are what make it look like it’s dancing to something.
And the target doesn’t have to be scale — a band can drive anything a frame can set. Here the kick shakes the camera, and every beat flashes one random solid:
Beats can do something discontinuous instead: spawn geometry, cut between cameras, step through whole scenes. When the edits land on the grid, a piece stops feeling animated and starts feeling edited:
Two more ideas worth stealing: pin a lyric sheet to the beat grid — the flywheel makes a karaoke cursor that doesn’t drift — or kick a physics world on every beat: debris that jumps, a hanging lamp the snare keeps knocking, choreography you’d never keyframe.
Where to go next
That’s the whole toolkit: volume, spectrum, bands, and beats, plus the easing that keeps them watchable. If you want to go deeper, here’s where I’d go.
On the audio side, MDN’s Visualizations with Web Audio API guide is the ground this notebook’s three.js conveniences stand on — the same AnalyserNode, with nothing wrapped. Meyda computes real audio features in the browser — RMS, spectral centroid, perceptual loudness — each a drop-in upgrade for a useBand signal. And if the beat tracker left you curious, the FMP notebooks by Meinard Müller are a free, runnable textbook — their chapter on tempo and beat tracking is our election and flywheel with the mathematics shown.
On the visual side, The Book of Shaders is still the friendliest door into GLSL — the hero’s noise field is forty lines away from being yours. Maxime Heckel’s study of shaders with react-three-fiber picks up where this notebook stops, and Bruno Simon’s Three.js Journey is the long road. The dotted look of every figure here is ordered dithering; Surma’s Ditherpunk unpacks it if you want a pass of your own. And a shader doesn’t have to be decoration: give one a frame of memory — a texture it writes and reads back the next tick — and it stops drawing a picture and starts running a simulation, one the music can only push around from the outside:
Music used to be something we watched happen. For the length of a WinAmp window, it was again. Build the next window.
Thank you pugson for reading the early draft.