Weight atlas

MLP neuron order — one checkpoint, four physical layouts

The one derived view on this page, labeled as such. The sibling int-llm-coordinate-permutation repo reorders this checkpoint's 128 MLP neurons (the "hidden unit" axis of the two panels above) into three alternative physical layouts. A layout is a pure re-indexing: physical slot j holds original neuron new_to_old[j] — the mlp_fc1 row and the matching mlp_fc2 column move together, and no stored value changes. The self-check above re-permutes the embedded tensors per order and matches their checksums against the variant checkpoints this repo's C oracle replayed bit-identically; it also recomputes every adjacency score in exact BigInt arithmetic.

Residual coordinate swap — one transposition through all nine matrices

The page's second derived view, labeled as such. The sibling int-llm-coordinate-permutation repo also swaps two coordinates of the 32-wide residual (hidden) basis — lane 0 with lane 31 — through every learned matrix at once: tensors that carry the residual axis on their columns (wte, wpe, lm_head, wq, wk, wv, fc1) swap two columns; tensors that carry it on their rows (wo, fc2) swap two rows. Physical index j holds original coordinate new_to_old[j]; 892 of the 14,272 stored positions move and no value changes. Applied consistently everywhere, the swap preserves the model function exactly — this repo's C oracle replayed the swapped checkpoint bit-identically. The self-check above re-permutes all nine embedded tensors and matches their checksums against that C-verified variant. All nine pairs are drawn below one another; in every pane the green lane is original coordinate 0 and the orange lane is original coordinate 31, so the trade itself is visible: they sit at positions 0 and 31 in the original pane and at 31 and 0 in the swapped one.

All nine matrix pairs are drawn below — click a table row to jump to its pair. Hover any pane, or focus one and use arrow keys, to follow one residual coordinate across the swap; Esc clears.