Executive summary
int-llm-coordinate-permutation applies reversible coordinate changes to Llama-family checkpoints. The small scope reorders vocabulary rows and translates token IDs at the interface. The all-axis scope also reorders feed-forward neurons, whole grouped-query attention heads, and the global residual hidden basis with compensating row and column moves. Implementations work directly on BF16, F16, and F32 SafeTensors as well as the Q16.48 integer format. These are known model symmetries and layout experiments—not encryption, access control, or a method for making public weights private.
Problem and context
A neural network's stored coordinate order is not always part of its function. If every connected tensor is transformed consistently, many row and column orders can change while preserving the logical model. This repository turns that observation into strict tooling with explicit boundary maps and inverse operations.
What was built
- Deterministic token-map creation from a private random key or explicit order.
- Vocabulary-only SafeTensors and integer-model converters.
- All-axis converters for vocabulary, MLP, attention-head, and residual-basis
- Structural verifiers that check every prescribed destination coordinate.
- Integer and floating-point inference comparison harnesses.
- Explicit MLP layouts (original, smooth, spread, and seeded-random) and a
- Format, provenance, security, and prior-art documentation.
coordinates.
minimal residual-basis swap across all nine small-model matrices.
How it works
Vocabulary rows move together in the input embedding and any separate output head. Token IDs are translated into the physical namespace on input and back to logical IDs on output. Internal coordinate maps require no runtime sidecar because their inverse relationships are baked into the affected matrix axes.
Attention transforms move whole heads and preserve grouped-query associations. Coordinates inside a head stay in place so rotary pairs remain canonical. The residual-basis permutation is global because every residual addition must use the same basis.
Evidence and validation
On the documented TinyLlama checkpoint, the all-axis composition classified and transformed all 201 learned tensors. The Q16.48 path passed prescribed-axis verification and the original 80/80 greedy-token gate. The direct BF16 path restored the original checkpoint byte-for-byte after the inverse transform; its floating-point control retained 48/48 top-1 decisions while showing small logit drift from a different reduction order.
The August follow-up adds explicit small-model layout experiments. Swapping residual coordinates 0 and 31 changes all nine learned matrices while preserving all 5,832 checked Q16.48 logits; the inverse restores the checkpoint exactly. Four MLP-neuron orders compare adjacency distances without claiming sparsity or a performance improvement. The sibling visualization project makes those verified changes inspectable.
Verifiers fail closed when the tensor layout, data type, or configuration falls outside the supported Llama shape.
Boundaries
Permutation is not encryption and does not hide the model function. A public base model makes a transformed checkpoint particularly easy to analyze. The floating-point path need not reproduce every logit bit because changing the reduction order changes rounding. Current full-checkpoint tools accept one unsharded SafeTensors file and a constrained set of Llama tensor names and types.
Repository map and first run
make test
python3 test_safetensors_coordinate_permutation.pyStart with the vocabulary-only fixture before using a full checkpoint. Read FORMAT.md for the map representation, RESULTS.md for measured gates, SECURITY.md for non-claims, and SOURCE_PROVENANCE.md for copied and derived components.