Executive summary
int-llm-precision-ladder follows the wide Q16.48 oracle from int-llm with a narrow question: how far can stored weights be rounded while preserving documented decisions? Its main TinyLlama candidate uses mixed F11/F12 signed 16-bit weight codes while keeping activations, nonlinear functions, the key/value cache, and accumulation wide. It matched the oracle on 632 evaluated greedy decisions and removed the fourfold storage expansion of the Q16.48 weight file. The repository presents this as an explicit regression result, not general model quality parity or a production quantizer.
Problem and context
The exact Q16.48 path is useful as a numerical reference but inefficient as a stored model representation. Starting from an oracle allows rounding choices to be evaluated against a fixed target instead of another approximation. The project measures behavior, storage, memory, and runtime while keeping each claim tied to its particular host or microcontroller.
What was built
- Precision sweeps over small integer models to choose candidate fractional
- Mixed F11/F12 packing for TinyLlama weights in a signed-int16 MGWI format.
- Direct SafeTensors-to-MGWI conversion with byte and integer-code gates.
- Packed and rounded-wide comparison runtimes.
- Physical-board MicroGPT experiments across Arm and RISC-V targets.
- CUDA reference experiments retained even where results were negative.
widths.
How it works
F12 means twelve fractional bits after deterministic rounding; it does not mean a twelve-bit container. Each retained code occupies a little-endian signed 16-bit value. Matrix inputs are widened for computation, and signed 128-bit intermediates preserve the accumulation path. The original Q16.48 runtime remains the behavioral oracle.
This distinction separates representation savings from arithmetic claims. The packed and rounded-wide forms can have identical logits even though both differ numerically from the original model.
Evidence and validation
| Experiment | Recorded result | Scope |
|---|---|---|
| TinyLlama mixed F11/F12 | 632/632 evaluated greedy decisions matched | 32 short deterministic prompts |
| Direct conversion | 1,100,048,384 codes and all output bytes matched | One pinned checkpoint and plan |
| AMD packed vs rounded-wide | 23.209% lower generation time and 73.303% lower peak RSS | One host, same integer model |
| Practical FP32 control | FP32 was 3.168899× faster | Different runtime stacks |
The microcontroller results show that smaller storage often helps but does not guarantee a runtime improvement; one documented board was effectively at parity.
Boundaries
The checked prompts do not establish arbitrary-prompt or chat-quality parity. The x86 timing is not an arithmetic-only comparison, and the FP32 control was substantially faster. The CUDA evidence does not show an integer inference advantage. MGWI files are treated as trusted experimental inputs.
Repository map and first run
sh scripts/validate.sh localRead RESULTS.md for the small-model sweep, TINYLLAMA_RESULTS.md for the full model gate, MCU_RESULTS.md for physical targets, GPU_RESULTS.md for retained negative and mixed evidence, and VALIDATION.md for the exact claim boundary.