int-llm

Executive summary

int-llm asks whether a useful language-model reference can be made exact across machines. It implements a small character GPT that trains and samples in integer arithmetic and a CPU Llama-family inference path for TinyLlama-1.1B. The compute core uses Q16.48 fixed point: normal source weights retain enough precision to make disagreement with the floating-point control debuggable rather than ambiguous. The result is intentionally slow. Its value is as a deterministic oracle for regression testing, portability work, and later lower-precision experiments—not as a production inference server.

Problem and context

Most model runtimes accept small numerical differences across hardware, libraries, and compiler paths. That is appropriate for production throughput, but it makes numerical drift and quantization bugs difficult to isolate. int-llm starts from the opposite requirement: the same input should produce the same integer state and the same output bits on every supported target.

Q16.48 is deliberately wider than a practical quantized model format. It is the reference against which narrower representations can be measured.

What was built

gpt_int, a character-level GPT that trains, saves, loads, and samples using

integer arithmetic.

llama_int, a CPU Llama-family path that loads TinyLlama SafeTensors or a

native all-integer .mgw file.

fp_math.h, a dependency-free fixed-point math layer with native and portable

wide-integer backends.

Determinism harnesses for desktop CPUs, microcontrollers, and a supplementary

QEMU portability matrix.

Retained CUDA experiments documenting why the exact integer path did not

provide a GPU performance advantage.

How it works

Model values are represented as signed 64-bit Q16.48 integers. Multiplication and division use either native __int128 intermediates or a portable two-limb implementation. Nonlinear functions—including square root, exponential, logarithm, trigonometry, sigmoid, and SiLU—are implemented in the integer math layer.

The Llama path covers tokenization, embeddings, grouped-query attention, rotary position encoding, RMS normalization, the SwiGLU feed-forward path, and output selection. Human-readable conversion stays at the input and output boundaries.

Evidence and validation

The repository records three distinct kinds of evidence:

Gate	Recorded result
Float-reference comparison	80/80 greedy verification tokens matched
Arithmetic determinism	One committed hash reproduced on the documented real and emulated targets
Embedded model run	The small model reproduced the same sampled output on recorded 32-bit microcontrollers

The GPU work is reported as negative or mixed evidence: exact Q16.48 did not find a competitive tensor-core path, while faster lower-precision formats gave up the required exactness.

Boundaries

This is a feasibility and reproducibility artifact. It makes no competitive throughput claim, does not establish general chat-quality parity, and does not validate models larger than the documented TinyLlama target. Native model files are trusted inputs; the loaders are not a hostile-file security boundary.

Repository map and first run

The fastest self-contained path builds the integer math tests and committed determinism gate:

make test
make determinism
make determinism-portable

Use FP_MATH.md for the math API, validation/cpu/ for real-hardware records, validation/qemu/ for supplementary compiler and ABI coverage, and gpu/ for the measured CUDA side branch. Running TinyLlama requires obtaining the upstream checkpoint separately.