Research
What we build and how we know it works. Five research areas, each ending in a machine-checked gate — read the evidence, then run it yourself.
Arena autograd
A dynamic-graph autograd engine written from zero: Value nodes appended to one Graph, ~20 op builders, backward dispatch on the op string, gradients accumulated in place. One epoch = reset + re-register + ONE backward, with full-batch gradients verified equal to per-sample accumulation within 1e-12.
ml/autograd.mojofull-batch == per-sample < 1e-12Numeric verification
Trust is a gate, not a claim. Every op's analytic backward is checked against central differences at init time — max |analytic − numeric| < 1e-6, abort() on failure. All 23 checks pass at ≤ 2.4e-8, about four orders of magnitude inside the bar. Exit codes are the only honest CI signal in release-mode Mojo.
grad checks ×23 ≤ 2.4e-8gate < 1e-6Grokking, reproduced honestly
An MLP trained on all-but-diagonal (a+b) mod 14 pairs memorizes by epoch ~1,000, then — thousands of epochs later — generalizes to the 14 held-out pairs. Reproduced at the full 30k epochs for both weight decays; stronger decay crosses earlier (epoch 9,000 vs 25,000), matching the reference ordering.
diag acc 1.00 @ epoch 9,000 (wd=1.0)30k epochs × 2 decaysAttention that pays rent
MiniGPT — Embedding, learned positional table, two post-LN TransformerBlocks with causal multi-head attention — fits a deterministic char grammar through real attention: train 97.7%, held-out 94.5%. Sequences travel as flattened [1, T·D] rows; no 3D tensors anywhere.
MiniGPT · 26,182 params · held-out 94.5%Convolution without libraries
Conv2D forward via im2col, backward via col2im, MaxPool2D by max-index routing — all on plain row-major Lists of Float64. The SimpleCNN fit gate closes at stripes-task held-out accuracy 0.80, train 0.978 > 0.9 threshold.
SimpleCNN · 114 params · held-out 80.0%
Method notes live in Commitments — the rules every result on this site is held to.