Bonsai 2 9B Got Six Times Faster Without Changing a Token

Faster but wrong did not count.

PrismML gave day-zero access to Bonsai 2 9B, an open-weight ternary model, @pratikg explains. YukonResearch froze a reference engine: MLX on Apple Silicon and llama.cpp on DGX Spark. The weights stayed locked, and everything that runs the model was open to edit.

Every submitted run was verified on an Apple M5 Max or a DGX Spark and had to produce exactly the reference tokens. Beat the record and your code became everyone’s new base. That happened 158 times.

Decode went from 144.8 to 959.6 tok/s on Apple Silicon. The speedups came from speculative decoding, a DFlash 2 drafter shrunk to 4-bit, draft trees, int8 prefill and kernel fusion.

@Mahone_AI likes the exact-token rule most, because many local speedups quietly swap in a lower-bit quant or a different sampler. One caution: the 471% headline on the Mac board is a blended score, and the decode jump alone works out closer to 6.6x.