A 27B Model Now Runs on the Card Already in Your PC
@sudoingX maps a 27B across 8, 12 and 16 GB cards at 42, 50 and 67 tok/s.
@sudoingX tested one file on every VRAM tier and published the map. Bonsai 2 27B, PrismML's ternary compression of Qwen 3.8 27B, squeezes into 5.95 GB. It holds 42.8 tok/s on an 8 GB card out to a 96k window and 50.1 tok/s fresh on 12 GB. On 16 GB it hits 67.3 tok/s with the full 262k window, the MTP head and vision together in 15.2 GB.
Two fixes did the work. @sudoingX repaired the kernel in PrismML's llama.cpp fork that left most of the card idle, taking a 3060 from 25-26 tok/s to 40. Then he grafted Qwen 3.8's own MTP draft head back on and it reached 50. The kernel is PR #218, approved and waiting to merge. The packaged model counts 27k downloads, while the PrismML release is the #1 trending text model on Hugging Face at 3.97 million.
The Steam hardware survey sets the audience: 67% of Steam GPUs carry 8 to 16 GB. 16 GB is the largest tier at 27.21%, and the RTX 5070 12 GB is now the most owned card at 5.86%. At 24 GB he says skip the compression: Qwen 3.8 27B at q4_k_m fits its whole 262k window in 22.2 GB. On a 3090 it runs 41.3 tok/s with the MTP head.