Models & Hardware
DeepSeek-1017 Runs at 37.1 tok/s on a Strix Halo and Two Radeons
The model does not fit in 128GB, so experts stream from an NVMe drive over USB4.
A model bigger than the box’s RAM, served anyway.
@jgattermayer’s Strix Halo machine with two second-hand RX 7900 XTX cards now runs DeepSeek V4.1 Flash 1017. The Maya-S GGUF does not fit in 128GB, so experts stream from an NVMe drive over USB4; the M.2 slots are taken by the cards’ OCuLink cables.
To make it work: an uneven tensor-parallel split, the speculative-decoding drafter on both GPUs so decoding reads the disk less, the Strix Halo iGPU serving experts that don’t fit on the cards, and math library fixes.
| Test | Result |
|---|---|
| Generation after a cold 130k prompt | 37.1 tok/s (42.8 warm) |
| Cold prefill at 130k | 155 tok/s |
| Generation across the 256k ladder | 31 to 35 tok/s |
| Disk reads per decoded token | About 24% fewer |
Four patches are merged upstream in Project Maya, and the uneven split is in review. The full report and an Ansible setup are in the issue.
Sources
- @jgattermayer on X · Oct 23, 12:55 AM PT
- Project Maya issue 410 · github.com