DeepSeek-1017 on a Mac Studio M5 Ultra Runs at 54.5 Tok/s With ds4
@niklaslenz_ai used antirez’s Q2 GGUF; the Engram tables stream from SSD.
A single Mac with 256 GB, the weights in memory and the Engram tables on SSD.
@niklaslenz_ai ran antirez’s Q2 GGUF of DeepSeek-V4.1-Flash-1017 on a Mac Studio M5 Ultra with 256 GB, in kernelpool’s fork of antirez’s ds4 engine, in batched mode with five sessions. The 152 GiB of main weights stay in memory; the 189 GiB of Engram tables are read from SSD. He used the model card’s sampling, temperature 1.0, with thinking off.
| Measure | Result |
|---|---|
| Decode, code and prose | 54.5 tok/s |
| Reading a 32K prompt | 1,475 tok/s |
| First token after 32K | 22.2 s |
| Four requests, total | 56.2 tok/s |
| Eight requests, total | 50.5 tok/s |
His scores carry wide error bars: 72.3% on MMLU-Pro (plus or minus 4.4 points), 67.5% on LiveCodeBench (plus or minus 14.0) and 87.5% on IFEval. GPQA Diamond rose from 49.0% to 76.7% with thinking on. He set 524,288 tokens of context per session and tested up to 131,072.
Sources
- @niklaslenz_ai on X · Oct 24, 4:00 AM PT