DeepSeek-V4.1-Flash-1017 Hits 68.5 Tok/s on Two Sparks, @niklaslenz_ai Measures
@MiaAI_lab’s EXL3 2.9 bpw quant on TensorFold: 2,124 tok/s prompt reading and 524,288 tokens of context.
Community Day, post 7: the numbers.
@niklaslenz_ai measured DeepSeek-V4.1-Flash-1017 on two DGX Sparks at 68.5 tok/s on @MiaAI_lab’s latest 2-Spark recipe: her EXL3 2.9 bpw quant on TensorFold, one model split across both Sparks. He used her recipe as of Oct 21, commit 370daf0, with her sampling and context. Thinking was off for every number.
| Measure | Result |
|---|---|
| Decode, code | 72.0 tok/s |
| Decode, prose | 65.0 tok/s |
| Four requests at once | 98.4 tok/s total |
| Eight requests at once | 99.6 tok/s total |
| Reading a 32K-token prompt | 2,124 tok/s |
| Context | 524,288 tokens |
| sparkDash 2.50.0 QA | 94.0% ± 3.9 points |
| MMLU | 91.9% ± 3.2 points |
| Tool Eval | 93/100 |
Against the Oct 19 version.
On the same two Sparks, code decode went from 73.0 to 72.0 tok/s and prose from 61.7 to 65.0. Code stays the same and prose got about 5% faster, he writes; the sparkDash QA, MMLU and Tool Eval scores came out exactly the same.
His pick on two Sparks is DeepSeek-V4.1-Flash-1017 on Mia’s recipe. He credits the recipe, quant and sparkDash to @MiaAI_lab, the TensorFold engine to Ash Hart and its DeepSeek-V4.1 support to BertholomusAI. @MiaAI_lab reposted the results.
Sources
- @niklaslenz_ai on X ·
- @MiaAI_lab on X · Oct 22, 11:30 AM PT