Models & Hardware
TensorFold Beats SGLang on DeepSeek-1017 Across Three Sparks
108.6 against 68.3 tok/s at four sessions, with 95.2% on a 133-task German benchmark.
One cluster, one quant family, two engines: a rare like-for-like on three Sparks.
@HandwerkAiLab ran DeepSeek-V4.1-Flash-1017 on three DGX Sparks at tensor-parallel 3, comparing TensorFold by @ashxhart with SGLang. The TensorFold side used @easthambuilds’ 3.5 bpw EXL3 quant.
| Measure | TensorFold | SGLang |
|---|---|---|
| Four sessions, tok/s | 108.6 | 68.3 |
| 16K agent wait | 11.8 s | About 20 s |
On quality, the build scored 95.2% on his 133-task DACH-Bench, a German and EU benchmark, and 97.5% with thinking on. In an earlier reply the same day he gave the TensorFold build 74.7 tok/s, and 229.8 tok/s across four streams.
He did not post the SGLang build’s quant or a recipe, so the comparison rests on his post.
Sources
- @HandwerkAiLab on X · Oct 24, 4:15 AM PT
- @HandwerkAiLab on X · Oct 24, 1:48 AM PT