MiniMax-M3.1-Flash on Four Sparks, the vLLM Way: 22 Tok/s
A stock-image recipe with a long context. On two Sparks, there are still no numbers.
A slower but stock route, and still nothing measured on two Sparks.
@Tech2Wild posted a tested recipe for MiniMax-M3.1-Flash, 428B total and 23B active, on four DGX Sparks. It uses the official vLLM arm64 image, MiniMax’s own DSpark speculative decoding and a 300,000-token window, with the weights read from one node over NFS.
| Test | Result |
|---|---|
| Single user, short chat | 22 tok/s |
| Single user, 32K and 97K prompts | 33 and 31 tok/s |
| Six users, short prompts | 84 tok/s in total |
| Prefill, 8K to 112K | About 1,480 tok/s |
| KV pool | 1,415,656 tokens, 4.72 full 300K requests |
| 69-scenario tool eval | 82.6 (54 pass, 6 partial, 9 fail) |
Its README calls these a baseline. On the Sparks’ GB10 chip, vLLM runs the FP4 experts weight-only through Marlin, because the per-token activation scales the model expects are implemented only in SM100 kernels. This morning’s SGLang recipe from @majewskizby measured 46.5 tok/s on prose; the two use different engines and drafters.
Two Sparks: no numbers yet.
@jaita tried Mia’s 3.06 bpw MiniMax pack on two Sparks and TensorFold reported no registered family for the model type. Mia replied that it needs engine work that has not been published yet. She says she is spending her time on DeepSeek for four Sparks instead and will probably skip MiniMax altogether.
Sources
- @Tech2Wild on X · Oct 23, 7:17 AM PT
- @MiaAI_lab on X · Oct 23, 7:18 AM PT
- tonyd2wild/MiniMax-M3.1-Flash-TP4-DGX-Sparks · github.com
- @jaita on X · Oct 23, 8:34 AM PT
- @MiaAI_lab on X · Oct 23, 8:39 AM PT
- @MiaAI_lab on X · Oct 23, 7:09 AM PT