MiniMax-M3.1-Flash Runs on Two Sparks at Up to 39 Tok/s

The first measured two-Spark run of MiniMax’s 428B model, on the smaller of two new quants.

@ViC305 released two SAGE EXL3 quants of MiniMax-M3.1-Flash on Friday afternoon: a 2.15 bpw pack of 108.64 GiB for one DGX Spark, and a 4.26 bpw pack of 209.88 GiB for two. Measured against the original weights, the smaller one agrees on the top token 96.89% of the time, the larger 98.44%. He calls these fidelity measurements, not task pass rates. His one-Spark vLLM recipe reaches 32K context, and he asked two-Spark owners to build the TP2 recipe.

@Tech2Wild did. His repo serves the 2.15 bpw pack at tensor-parallel 2 over RoCE, with the weights on one node and read by the other over NFS. Context is 300,000 tokens per request, the KV pool holds 726,578 tokens, and DSpark speculative decoding drafts three tokens a step. It took two small patches to the vllm-exl3 plugin.

With more users, total throughput peaks at four streams: 66.2 tok/s on code, 78.1 on counting and 48.4 on prose. At five streams, per-stream speed halves; the README suspects a slower kernel path and says it has not investigated yet. Prefill is the weak spot, at about 500 to 570 tok/s, so a 128K-token prompt takes about four minutes to the first token.

The 4.26 bpw pack, released for two Sparks, did not fit. At TP2 it took 106.1 GiB of weights per node and left 1.72 GiB free before any KV cache, the README says. With a config overlay it should fit on four Sparks, which has not been run yet.