The State of the Sparks: GLM-5.3 and DeepSeek V4.1 Flash on Two and Three DGX Sparks
What fits in 256 GB and 384 GB, how fast it runs, how close the quants stay to full precision, and which inference engine to pick for each build.
Two NVIDIA DGX Sparks give you 256 GB of unified memory. Three give you 384 GB. This month, the two best open models builders run at that level are Z.ai's GLM-5.3 family and DeepSeek's October 17 refresh of V4.1 Flash, known as 1017. Here is what fits, how fast it runs, how close it stays to full precision, and which engine to use.
At a glance
- Two Sparks. GLM-5.3-Flash fits at about 4 bits with room for a 1M-token context. DeepSeek-V4.1-Flash-1017 fits only at 2.7 to 2.9 bits, with its Engram tables kept on disk.
- Three Sparks. DeepSeek-1017 runs from its official weights. The full 753B GLM-5.3 fits at 2.75 bits. GLM-5.3-Flash gains context and concurrency more than single-stream speed.
- Speed. On two Sparks, the best measured single-stream prose rates are 60.4 tok/s for GLM-5.3-Flash and 65.0 tok/s for DeepSeek-1017, both on TensorFold.
- Quality. NVIDIA's 4-bit GLM-5.3-Flash scores within a point of BF16 on its two published tests. For DeepSeek under 3 bits, builders publish drift figures but no matched benchmark against the full model.
- Engines. TensorFold posts the top single-stream numbers. vLLM has the most recipes. NIM is the only vendor-supported path, for GLM-5.3-Flash on two Sparks.
- The builders' pick. @MiaAI_lab, whose recipes many owners run, recommends either Flash model on two or more Sparks, and would skip the full GLM-5.3.
The models, checked
We checked names and sizes against the model cards. Every builder figure below names its quant, because the same model can differ by 2x in speed between builds.
| Model | Total parameters | Active per token | Context | Released as |
|---|---|---|---|---|
| GLM-5.3-Flash | 320B | 18B | 1M | FP8 |
| GLM-5.3 (full) | 753B | about 40B | 1M (builds here run about 500K) | FP8 |
| DeepSeek-V4.1-Flash-1017 | 763.2B, including 196.6B of Engram tables | about 8B reading, 16B generating | 1M | FP8 dense, FP4 experts |
The DeepSeek count explains why you see "552B" in older posts. That figure is the backbone alone. The 1017 card lists 763,205,315,794 parameters, and the repository is 510.3 GB, or 476 GiB on disk. For comparison, the official GLM-5.3-Flash repository is 328.4 GB and the full GLM-5.3 755.7 GB.
The Engram tables are n-gram lookup memory. A token reads only a few rows, so builders keep the tables on NVMe and fetch rows as needed. That trick is what makes DeepSeek fit on Sparks at all.
The hardware: what two and three Sparks give you
Each Spark has a GB10 chip and 128 GB of unified memory at 273 GB/s. The recipes we read treat about 121.7 GiB per box as usable. Each box has two QSFP ports at 200 Gb/s each.
NVIDIA documents three layouts: a direct cable for two, a three-cable ring for three, and a switch for more. Our arithmetic puts 200 Gb/s at about 25 GB/s. That is roughly a tenth of each box's memory speed.
That gap shapes every recipe. Tensor parallel decoding works because each box keeps its own share of the weights, and only small activations cross the cable.
At $6,950 a box, the price @sudoingX quoted on October 19, two Sparks cost about $13,900 and three about $20,850. That is our arithmetic, before cables.
Three is the odd one out. Both models have 64 attention heads, which split evenly across two or four boxes but not three. Builders pad the model with dummy heads to make it divide.
"3 is a genuinely awkward middle: it works... but you pay in patch surface and you get memory rather than speed." (jc2375, on the NVIDIA developer forum)
Memory: what fits
| Build | Weights | Two Sparks | Three Sparks | Source |
|---|---|---|---|---|
| GLM-5.3-Flash, official FP8 | 305.8 GiB on disk (328.4 GB) | No | On paper, about 59 GiB spare; no report found | Hugging Face repo size, our arithmetic |
| GLM-5.3-Flash, NVIDIA NVFP4 | 181 GiB | Yes | Yes | kindling repo |
| GLM-5.3-Flash, NVIDIA FP4-Local | about 166 GiB in use | Yes, validated by NVIDIA | Yes | NVIDIA model card |
| GLM-5.3-Flash, EXL3 4 bpw | 175.7 GB (163.6 GiB) | Yes | Yes | Mia's AI Lab |
| GLM-5.3 full, official FP8 | 703.8 GiB on disk (755.7 GB) | No | No | Hugging Face repo size |
| GLM-5.3 full, EXL3 2.75 bpw | about 273 GiB | No | Yes, about 86 GiB a box | Mia's AI Lab |
| GLM-5.3 full, EXL3 3.38 bpw | 319 GB | No | No | @ViC305 (four Sparks) |
| DeepSeek-1017, official weights | 476 GiB on disk; about 305 GiB in memory with Engram on NVMe | No (145 GiB a box) | Yes (about 101 GiB a box) | Mia's DeepSeek repo |
| DeepSeek-1017, EXL3 2.9 bpw | 196 GiB, plus about 190 GiB of Engram on disk | Yes | Yes | Mia's 2-Spark repo |
| DeepSeek-1017, 2.70-bit (vLLM) | 196 GB | Yes | Yes | @sudoingX |
| DeepSeek V4.1 Flash, GGUF Q2_K | 246.3 GiB | Over the line | Yes, on a fork | vcruz305 card |
A rough rule for your own estimates: parameters in billions, times bits per weight, divided by eight, gives gigabytes. Add the parts kept at 16 bits, the KV cache and about 6 GiB of headroom per box.
Speed: what builders measured
Every figure below is a builder's own measurement, with their prompts and settings. Treat cross-build comparisons as rough. "Prose" is single-stream decode on ordinary text, the slowest and most comparable case.
Two Sparks
| Model and build | Engine | Prose, one stream | Code or structured | Several at once | Prefill | Context |
|---|---|---|---|---|---|---|
| DeepSeek-1017, EXL3 2.9 bpw | TensorFold | 65.0 | 72.0 code | 98.4 at 4 | 2,124 at 32K | 524,288 |
| DeepSeek-1017, 2.70-bit | vLLM | 59.2 | 100.7 code, 141.3 structured | |||
| DeepSeek-1017, EXL3 2.9 bpw, 1M build | TensorFold vs vLLM kit | 56.8 vs 27.0 | 92.2 vs 39.9 code | 1M, needle passed at 1,035,020 | ||
| GLM-5.3-Flash, EXL3 4 bpw | TensorFold | 60.4 | 114.7 structured | 108.8 at 4, 130.8 at 8 | 1,978.9 at 32K | 1,048,576 |
| GLM-5.3-Flash, NVIDIA NVFP4 | vLLM (kindling) | 42.6 | 66.2 code, 94.7 structured | 116 at 8, mixed | 2,931 at 32K | longest request 160K |
| GLM-5.3-Flash, NVIDIA NVFP4 | vLLM (knapcio stack) | 35.3 | 68.9 code | 262,144 | ||
| GLM-5.3-Flash, NVFP4-MXFP8 | Flare | 39.4, mean of prompt types | 107 at 8 | 2,276 | 570K prompt answered | |
| GLM-5.3-Flash, NVIDIA NVFP4 | Atlas fork (SparkGLM) | 36.3 | 65.7 code | 2,760 |
The top DeepSeek row comes from @niklaslenz_ai, measuring @MiaAI_lab's October 21 recipe. Prose ran about 5% faster than her October 19 version.
Three Sparks
| Model and build | Engine | Prose, one stream | Code or structured | Several at once | Prefill | Context |
|---|---|---|---|---|---|---|
| DeepSeek-1017, official weights | vLLM (spark-ds41f) | 59.7, the repo's single-stream test | 116.0 at 4, 152.7 at 8 | 4,465 at 32K | 1M | |
| DeepSeek-1017, official weights | SGLang (Mia) | 51.0 | 85.4 at 4 | about 2,000 | 256K set; about 32K usable on the head node | |
| GLM-5.3-Flash, EXL3 4 bpw | TensorFold | 77.6 | 146.2 at 4 | 1M | ||
| GLM-5.3-Flash, NVIDIA NVFP4 | vLLM (kindling) | 54.2 | 88.0 code, 121.4 structured | 146 at 8, mixed | 3,864 at 32K | longest request 524K |
| GLM-5.3-Flash, NVIDIA NVFP4 | vLLM (jetnet) | 35.2, decode | about 1,800 | 471,813-token retrieval | ||
| GLM-5.3 full, EXL3 2.75 bpw | TensorFold | 31.3 | 41.7 code | 48.2 at 4, with 32K windows | 591.2 at 32K | 499,712 |
For the full GLM-5.3, sampled replies run slower than those figures, since fewer speculative drafts survive. Mia's page gives 22.9 to 24.0 tok/s on prose with sampling.
What a third Spark buys
For GLM-5.3-Flash on TensorFold, the third box lifts one-stream prose from 60.4 to 77.6 tok/s, about 28% by our arithmetic. On kindling's vLLM build, the KV pool grows from 1.38M to 2.24M tokens and the longest request from 160K to 524K.
For DeepSeek, the third box is the difference between a 2.9-bit quant and the official weights. It does not buy speed: the best three-Spark single stream we found trails the best two-Spark one.
Quality: how close the quants stay
| Build | Published check | What it shows |
|---|---|---|
| GLM-5.3-Flash, NVIDIA FP4-Local | GPQA Diamond 91.3% vs 92.1% BF16; ToolEvalBench 92.4% vs 91.9% | Within a point on both |
| GLM-5.3-Flash, EXL3 4 bpw (Mia) | KL 0.0903 against Z.ai's FP8; GSM8K 98.8%, HumanEval 95.7% | Small drift; no FP8 score given beside it |
| GLM-5.3-Flash, NVFP4-MXFP8 on Flare | GSM8K 193 of 200; the same checkpoint on one GB300 scored 194 and 196 | Same quant, different hardware |
| GLM-5.3 full, EXL3 3.38 bpw (four Sparks) | KL 0.082 against BF16, 91.15% top-1 agreement | The best-documented full-model quant |
| DeepSeek-1017, 2.70-bit (@sudoingX) | KL 0.0243 from native 1017 | 20% closer than an earlier 2.9-bit pack, he says |
| DeepSeek-1017, EXL3 2.9 bpw (Mia) | QA 94.0%, MMLU 91.9%, Tool Eval 93/100 in sparkDash | Absolute scores; no full-precision baseline in the same run |
| DeepSeek-1017, EXL3 3.75 bpw, three Sparks | KL 4% to 20% better than 3.5 bpw | New today from @easthambuilds |
One quality trap is documented. NVFP4 builds that also quantize attention emit intermittent corrupted tokens, the tonyd2wild repo warns, citing a vLLM issue. NVIDIA's own NVFP4 checkpoint leaves attention unquantized and showed no corruption in the repo's tests.
What owners say
- @MiaAI_lab: "DeepSeek is better at some tasks, while GLM is more consistent and nicer to talk to." She calls the full GLM-5.3 about equal to DeepSeek V4.1 Flash, and much slower.
- @HandwerkAiLab: on German business texts with thinking off, DeepSeek-1017 scored 98.1% on three Sparks and GLM-5.3-Flash 93.3%, in single runs. "But for daily use throughput and stability count more, and GLM is easier to run."
- @GaryIngle77: on three Sparks, 1017 was sharper on facts than the September weights but paraphrased more. The original quoted 31 of his 32 messages verbatim; 1017 quoted 12.
Engines: which to run
| Engine | Multi-Spark | Formats used here | GB10 status | For | Against |
|---|---|---|---|---|---|
| TensorFold | Yes, 2 to 4 boxes | EXL3 | Community engine in Zig; Mia's GLM recipe adds 96 patches | Fastest single stream, 1M context, one-command setup | Young project; DeepSeek support sits on branches |
| vLLM | Yes, tensor parallel across boxes | NVFP4, FP8, EXL3 via overlay | Community images (eugr, Local Inference Lab) | Most recipes, best concurrency, OpenAI API | Upstream lacks pieces; three boxes need padding patches |
| SGLang | Yes | Official MXFP4 and FP8 | Mia's DeepSeek recipes on three and four Sparks | Runs DeepSeek's own weights | Tight memory on three boxes limits context |
| NVIDIA NIM | Two Sparks | NVFP4 | Officially supported for GLM-5.3-Flash | Vendor support, container ready | One model and one layout; no speed figure published |
| NVIDIA Flare | Two Sparks; a patch claims three | NVFP4, FP8 KV | New in October, Rust | Long prompts, many users | No EXL3; Mia measured it slower than posted claims |
| llama.cpp | RPC across boxes | GGUF | GLM-5.3-Flash merged September 30 | Broad hardware, simple | DeepSeek V4.1 needs a fork; no multi-Spark GLM report found |
| TensorRT-LLM | No recipe found for these models | NVFP4, FP8 | NVIDIA ships a Spark playbook | NVIDIA's own kernels | No Spark recipe found for either model |
| Ollama | One box | GGUF | Runs on Spark | Easiest start | No multi-node, so neither model fits |
| EXO | Clusters of devices | Varies | No report found | No report found for either model |
Picking by build
- GLM-5.3-Flash on two Sparks. TensorFold with the EXL3 4 bpw quant for speed and 1M context. NIM or NVIDIA's FP4-Local on vLLM if you want a vendor-backed path.
- DeepSeek-1017 on two Sparks. TensorFold with Mia's EXL3 2.9 bpw recipe, or @sudoingX's 2.70-bit build on vLLM for structured output.
- DeepSeek-1017 on three Sparks. vLLM via spark-ds41f for 1M context and concurrency, or SGLang via Mia's recipe. Both run the official weights.
- Full GLM-5.3 on three Sparks. TensorFold with the 2.75 bpw quant is the only recipe we found.
The same level elsewhere
Other models at these sizes, from builders' posts:
- One Spark: Qwen3.8-Flash-Next, @MiaAI_lab's pick, and Ornith-1.5-122B at 4 bpw.
- Two Sparks: the two Flash models above. Ling 3.1 Flash, a 560B model, did not fit for @sudoingX.
- Four Sparks: the full GLM-5.3 at 3.38 bpw, DeepSeek-1017 at 87.7 tok/s prose on SGLang, and Ling 3.1 Flash at Q4 over llama.cpp RPC, which he called "super slow."
Other hardware at the same memory level:
| Hardware | Memory | Reported result | Source |
|---|---|---|---|
| 2x RTX PRO 6000 Blackwell | 192 GB | GLM-5.3-Flash FP4-Local validated with 1.8M tokens of FP8 KV | NVIDIA model card |
| Mac Studio M5 Ultra | 256 GB | GLM-5.3-Flash MLX 4-bit: 2,331 tok/s prefill and 164.3 decode at 128K, 181.8 GB peak | oMLX benchmark |
| Mac Studio M5 Ultra | 256 GB | DeepSeek-1017 on TensorFold Metal: 70.7 prose, 104.9 code, 1M context | BobClawblaw repo |
| DGX Station GB300 | not checked here | DeepSeek-1017 to 1M context, 2,276.6 tok/s across 32 users | @SJRonanMD |
The oMLX decode figure varies with context: the same session lists 84.2 tok/s at 8K and 164.3 at 128K, on a code prompt with speculative decoding. We did not find a published price for the M5 Ultra configuration.
Sources
Model cards and official documentation
- Z.ai: GLM-5.3-Flash on Hugging Face (320B total, 18B active)
- Z.ai: GLM-5.3 on Hugging Face
- DeepSeek: DeepSeek-V4.1-Flash-1017 on Hugging Face (parameter count, size, license)
- NVIDIA: GLM-5.3-Flash-FP4-Local model card (166 GiB, validated hardware, GPQA and ToolEvalBench)
- NVIDIA: GLM-5.3-Flash-NVFP4
- NVIDIA: DGX Spark clustering guide (ports, two-, three- and four-box layouts)
- NVIDIA: Connect three Sparks in a ring
- NVIDIA NIM: support matrix and Deploy on DGX Spark
- NVIDIA: TensorRT-LLM Spark playbook
Recipes and measurements
- Mia's AI Lab: GLM-5.3-Flash on two to four Sparks with TensorFold (speeds, KL, benchmarks, 175.7 GB)
- Mia's AI Lab: GLM-5.3 on three Sparks with TensorFold (273 GiB, 31.3 tok/s, 499,712 context)
- MiaAI-Lab: DeepSeek-V4.1-Flash on 3x and 4x DGX Sparks (SGLang, memory per rank, speeds)
- MiaAI-Lab: DeepSeek v4.1 Flash EXL3 2.9 bpw for 2x DGX Sparks (196 GiB, Engram on disk, memory guard)
- kindlingai: glm-5.3-flash-gx10 (vLLM at TP 2, 3, 4 and 6 on GB10)
- tonyd2wild: GLM-5.3-Flash NVFP4 on 2x DGX Spark (knapcio stack, corrupted-token warning)
- jetnet: GLM-5.3-Flash NVFP4 at TP3 and the NVIDIA forum thread (padding, kernel fallback, jc2375 quote)
- FlyCockpit: GLM-5.3-Flash on 3x DGX Sparks
- christopherowen: spark-ds41f (DeepSeek-1017 at TP3 with 1M context)
- NVIDIA forum: DeepSeek-1017 EXL3 2.9 bpw on 2x Spark, TensorFold vs the vLLM kit
- al-engr: GLM-5.3-Flash on Flare across two Sparks
- eugr: spark-vllm-docker (community vLLM images)
- vcruz305: DeepSeek-V4.1-Flash GGUF (Q2_K and Q3_K_M sizes, upstream status)
- llama.cpp PR 27773: GLM-5.3-Flash support and PR 28696: DeepSeek V4.1 conversion
- oMLX: GLM-5.3-Flash MLX 4-bit on M5 Ultra
- BobClawblaw: DeepSeek-V4.1-Flash-1017 on one M5 Ultra
- Flowtivity: GLM 5.3 Flash vs DeepSeek V4.1 Flash on dual DGX Spark (September 13; vendor benchmark table)
On X
- @MiaAI_lab's best models list, on skipping the full GLM-5.3, on Flare and on a two-Spark GLM-5.3
- @niklaslenz_ai's two-Spark DeepSeek-1017 numbers
- @sudoingX's 2.70-bit 1017 build, Ling 3.1 Flash on four Sparks and the $6,950 price and Ling 3.1 Flash not fitting two
- @ViC305's 3.38 bpw GLM-5.3
- @easthambuilds's 3.75 bpw DeepSeek-1017 for three Sparks
- @HandwerkAiLab on German business texts
- @GaryIngle77 on 1017 against the original
- @SJRonanMD on one DGX Station