Models & Hardware
Ling 3.1 Flash Runs on Four Sparks Over llama.cpp RPC
A 560B MoE at Q4 across four GB10s: about $28k of hardware, and “super slow.”
llama.cpp RPC can spread a 560B model over four DGX Sparks. It is slow.
“holy shit, llama.cpp rpc actually works,” @sudoingX posted. The rig is 4x DGX Sparks hosting Ling 3.1 Flash, a 560B MoE, at Q4 across all four GB10s.
| Item | Figure |
|---|---|
| Model | Ling 3.1 Flash, 560B MoE |
| Quant | Q4 GGUF |
| Hardware | 4x DGX Spark (GB10) |
| Price per Spark | $6,950 |
| Total hardware | About $28k |
No one has shipped an official FP4 yet, he says, so he tried the GGUF over RPC. It is possible, and it is “super slow.”
Sunday’s edition covered how he got here: the model would not fit on his 2x DGX Sparks, so he borrowed a 4x Spark setup from @niklaslenz_ai.
Sources
- @sudoingX on X · Oct 19, 4:39 AM PT