DeepSeek-1017 Runs on One 12 GB GPU and 64 GB of RAM With Strata
ISTA-DASLab’s 2-bit GGUFs keep hot experts in RAM and the rest on SSD; a 5090 owner reports about 14 tok/s.
A 763B model on a gaming PC, if you can wait for it.
@coldniko says DeepSeek-V4.1-Flash now runs on 64 GB of RAM with Strata, using GSQ-RCO quantizations by @mhelcig. Strata 0.2.4’s release notes say it runs on a PC with one 12 GB graphics card and 46 to 64 GB of RAM. The engine keeps the experts it uses most in RAM and streams the rest from SSD.
| Build | Bits per weight | Total size |
|---|---|---|
| IQ1_M | 1.75 | 231.32 GB |
| IQ2_XXS | 2.00 | 248.84 GB |
| IQ2_XS | 2.25 | 265.83 GB |
The quants come from ISTA-DASLab, which gives each tensor its own quantization type under a size budget. Its card says the IQ2_XXS build matches the base model on all three of its benchmarks, and that llama.cpp did not support DeepSeek-V4.1 as of Oct. 23.
@jun1228909 tried the IQ2_XXS build on one RTX 5090 with 128 GB of DDR5 under Windows 11, writing in Japanese. With reasoning off, a Minecraft-style game took 2h 01m at about 14 tok/s. With reasoning on high, a second game took 5h 06m at about 8 tok/s, with steadier quality, he says.
Sources
- @coldniko on X · Oct 23, 8:30 PM PT
- @jun1228909 on X · Oct 24, 1:22 AM PT
- @jun1228909 on X · Oct 24, 6:42 AM PT
- Strata releases · github.com
- ISTA-DASLab GSQ-RCO GGUF card · huggingface.co