Two Sparks Run DeepSeek-1017 With the Drafter on Two RTX 3090s

Move the drafter and the Engram tables off the Sparks, and the freed memory goes to context.

@NicoTheGreco posted a progress report from PLI Labs on a fully local DeepSeek-V4.1-Flash-1017. The target model, an EXL3 build at about 2.75 bits per weight, runs across two DGX Sparks over their direct 200 Gb RoCE link.

Two parts move to an ordinary x86 server with 512 GB of RAM and four RTX 3090s. The DSpark drafter, which proposes up to five tokens per round, runs on two NVLinked 3090s. DeepSeek’s Engram tables, about 189 GiB, sit in the server’s RAM, and each Spark reads rows over its own 40 Gb/s RDMA link. The Sparks still verify every token, so the output keeps the target model’s distribution.

Prompt reading runs about 2,200 tokens a second up to 150K. Three 32K requests at once made about 105 tok/s combined. The production limit is 327,680 tokens per request; a 524,288-token profile works but is held back for a prefill slowdown, and 1M was not tested.