One Station Plus a Sidecar Beats Two Stations at 16 Users
One Station with an RTX PRO 6000 sidecar beats two Stations splitting DeepSeek-V4.1-Flash at 16 users.
@SJRonanMD ran one benchmark, an 8K random-token prompt with 1K tokens out, three ways on DeepSeek-V4.1-Flash. One DGX Station with an RTX PRO 6000 sidecar in its open slot served 1,807 to 1,995 tok/s at 16 users. The best published number for two Stations splitting one model is 1,678, from a different software build. At 64 users the pair pulls ahead, 3,402 against 3,047 to 3,160, and two full copies top both at 5,774.
Splitting costs network round trips. On DeepSeek-V4-Pro, one generated token needed 123 combine steps of about 20 microseconds each. The messages run about 14 KB over a link moving 392 gigabits per second, about 2.3 ms of pure transfer per token. The sidecar pays almost none of that: only 0.75 ms of a 22.6 ms step was exposed. @SJRonanMD’s reading: at 64 users each step carries more work, the tax spreads across all of it, and two GB300s bring twice the compute and HBM.
Splitting wins when a model does not fit one Station’s fast memory. NVIDIA’s 550B IOI coding model ran 39.8 tok/s for one user on one Station, with 100 GiB of experts in Grace memory. Split across two, every weight fits in HBM and it runs 99.1, 2.5x, 6.1x at 32 users, though part of that gain came from a faster MoE kernel. DeepSeek-V4-Pro, at 1.6 trillion parameters, does not fit one Station at released precision. Every number is in the public repo; the split figures are catid’s published benchmarks.