DeepSeek Open-Sources DeepInfer, Its Production Inference Engine

The stack behind DeepSeek V4.1 Flash is public; for now it runs on NVIDIA SM100 and Huawei Ascend 950.

DeepSeek has published the inference engine that serves its own models.

DeepSeek open-sourced DeepInfer, the inference engine behind its own production servers, @MiaAI_lab reports. DeepSeek describes it as the in-house framework “powering DeepSeek-V4.1-Flash at scale.”

@MiaAI_lab lists what it does:

Feature	What it does
Draft tokens	Faster output
Custom kernels	Fast inference
Long-prompt reuse	Skips repeated work on long prompts
Batch planning	Plans the next batch while the GPU is still busy
Batch invariance	Same answer no matter what else is in the batch
Determinism	Same input gives the same output every time

Supported hardware for now: NVIDIA SM100 and Huawei Ascend 950.

For now, it can only be used on NVIDIA SM100 and Huawei Ascend 950. In a reply, Mia said it does not support DGX Sparks “as of now.”

Weights, harness and now the engine.

@kimmonismus calls the release a big deal: “model weights alone don’t make AI cheap to run. But *this* helps.” He also notes it runs on Huawei chips, not just Nvidia.

@MiaAI_lab sums up DeepSeek’s position: a top of the line open model, a top of the line harness, “and now top of the line inference engine.” All open source. “Thumbs up for them, I love it.”