xAI's Colossus Fleet Grows Past 2 GW of Compute

@_avichawla published a tutorial that reproduces Jev's inference pattern on open models: instead of generating text, the server reads the logits of single-token labels and returns a probability distribution over known answers through SGLang's /v1/score. In his run against Qwen2.5-0.5B-Instruct, a billing ticket scored 0.67776233 for billing, 0.310878605 for technical support and 0.011359035 for account access. The demo covers Qwen 3 4B, SmolLM2 1.7B, TinyLlama 1.1B and DeepSeek-R1-Distill-Qwen 1.5B. Jev itself stays closed source; the post recreates the inference path only.

A discussion in the open-webui repository lays out a complete, source-verified API reference: management endpoints under /api, files and knowledge bases under /api/v1, a stateless OpenAI-compatible layer at /v1, and native Ollama passthrough at /ollama. It documents bearer-token auth, the /api/chat/completions flow with chat_id association, background task flags, and a curl example for the full chat session workflow. A commenter, @techelpr, adds a correction: create-chat payloads need a "chat" wrapper object.

@0xBakeer asks which he would run at equal speed: Qwen3.8-27B in NVFP4 or Qwen3.8-Flash-Next in EXL3 at 2-bit, both at 50 tok/s. @jaita picks NVFP4 for concurrency and EXL3 for single streams. @jtdavies reports a week of customer data made Flash-Next the clear winner. @niklaslenz_ai counters that the 27B won a lot of tasks in his test against Flash running on a Spark.

@JoelDeTeves ran the Pagoda test on @ViC305's EXL3 single-Spark recipe with @turboderp_'s Qwen3.8-Flash-Next at 4.05 bits per weight. It took 3 shots, on minor syntax errors, and the work itself was mostly done in a single shot, around 18 minutes, similar to @WescheNex1q's result. He credits the detail level: this one added a small worker who walks around the grounds. In the replies he confirms it runs on a single DGX Spark.

@TeksEdge reports Qwen3-TTS can clone a voice from about a 3-second reference and now runs in llama.cpp, including Windows with Nvidia CUDA. A user running the larger 1.7B model on an RTX 5050 measured about 7.1GB VRAM at default context and about 2.5GB at 4096 context, with no measured speed hit in that test. @TeksEdge is now looking at the 0.6B Base model.