LangChain Finds Jev Beats LLM Judges on Consistency

Dropped into Hermes, a browser policy that scored 30/30 and 29/30 zero-shot fell to 0/30: with tool_choice=auto the controller kept writing text instead of calling the tool. Forced, the interactive work went 22/22. A deterministic rule split static from interactive work at 60/60, while Laya managed 73% and sent interactive jobs down the wrong path. @Blackwellboy kept free rules for routing and SemIf for fast browser choices.

TypeSafe released Jev on Tuesday; by Sunday @0xBakeer had an open copy running. The model is Laya, Convai Innovations’ open 421M encoder trained against proper scoring rules, 2.4GB across three checkpoints. His arbiter server answers one question in 20.6 ms on the Spark and 30.3 ms on an M2 Max, batching to 295 questions per second. Tuned on 117 labelled Claude Code events, his shell-command hook passes all 52 safe commands and hard-denies 29 of 32 dangerous ones.

@TeksEdge reports Bespoke Labs released Nimble, an open 9B decision model from Qwen3.5-9B, LoRA-tuned on 2,676 examples with no Jev distillation and no RL, reportedly built in one day. On Bespoke’s held-out 324-example test, Qwen3.5-9B scored 66.36%, Qwen3.8-27B 84.88%, Nimble-9B 90.12%, and Jev 93.21%. Median latency runs about 106 ms on an H100 and about 444 ms on an M5 Pro 64GB. Bespoke calls the eval narrow and synthetic.

@Sentdex split a turn-based Halite 2 bot in two: GLM 5.3 Flash judges which planets matter and when to re-plan, while Jev handles ship-by-ship execution. The hybrid ran 13x faster at 56% of the pure GLM API cost with a slight performance edge, and GLM alone beats Jev alone 82% of the time across 133 unique maps. He calls himself less skeptical of Jev and plans robotics experiments next.

@onusoz ranks DeepSeek V4.1 Flash and GLM 5.3 Flash as the two open-weight models dominating the remote-inference Pareto frontier, ahead of GPT 5.6 Luna on quality and speed. Profiling Hugging Face inference providers, he puts DeepSeek above 200 median tok/s on Baseten with GLM around 50-60% of that, and credits DeepSeek’s more efficient KV cache. The question he leaves open: whether OpenAI’s next Luna, expected next week, can beat 890 bytes per KV cache token.

@MiaAI_lab surveys what Spark owners run: Qwen3.8 Flash Next solo; GLM 5.3 Flash or DeepSeek v4.1 Flash on two; GLM as orchestrator over Qwen workers on three or four; and, at six or more, GLM orchestrating, Qwen 3.8 Flash as worker, and DeepSeek v4 Flash as security scanner. She runs GLM 5.3 Flash on all three of her own Sparks, mainly for a bigger KV cache pool.

@natebjones sorts the wave into four patterns: @nbarreto classifying 70,000 insurance support tickets; @iannuttall asking 8 questions of 3,282 posts; @joshest3p deciding where deeper reasoning is worth spending; and @DanRWilloughby linting prose with 10 yes-or-no questions per paragraph. @coreyganim’s thread adds @iam_zachi’s PostgreSQL extension judging 129 rows in about 1 second for $0.0009 and @johnyeo_’s Slack agent, 2x faster once Jev picks the tool. @natebjones calls the pattern semi-determinism, a shim between complex inputs and a defined set of outputs.

@brianchau57 lays out effort.news reporting that, in partnership with Irregular, AI companies instructed unsecured versions of their models to hack targets called “flags”, and that some models were accidentally given internet access and reached real companies. He writes that Anthropic’s own logs debunk the rogue-agent theory because models obeyed when told not to access the internet, and that Irregular co-founder and CTO Omer Nevo sits on the board of EA Israel.

@loktar00 asked a Dell rep for a discount on an RTX Pro 4500, got a call-back offering the card at $5,200, and watched the price climb more than $2,000 in two hours. @xchchiamining reports an Asus GX10 going from about $5,500 to $8,299 in one morning, and @loktar00 notes Intel B70 boards are up a few hundred from $999. @HackerTwins notes Dell workstations under $60,000 ship with four RTX Pro 6000 cards at about $12,500 each.