The Case for Gaming GPUs and Dense Models

@sudoingX argues that the door to local AI is a 12, 16 or 24GB gaming GPU that many people already own. He points out that the RTX 3060 is still the most common desktop GPU on Steam, that 16GB is now the most common amount of VRAM (26.9% of users), and that 65.7% of Steam users have between 8 and 16GB.

He lists what he has measured at each tier: a 27B model with 48k context on 8GB; the same 27B at 50.8 tokens per second with 131k context on 12GB; 262k context with vision on 16GB; and Qwen 3.8 27B Q4 on 24GB. GPU prices are “dumb right now,” he concedes, but he calls a card in that range the cheapest seat in the race. He adds that he watched Opus 5.5 get cut off for reading Anthropic’s own blog, while his local model has never cut him off and runs with the Wi-Fi off.

He also questions the week’s run of mixture-of-experts releases. In his view the last good dense model that fits on a decent GPU was Qwen 3.8 27B in August, and before that Gemma 4 31B in April. The 20-30B dense class, “where small GPUs live,” has gone quiet, even though dense models pay off on small GPUs. He has asked open labs directly for a 12B dense model for the RTX 3060 and 3090 crowd.

NVIDIA, he says, is “not backing down.” Its Nemotron 3.5 Bolt is a 74B mixture-of-experts model designed for RTX Sparks, and he is asking anyone who has run it on the new Windows RTX Spark what speeds they are getting.