@simonw Suspects Prompt Injection Is Fading in the Strongest Models

A cautious shift from the person who named the problem.

Replying to @nateberkopec, @simonw says he is cautiously starting to suspect prompt injection may not be a problem with today’s strongest models. Anthropic in particular is very confident about its ability to avoid prompt injections, he writes. It is still a real concern for cheaper, weaker models.

“The bigger problem is /proving/ it,” he adds. He is not sure how to reach full confidence that a model or agent harness has solved it, to the point that we no longer have to worry.

@emollick, who calls Willison the inventor of the term, suspects the same. His own tests suggest you can’t simply prompt inject a request for higher grades into a paper anymore.