DeepSeek 4.1 Flash mogs OpenAI on our PR Security Review benchmark!
Summary
We ran our PR security-review benchmark across 16 repos, each with a planted vulnerability over the latest models. DeepSeek 4.1 Flash is our pick for day-to-day review: 64% of the planted bugs found, about 6 minutes a PR. We're moving our workloads over to it. Although OpenAI GPT6 Astra finds 5% more vulnerabilities, it is pricey and tries to break out of its sandbox.
NOTE: this PR Benchmark is specifically scanning differential code-commits (PRs) and looking for recently introduced vulnerabilities. This is not an exhaustive "full scan" across a code base.




