The answer first: there is no safest AI coding model. Across five tools from three vendors, the share of real web apps with at least one high-severity security flaw ranged from 9.8% to 17.7%, and with 8 to 21 flagged apps per group, every confidence interval overlaps every other one. If someone tells you their model of choice writes meaningfully safer code, they are reading a ranking into noise. We would know; we wanted to find one.
How we measured it
Each cohort was sourced from GitHub using a structural fingerprint, not keywords: Claude Code and Cursor both stamp commits with a co-author trailer naming the tool, and our Codex cohort was attributed the same way. We verified the trailer across each repository's history, filtered every cohort to genuine web applications with a rule fixed before any results were seen, and scanned all of them with Prbl under identical rules. Every high-severity finding was reviewed by hand, and when that review surfaced a false-positive pattern, we fixed the scanner and re-scanned all five cohorts so no tool got the benefit of a newer ruleset.
Want to see exactly what Prbl flags? Watch it scan a demo app, no repo or account needed.
See a live scan →The numbers
| Tool | Vendor | Web apps | High-severity flaw rate |
|---|---|---|---|
| OpenAI Codex | OpenAI | 82 | 9.8% |
| Cursor | Anysphere | 120 | 12.5% |
| Claude Fable 5 | Anthropic | 116 | 13.8% |
| Claude Opus 4.8 | Anthropic | 130 | 16.2% |
| Claude Opus 4.5 | Anthropic | 96 | 17.7% |
544 apps in total. The spread looks like a ranking until you do the statistics. The widest gap in the table, Codex against Opus 4.5, does not clear the p < 0.05 bar, and every other pairwise comparison is further from it. The honest reading of this table is one flat line with sampling noise on top.
Negative results are rare in public, because nobody markets a tie. That is exactly why this one is worth publishing: it cannot be cherry-picked into a vendor pitch, ours or anyone else's.
Why the model was never the problem
If security flaws in AI-assisted apps were reasoning failures, the capability curve would bend the flaw rate down. Fable 5 is a genuine leap over Opus 4.5 at writing software, and the two rates are statistically the same. The explanation is in what the findings actually are. Across every cohort, the same short list dominates:
- Hardcoded credentials. A live backend key, JWT, or API token written into a committed file. The single most common high-severity finding for every tool, in every study we have run.
- TLS verification disabled. A database or API connection told to skip certificate checks to get past a local error, then shipped that way.
- Weak randomness.
Math.random()minting tokens, reset codes, or OAuth state values that need a cryptographic source.
None of these are decisions a model makes. They are decisions a person makes to unblock themselves, meaning to fix later, and the tool faithfully commits what the person wrote. A smarter model writes a better app around the leaked key. It does not remove the key.
What this means for the argument everyone is having
The model-safety debate treats the tool as the security variable. Our data says the security variable is the workflow: whether a human (or a scanner) checks the code for the boring, predictable paste-in mistakes before it ships. That check costs the same whether you use Cursor, Codex, or Claude, and skipping it costs the same too. Switching models for security is rearranging the one part of the stack that our measurements say does not matter.
This also means the number to watch in future model releases is not ours. We expect the next generation from every vendor to land in the same band, and we will keep scanning to check. If one ever breaks the pattern, that will be a genuinely big deal, and you will read it here with the confidence intervals attached.
Frequently asked questions
Which AI model writes the most secure code?
None of them, measurably. Across 544 real web apps built with Cursor, OpenAI Codex, and three Claude model generations, the high-severity flaw rate ranged from 9.8% to 17.7% with overlapping confidence intervals and no statistically significant difference between any two tools. Picking a coding tool for security is picking on noise.
Why doesn't a smarter model produce safer code?
Because the flaws that ship are not reasoning failures. In every cohort, the findings were dominated by hardcoded credentials, disabled TLS verification, and weak randomness — shortcuts a human takes to get something working. The model was never the one deciding to commit a live key, so a better model does not stop it.
So what should I do instead of switching models?
Review the code the same way regardless of which tool wrote it. Move every secret to an environment variable, rotate anything that was ever committed, never ship with certificate verification off, and use a cryptographic random source for security-sensitive values. Or scan the repo free with Prbl and get the exact file and line.
Methodology and limits
120 Cursor, 82 Codex, 116 Fable 5, 130 Opus 4.8, and 96 Opus 4.5 web apps, all sourced from GitHub via each tool's commit fingerprint, filtered by a scope rule fixed in advance, scanned with Prbl under identical rules, and manually reviewed. The individual studies with full breakdowns: Cursor, Codex versus Claude, and three Claude generations.
The honest limits: this is observational, not a controlled experiment. The cohorts are different apps by different people, and users of coding agents skew technical, so these rates describe their apps, not all software. Small per-tool differences below our detection threshold could exist; what cannot exist in this data is the large, decision-worthy gap the safest-model argument assumes. And no repository, owner, or file is identified anywhere in this post.
The takeaway fits in a sentence: the tools are tied, the flaws are human, and the fix is a review that costs thirty seconds. Which model you pick can be about anything you like. It just can't be about this.