We have run this scan on Claude models before, comparing three generations against each other and finding no meaningful difference. The natural next question is cross-vendor: is OpenAI's Codex any better or worse than Claude at writing secure code? This is the first study we have seen that actually measures it on real apps, so here is the answer.
How we sourced Codex-built apps
Codex, like Claude Code, stamps its commits with a co-author trailer, Co-authored-by: Codex <[email protected]>. The OpenAI email makes it an unambiguous fingerprint: it identifies genuine Codex commits, not a random GitHub user who happens to be named codex. We searched for repositories majority-authored by Codex, filtered to actual web applications (a scope rule fixed before any results were seen), and scanned every one with Prbl, the same scanner, same rules, same reviewer used on the Claude cohorts.
Wondering if your code has any of this? Scan a public repo free, no account needed.
Scan a repo →The result
| Model | Vendor | Web apps | High-severity | 95% CI |
|---|---|---|---|---|
| Codex | OpenAI | 83 | 9.6% (8) | ±6.3 pts |
| Claude Fable 5 | Anthropic | 117 | 12.0% (14) | ±5.9 pts |
| Claude Opus 4.8 | Anthropic | 142 | 15.5% (22) | ±6.0 pts |
| Claude Opus 4.5 | Anthropic | 92 | 16.3% (15) | ±7.5 pts |
Codex came in at 9.6%, actually the lowest point estimate of the four, with the Claude models spread from 12.0% to 16.3%. But every one of those gaps sits inside the confidence intervals, which overlap heavily. We ran a significance test on Codex against each Claude model and none came close: the p-values were 0.60, 0.21, and 0.19, all far above the 0.05 threshold that would let you call a difference real. In plain terms, these are the same number. No model in this study writes measurably more or less secure code than any other.
How this number survived scrutiny
This one is worth showing, because it is a case study in why the raw output of a scanner is not a finding until a human has checked it. The first Codex sample we pulled was small, 24 apps, and it came back at 25%, roughly double Claude. That would have been a sensational headline: “Codex writes twice as many security holes as Claude.” We did not publish it, for two reasons.
First, 24 apps is far too few. The significance test on that sample was p = 0.12, nowhere near real. So we widened the search and grew the Codex sample to 83 apps. The rate fell from 25% to 14.5%. A number that moves ten points when you add apps is noise, not a finding.
Second, when we reviewed the surviving Codex findings by hand, many were false positives specific to the kind of code Codex tends to write. Codex is used heavily in machine-learning and AI-tooling repositories, which are full of strings like api_key="not-used-by-llama-server", "local-test-key", and status labels like "(configured)", none of which are real credentials. Our scanner was counting them as hardcoded secrets. We fixed those patterns (with regression tests), and to keep the comparison fair we re-scanned all four cohorts through the corrected scanner. That dropped Codex to its true 9.6% and barely moved the Claude numbers, because those false positives were concentrated in Codex's corpus.
The scary version of this story, “Codex is twice as dangerous as Claude,” was available at three separate points, and was wrong every time. The real finding is quieter and holds up: they are the same.
Why the tool does not predict the outcome
The reason is the same one we keep arriving at. The flaws that dominate every cohort, in this study and the last, are hardcoded credentials, weak randomness in a security-sensitive spot, and disabled TLS verification. None of those is a reasoning failure that a better model avoids. They are the result of a human pasting a value inline to make something work and meaning to fix it later. Codex and Claude are both good at writing the application. Neither is the one deciding to commit the key, so neither moving to the other changes the rate at which it happens.
What this means for you
If you were hoping this post would tell you which AI tool to use for safer code, the honest answer is that the choice does not matter for this, and picking a tool is not a security control. What moves the number is the review the code still needs before it ships, and that is constant no matter whose model wrote it. Before a repo goes public: pull secrets into environment variables and rotate anything ever committed, use a cryptographic random source for anything that guards access, and never ship with certificate verification turned off.
Methodology and limits
434 web apps: 83 built with OpenAI Codex and 351 with Claude (Fable 5, Opus 4.8, Opus 4.5), each sourced from GitHub via the model's own commit trailer, filtered to web applications by a rule fixed before results, and scanned with Prbl. Every high-severity finding was manually reviewed, and the false-positive patterns that review surfaced were fixed in the scanner and all cohorts re-scanned before these numbers were computed. The honest limits: this is observational, not a controlled experiment; the Codex sample (83) is smaller than the Claude cohorts; and users of each tool are different populations. No specific repository, owner, or file is identified anywhere in this post.
Every few months a new coding model launches and the discourse asks whether it is the safe one. We now have four of them measured the same way, across the two biggest vendors, and the answer has not changed. The model keeps getting better at writing code. The rate at which that code ships a secret it should not is set by something the model does not touch.