All posts

Field guide

How to review AI-generated code for security before you merge

Your assistant optimizes for code that runs, which is not the same as code that is safe. The good news is that it fails in predictable ways, so a short, repeatable review pass over five weaknesses catches most of it in the diff. Here is the pass, and the exact question to ask at each step.

Last updated

Start from the right assumption: the code your assistant just wrote is probably functional and possibly insecure, and those two facts are unrelated. It reaches for whatever makes the current step run, which is why the same handful of weaknesses show up again and again in AI-generated code, regardless of which tool wrote it. That predictability is the opening. You do not need to audit everything; you need to look for five specific things, and you can learn to spot them in a diff in under a minute.

1. Secrets: did a credential land in the code?

The most common high-severity issue, in every set of apps we have scanned. When an integration needs a key, the assistant tends to inline the literal value because that is the shortest path to working code, and that value then ships to the repo and sometimes to the browser. Scan the diff for anything that looks like a key, a token, or a connection string sitting as a string literal, and for a new .env that is not in .gitignore.

The question to ask: is any secret in this diff a literal value rather than a reference to an environment variable? If yes, move it out and read it from the environment. If it is already committed, rotate it. Background: what a hardcoded secret is, why environment variables exist, and client-side vs server-side secrets.

Want to see exactly what Prbl flags? Watch it scan a demo app, no repo or account needed.

See a live scan →

2. Authorization: does this route check who is asking?

The second most common serious issue, and the easiest to exploit. An assistant builds the login, so authentication is usually present, but the per-request ownership check is not needed to make a feature work, so it gets skipped. The result is a route that looks up a record by an id from the request and returns it without confirming the record belongs to the caller. Change the id, read someone else's data.

The question to ask: for every endpoint that reads or changes data by id, is there a server-side check that the record belongs to the current user? Not a check in the UI, which anyone can bypass by calling the route directly. Background: authentication vs authorization, what IDOR is, and the fix for a missing authorization check.

3. Injection: does untrusted input reach a query, shell, or parser?

Any place where input is glued into something that gets interpreted is a candidate: a SQL string, a shell command, an unsafe deserializer, a template. The assistant writes the interpolated version because it is the obvious one and it works on clean input. It has no signal that the input will ever be hostile.

The question to ask: is user input ever concatenated into a query, a command, or a parser call? If so, switch to the parameterized or safe-by-default form. Fixes for SQL injection, command injection, and unsafe deserialization; concepts behind command injection and insecure deserialization.

4. Insecure defaults: did it turn a protection off to unblock itself?

When the assistant hits an error, the fastest fix is often to disable the thing throwing it. That is how you get TLS verification turned off to silence a certificate error, a permissive CORS wildcard to make a request succeed, or a cookie set without its security flags because they broke local testing. Each one unblocks the step and ships as configuration.

The question to ask: does this diff disable, widen, or skip a security control to make something work? Look for a verification set to false, a wildcard where a specific value belongs, or a missing flag. Fixes for disabled TLS verification, a CORS wildcard, and insecure session cookie flags.

5. Data exposure: does anything leak to the client or the logs?

The last pass is about where data ends up. A secret referenced in a client component or prefixed for the browser is public no matter where you stored it. A token kept in localStorage can be read by any script on the page. An error handler that returns a stack trace hands an attacker your file paths and query structure. None of these break the app, which is why they survive to production.

The question to ask: does this diff put a secret, token, or internal detail somewhere the client or an attacker can read it? Fixes for a JWT stored in localStorage and a leaked stack trace.

The pass, in order

  • Secrets — any literal key, token, or connection string? Any committed .env?
  • Authorization — does every data route check ownership on the server?
  • Injection — is input concatenated into a query, shell, or parser?
  • Insecure defaults — is a protection turned off, widened, or skipped?
  • Data exposure — does anything reach the client or the logs that should not?

Five questions, same order every time. Once it is a habit it costs a minute on a normal diff, and it catches the issues that a functional test will never fail on.

Frequently asked questions

Can't I just trust the AI if the code works and tests pass?

No, because the weaknesses that matter here do not break the feature and do not fail a functional test. A route with no authorization returns the right data to the right user in every test you wrote; it only misbehaves when a different user changes an id. Working and passing tests tell you the happy path is correct, not that the unhappy path is safe. Security review is a separate pass with a different question.

How is this different from running a linter?

A linter checks style and some correctness patterns; it does not reason about whether an endpoint checks ownership or whether a secret reached the browser. A security scanner is closer, and it is faster than reading every diff by hand, but the review habit still matters because you know your app's authorization rules and the scanner does not. Use both: the habit for intent, the scan for coverage.

Which of the five should I check first?

Secrets and authorization, in that order. A leaked secret is the most common high-severity finding across every set of AI-built apps we have scanned, and it is scraped within minutes of a public push. Missing authorization is the next most common and the easiest to exploit, since it needs nothing more than changing an id in a request. If you only have time for two checks, make them these.

Does this apply to autonomous agents too, not just autocomplete?

Even more so. An agent that edits many files and runs commands in one task spreads the same weaknesses across a larger surface before you see any of it, so the review pass moves to the pull request or the diff the agent produces. Treat an agent's output like a contribution from someone you have not met: useful, and reviewed before it merges.

Where a scan fits

The review habit is for intent: you know your app's rules and can judge whether a check belongs. A scan is for coverage, reading the whole repo the way an attacker's bot does and handing you the file and line for each of these five classes, including the ones that slipped past a tired review. We have scanned nearly 2,000 AI-built apps, and one in eight shipped a high-severity flaw from exactly this list. Run a free scan on a repo and see which of the five are already in it.

Prbl scans the AI-generated parts of your codebase for exactly the kinds of issues above.

How to Review AI-Generated Code for Security Before You Merge: Prbl