A scanner that never produces a false positive either hasn't been run against enough real code, or it isn't looking hard enough to be useful. Ours gets run against a lot of real code. Before we published our study of 628 Lovable and Bolt-generated apps, we didn't just run the scanner and count what it found. We manually checked every HIGH-severity finding, one by one, and traced every wrong one back to a specific line of scanner code.
We've done this before, on a smaller scale, when the scanner flagged its own homepage copy as a leaked password. This time the corpus was two orders of magnitude bigger, and the false positives were stranger, because AI-generated apps write code in patterns a human developer rarely would. Here is every fix that shipped this round, with a permanent regression test behind each one.
The 11 fixes
Grouped by what actually broke. Most of these are one narrow suppression pattern, not a rewrite of the rule.
I003 · Code injection
A committed Python virtualenv
One repo had its entire myenv/Lib/site-packages/ folder committed to git. The scanner read pip's own internals and flagged 24 exec/importlib calls inside pip and setuptools as code injection. Fix: site-packages and dist-packages joined the vendor-directory skip list, so any rule ignores anything living inside a committed dependency tree.
R001 · Weak randomness
Simulated blockchain data, two lines up
A repo used Math.random() to fake a transaction hash for a demo UI. The line right above it said // Simulate transaction. Our safe-context check only looked at the current line, so it missed the comment. Fix: widened the check to a 3-line backward window and added simulat\w+ to the safe-context pattern. Cleared 13 findings in one repo alone.
T001 · Path traversal
window.open() is not Python's open()
Our path-traversal sink regex matched bare open( calls, which is correct for Python file I/O and wrong for JavaScript's window.open(url). A Bolt app opening a payment link in a new tab got flagged as a file-system traversal risk. Fix: added a browser-API suppression that recognizes window.open, self.open, and globalThis.open before the sink pattern fires.
C001 · Hardcoded credentials
Enum members read as leaked secrets
TypeScript enums like STATUS: 'active' or a display-name lookup table shaped just like our credential pattern: NAME = 'value'. Our self-referential-enum suppression only caught the = form; camelCase values and the : form used in object literals slipped through. Fix: extended the pattern to match both := assignment styles and camelCase values.
C001 · Hardcoded credentials
Route paths and type discriminators
Two more shapes of the same underlying problem: a Next.js route constant like path: '/dashboard/settings' and a Redux-style type discriminator like type: '.success' both matched the credential-value pattern closely enough to fire. Fix: added dedicated suppressions for route-path values and leading-dot type discriminators.
A002 · JWT decoded without verification
Tokens an app issues to itself
Several apps decode a JWT they just signed themselves, inside the same request, purely to read a claim back out, no external trust boundary involved. That is a legitimate pattern, not a missing-verification bug. Fix: added detection for self-fetched tokens so the rule only fires when the JWT came from somewhere else.
I003 · Code injection
A dispatch dictionary named like a dynamic eval
A handler pattern like _HANDLERS[name](args) looks structurally identical to a dynamic-dispatch injection risk unless you know _HANDLERS is a fixed, hardcoded dict defined three lines above. Fix: added a lazy-dispatch-dict pattern that recognizes the all-caps constant-dict convention and skips it.
R003 · Weak encryption
AES-GCM read as broken
An early version of the rule flagged any AES usage without checking the mode. AES-GCM is authenticated encryption and is the correct choice; the finding was penalizing apps for doing the right thing. Fix: added mode-aware detection so GCM specifically clears without a finding.
C001 · Hardcoded credentials
Placeholder values that look real
Strings like securepassword123, [email protected], [REDACTED], mock_token, and dev-token kept showing up as findings in seed data, fixtures, and onboarding docs. Fix: extended the placeholder-value list to cover all of these, verified against real credential values (aB3xK9... style) to make sure the expansion didn't swallow anything genuine.
I005 · Prototype pollution
New pattern, added mid-study
Not a false-positive fix, a coverage gap: the scanner had no rule for prototype pollution at all going into this study. Added detection for the __proto__ and constructor.prototype assignment patterns that show up in several of these apps' client-side state management code.
I001 · SQL injection
Parameterized queries flagged as raw concatenation
A query built with a parameterized-query API (placeholders + a separate params array) was being read the same as raw string concatenation because both contain a SQL keyword next to a variable. Fix: added a check for the parameterized-call shape so placeholder-based queries clear correctly.
Wondering if your code has any of this? Scan a public repo free, no account needed.
Scan a repo →What this actually cost
Every fix above shipped with at least one new regression test, and several shipped with a second test specifically checking that the fix didn't over-suppress and hide a real finding written in a similar shape. The full suite sat at 373 passing tests by the end of this round, up from 303 when the year started. Nothing here was a blanket exception. Every suppression is scoped to the specific structural pattern that caused the false positive, not the rule as a whole and not the repo it came from.
That distinction matters more than it sounds like it should. A blanket exception makes a scanner quieter. A structural fix makes it more accurate. The first number gets worse over time as people find ways around the exception. The second number gets better, because the thing you fixed stays fixed.
What didn't make the cut
Not every judgment call was a false positive to fix. Lovable's default Supabase scaffold hardcodes a project URL and anon key directly in src/integrations/supabase/client.ts. That key is designed to be public if Row Level Security is configured correctly on the Supabase project. We debated suppressing it the way we suppress other known-public key formats, and decided against it: most vibe-coded apps we reviewed hadn't configured RLS, which means the key's safety was assumed rather than verified. We kept it as a real finding and said so explicitly in the study's methodology section, because undercounting a real risk to make our own numbers look cleaner isn't a fix, it's a different kind of false negative.
Why we publish this instead of just shipping it
Anyone can claim low false positives. The claim is only worth anything if you can see the actual list of what broke and how it got fixed. This is that list, from the study we just ran, on real apps with real code, checked by hand before a single number went into a blog post.