Our scanner flagged its own marketing copy as a leaked password

Prbl's homepage has a section showing visitors real examples of what the scanner catches. One of those examples is a string of marketing copy:

code: "create_blog_posts.py:22 — password='password123'",

That line is display text. It tells a visitor: here is what a hardcoded credential finding looks like. It is not a real assignment anywhere in the Prbl codebase. The scanner does not know that. It read the literal string password='password123' sitting inside a .tsx file and reported it as a HIGH severity hardcoded credential, on its own marketing page, in its own repo.

Why this happened

The rule that catches hardcoded credentials, PRBL-C001, is intentionally aggressive about exactly this pattern. A real password = 'password123' in application code is a serious finding and the rule has to fire on it without hesitation. It has no way to know, from the regex alone, that this particular instance is inside a string describing a finding rather than an actual assignment.

We had a second instance of the same root problem in a different file: prbl/rewriter/prompt.py, the module that builds prompts for the AI rewriter. That file is full of intentional “Before:” and “After:” code examples for every fixable rule, TLS verification disabled, JWT decoded without checking the signature, hardcoded session secrets, by design, so the rewriter knows what a fix looks like. Scanning it produced a wall of HIGH severity findings for content that exists specifically to teach the AI rewriter what bad code looks like.

The fix

Two separate, narrow fixes, not a blanket exception for our own repo:

• A new suppression pattern that recognizes the "path/file.ext:LINE — description" format common to security writing and marketing copy describing a finding. If a line looks like documentation referencing a finding elsewhere, it is not treated as a live assignment. Verified this does not suppress real credential assignments, only the documentation-prefixed format.
• prbl/rewriter/prompt.py is now skipped entirely by the scanner. Its only purpose is holding intentional vulnerability examples for the rewriter, so there is no scenario where a real production bug would hide there.

Both fixes shipped with regression tests that check the fix works and that it does not accidentally suppress a real finding written in a similar style. Full test suite: 327 passing.

Why we are posting this instead of quietly fixing it

A security tool that never finds anything wrong with itself is not impressive, it is unverified. This one found two real false positives in its own codebase the same week it shipped, and the fix for each one is now a permanent regression test instead of a one-off patch. That is the actual argument for using a scanner: not that it is perfect, but that when it is wrong, the wrongness gets caught, explained, and locked down so it cannot happen again.