Published 2026-05-11 · 9 min read

Vulnerability scanner false positives are a UX bug, not a feature

Trust dies in the first hour of triage. Open a scanner you've never used, get back 200 findings on 12 hosts, find the third one you check is the same OpenSSL CVE the vendor already backported a fix for — and you start ignoring the rest. That instinct is correct. It's also the failure mode every vulnerability scanner has to design against.

The first hour of triage decides everything

The way someone evaluates a security scanner isn't by counting coverage matrices. It's by reading the first run. If the first run reads like a 200-row spreadsheet of "you have a problem here, and here, and here, and here, and here…" — most people quit reading by row 30. The findings after row 30 might be real. They might be the ones that matter. Nobody finds out, because nobody gets there.

Scanner noise isn't a "small annoyance to filter out later." It's the thing that decides whether the tool gets used a second time. Every vulnerability scanner has the same product job: surface what matters in a way the operator can actually act on. Most scanners ship a coverage matrix instead.

Where vulnerability scanner false positives actually come from

"False positive" is a fuzzy term. In practice, what gets called a false positive falls into five categories — only one of which is truly the scanner's fault:

1. Backport-blind matching. The CVE database says "OpenSSH 8.7 is vulnerable to CVE-2024-6387." Your Ubuntu 22.04 box runs openssh-server 8.9p1-3ubuntu0.10, which Canonical patched against that CVE — but a naïve scanner compares upstream version strings and yells. This is the most common form of homelab false positive and it's exactly why the feed source matters. (We cover this in continuous CVE scanning vs periodic patching.)
2. Out-of-context severity. A CVE with a CVSS of 9.8 that requires authenticated local access on a box you don't expose is, in your environment, not a 9.8. CVSS is a worst-case score; your context turns it into a real number. Scanners that don't ask the context question can't tell you which 9.8s actually matter. (See how to triage CVE findings.)
3. Stale matches. The CVE applied to your installed version last Tuesday. You patched on Wednesday. A scanner that doesn't re-check on every run, or that caches results too aggressively, still shouts about it on Thursday. Now you don't trust Thursday's run either.
4. Detection ambiguity. The scanner sees a process listening on a port and infers a service. The inference is wrong: it's actually caddy proxying, not a bare Grafana, and your reverse-proxy auth is in front. The scanner flags an unauthenticated admin surface that doesn't exist as described. (More in exposed admin surfaces.)
5. True positives the operator doesn't want. "I know the password auth is on, I'm fine with it." Not actually a false positive — but if the scanner can't be told "acknowledged" and re-asks every run, it functionally becomes one. Acknowledgement state is one of the most overlooked scanner features.

Of those five: only #4 is a pure detection bug. The other four are workflow failures — the scanner has the right raw data but presents it wrong.

The diff-first approach: only show what changed

The cheapest way to make a scanner trustworthy isn't to chase perfect detection. It's to change the unit of presentation. Most scanners present the union of every finding they currently know about. A diff-first scanner presents only what's different from the last run.

The math works because most of what a scanner finds on day N is identical to what it found on day N-1. If yesterday's report flagged 47 issues and today's report flags 48, the only thing the operator actually needs to read this morning is the one that's new. Plus, ideally, the one that got fixed.

Concretely, a diff-first morning report looks like this:

2026-05-11 — overnight changes across 7 hosts

NEW
  proxmox-rack    CVE-2024-6387 (critical)    openssh-server 8.9p1-3ubuntu0.9
  vps-london      sshd_config drift           PermitRootLogin yes (was: no)
  pi-bedroom      newly exposed service       grafana on :3000 (no auth)

RESOLVED
  nuc-plex        CVE-2024-3094 (critical)    xz-utils upgraded to 5.4.6-3

UNRESOLVED FROM PRIOR RUNS
  3 findings still open across 2 hosts (see dashboard)

Five lines. Three new things to look at. One satisfying resolution. A pointer to the things you've already seen but haven't dealt with. That's what gets read at 8am with coffee. The 47-row dashboard, with the noise filtered to the delta, is what makes that report short.

What gets lost when you cut noise (the honest tradeoff)

A diff-first view has a real cost. Three things to be aware of:

The first run is brutal. Before there's a yesterday to diff against, you see everything. A fresh enrolment across 7 hosts can surface 60+ findings on day one. The right way to handle this is up-front — walk through the first run with the operator's expectation set, and let them mark broad categories as acknowledged. Hide it under "your first scan reveals 0 findings" and you've built a different product: a scanner that lies.

Drift you accepted re-appears when context changes. If you acknowledged "I know password auth is on, the box is on Tailscale-only" and then six months later you move the box to public DNS, that acknowledgement should re-fire. Scanners that flatten acknowledgement into "muted forever" make this worse, not better. Acknowledgement needs to be context-aware: tied to the reason, not just the finding ID.

The unresolved list is the safety net. Diff-only reports without a persistent "still on you" list become a way to forget about real problems. The morning email shows what changed. The dashboard, when you click in, shows what's open. Both have to exist.

How Noxen handles each category

None of the above is theoretical for us — it's the product design. Quickly, for each false-positive category above:

• Backport-blind matching (#1): the CVE feed is sourced from VulnCheck NVD++ and OSV.dev's distro-aware triage data (Ubuntu, Debian, Rocky/Alma), so a backported fix in your installed package is recognised as a fix. Distro-tagged severity is surfaced first; upstream CVSS is secondary. (See agent vs agentless for the architectural side.)
• Out-of-context severity (#2): exposure matters. A CVE on a service bound to localhost on a Tailscale-only host reads differently from the same CVE on a public-facing reverse proxy. We don't pretend to score this for you — we surface the inputs (exposure, severity, KEV / EPSS flags) so you can.
• Stale matches (#3): every run is a full re-scan against a fresh feed snapshot. There's no per-CVE cache to go stale; if you patched Wednesday, Thursday's report clears Wednesday's finding.
• Detection ambiguity (#4): Noxen flags exposed admin surfaces by fingerprint, with the response banner and the inferred service name both surfaced. You see what was detected and the evidence, not just the verdict. We never authenticate against a flagged service, which means false positives stay cheap — they're a noisy line in a report, not a logged login attempt.
• True-but-acknowledged findings (#5): per-finding "acknowledged" state with a reason. Acknowledged findings drop out of the diff view but stay on the unresolved dashboard so they're not forgotten.

The trust step: how a scanner earns the second run

Every scanner gets one first run. That first run is a negotiation: does this tool save me time, or does it cost me an hour to figure out what's noise?

The scanners that win the second run are the ones that respect your attention. The ones that lose are the ones that confuse "comprehensive output" with "useful output." Coverage matters, but coverage without prioritisation is the same as no coverage — the operator's eyes glaze before they reach the row that matters.

Quiet scanners win not because they find less, but because they show less. The findings are still there when you go looking; they're just not in the way of the things that need to be looked at today.

FAQ

How does this compare to suppressing findings?

Suppression hides things forever; diff-first surfaces only what's new. The distinction matters: a suppressed finding never tells you when it changes (e.g. a previously low-severity CVE that gets added to CISA KEV). A diffed finding stays on the unresolved list and re-surfaces when its context changes.

What about the first scan? Doesn't it just dump everything?

Yes — and that's the right behaviour. Hiding the first run's true state would be lying. The diff-first model kicks in from scan number two onward. Most people work through the first run's list with the morning the tool gets installed, then the day-two experience tells them the rest.

Doesn't this miss findings that don't change?

No — unresolved findings persist on the dashboard. The diff view is "what's new today"; the dashboard view is "what's still open." Both are always available; only the diff lands in your morning attention budget by default.

Isn't a 200-row report better when something is actually wrong?

In an incident, yes — and the dashboard is there for it. The morning report is for the 364 days a year when nothing is on fire. Optimising the daily experience for the rare-incident case is how scanners get abandoned during the calm 364.

Try Noxen — $79 one-time. Agentless audits from your Mac, diff-first reporting, no alert fatigue.