Why is the number of open vulnerabilities a bad metric?

Because it responds to your tooling rather than your risk. The count rises when a vendor adds rules, when advisory databases update, and when you extend scanning to more repositories, and it falls when you narrow coverage or disable a check. That creates an incentive to scan less. A metric that moves for reasons outside your control cannot support accountability and should not be reported upwards.

What does a CVSS base score not tell you?

Anything about your environment. The base score describes the worst plausible impact for any user of the affected component, assuming the vulnerable interface is reachable. It does not know whether your service exposes that interface, whether the code path is ever executed, what authentication sits in front of it, or what an attacker would reach after succeeding. Those environmental facts usually change the priority more than the score does.

What is EPSS and how is it different from CVSS?

EPSS, maintained by FIRST, estimates the probability that a vulnerability will be exploited in the near term, so it is a forecast. CVSS describes the potential impact if exploitation happens, so it is a severity rating. They answer different questions, and using them together separates the small set of issues attackers actually pursue from the much larger set that scores highly and is never touched in practice. Neither replaces knowing whether the affected code is reachable in your own deployment.

What is the escape rate in application security?

The proportion of security defects first discovered after release rather than before it. It is one of the few metrics that directly measures whether your controls work, since a control catching nothing produces a high escape rate regardless of how busy its dashboard looks. Collect it by recording where each defect was first observed: pipeline, staging, penetration test, bug bounty, customer report or incident.

How do you measure security scanning coverage properly?

Build the denominator from an independent source. Take your deployment system or cloud asset inventory, list every running service, repository and image, then check which of them appear in the security tooling at all. Reading coverage from the scanner's own project list is circular and always looks close to complete. For dynamic testing, apply the same logic at route level by comparing requested URLs against your framework's route table. This reconciliation regularly surfaces live applications that no security tool has ever examined.

What security metrics should go in a board report?

Time to remediate for exploitable findings, escape rate to production, and the share of the estate with no coverage. Add one line naming security work that was deferred and why. If a volume figure is required, report only findings that are both known-exploited and reachable in your environment, and state that filter, because a small defined number withstands questioning while a large undefined one does not.

Application security guide

Why the finding count is the wrong metric

You do not control the number, so you cannot be accountable for it. Open findings rise when a vendor adds rules, when an advisory database updates, when you onboard a repository and when upstream maintainers publish something about a library you have not touched in a year. It falls when someone disables a check. A metric that moves for reasons unrelated to your decisions is a reporting liability, and replacing it is one of the highest-value changes most programmes can make.

Why does the count rise when the code gets safer?

Because the count is a function of detection, not of exposure. Every scanner reports the intersection of your inventory and a knowledge base that grows daily. Add a rule pack, extend coverage to three more services, or wait for a widely used library to receive an advisory, and the number goes up without a single line of your code becoming more dangerous. Conversely, the fastest way to reduce it is to narrow the scan.

This creates a perverse dynamic that is easy to spot once you look for it. A team that onboards more repositories looks worse than a team that onboards none. A team that upgrades to a scanner with better framework support looks worse than a team on an engine that loses half its traces. If the number is what gets reported upwards, people learn to keep coverage small.

Test it on your own data. Plot open findings against the dates you changed tool configuration or added repositories. If the steps in the graph line up with your own administrative actions rather than with releases, you are measuring your configuration.

What does a severity score actually tell you?

The worst plausible outcome for any user of the affected component, assessed without reference to your deployment. A CVSS base score is deliberately environment-free: it assumes the attacker can reach the vulnerable interface, and it says nothing about whether your service exposes it, whether the code path is ever executed, or whether an attacker who succeeded would reach anything of value. Treating base score as priority order means letting a stranger rank your work.

Two signals narrow it usefully. EPSS, maintained by FIRST, estimates the probability that a vulnerability will be exploited in the near term, and it separates the small set of things attackers actually use from the large set that scores highly and is never touched. The CISA Known Exploited Vulnerabilities catalogue is narrower and stronger: inclusion means exploitation has been observed, and for US federal civilian agencies it carries binding remediation timelines under a standing directive. Neither is a substitute for the other, because one is a forecast and one is an observation.

The third input is yours and no tool has it: whether the affected code is reachable, whether the interface is exposed to untrusted callers, and what an attacker reaches next. A decision framework such as CERT's SSVC exists precisely to combine those with the published score, and the value of adopting one is less about the tree and more about forcing the environmental questions to be asked out loud.

Which metrics actually reflect risk?

Ones you influence directly, that move when your engineering behaviour changes and stay still when a vendor ships a rule pack. Four do most of the work, and each has a specific failure mode worth naming before you commit to it.

Metric	What it tells you	How to collect it	How it gets gamed
Time to remediate, by exploitability tier	Whether the pipeline can actually deliver a fix	Timestamp from finding creation to deployed commit	Reclassifying findings into a slower tier
Escape rate	How much your pre-release controls really catch	Ratio of findings first seen in production to those caught earlier	Testing less in production so less is found there
Dependency lag	How far behind current your third-party code is	Median age gap between pinned and latest stable versions	Bulk upgrades that pass no tests
Image and artefact freshness	Whether running workloads carry months of unpatched packages	Build date of every image currently deployed	Rebuilding without changing the base image tag
Findings with a named owner	Whether the backlog is work or wallpaper	Proportion of open findings assigned to a person and sprint	Assigning everything to one unlucky team
Recurrence of a fixed class	Whether fixes changed the cause or the symptom	Count of the same rule firing on new code after a fix	Suppressing the rule instead of fixing the pattern

What is escape rate and why does it matter more than volume?

Escape rate is the proportion of security defects first discovered after release rather than before it. It is the only metric in common use that directly measures whether your controls work, because a control that catches nothing produces a high escape rate no matter how many findings its dashboard displays.

Collecting it requires one discipline: recording where each defect was first observed. Pre-commit, pipeline, staging test, penetration test, bug bounty, customer report, incident. Six buckets, one field, and the distribution tells you where to spend. If most of your real defects arrive from penetration tests, your automation is not covering the categories that matter, and buying a fifth scanner will not change that.

The metric has a real weakness and you should state it when reporting: it is sensitive to how hard you look in production. A team that starts a bug bounty programme will see escape rate jump. Handle that by tracking the buckets separately rather than as a single ratio, so an increase in external reports is visible as an increase in detection rather than as a decline in quality.

How do you measure coverage without fooling yourself?

Count the things that were not scanned. Coverage reporting almost always describes the estate the tool can see, which is circular. The useful figure is the proportion of your deployed services, repositories and images that appear in the tool at all, computed from an independent inventory such as your deployment system or cloud asset list rather than from the scanner's own list of projects.

For dynamic testing the same logic applies at route level. Compare requested URLs against your framework's route table and report the percentage exercised while authenticated. For static analysis, check whether the build actually compiled: many engines degrade quietly to a partial parse when a build step fails, and a partial parse reports fewer findings, which reads as improvement.

The single most revealing coverage question is about repositories nobody owns. Every organisation has services that are deployed, occasionally patched and absent from every list. Reconciling the deployment inventory against the scanner inventory is a half-day exercise and it routinely finds live applications that no security tool has ever looked at.

What belongs in a report someone senior reads?

Three things: how long a fix takes for the categories where speed matters, how much is escaping to production, and which parts of the estate are not covered at all. That is a page, it prompts decisions about capacity and ownership, and none of it moves because a database updated overnight.

Drop the total. If governance requires a volume figure, report the subset that is both known-exploited and reachable in your environment, and state the filter alongside it. A number of eleven with a stated definition is more useful to a board than four thousand with none, and it does not collapse the first time someone asks what the four thousand means.

Then add one qualitative line that no metric captures: the security work that was deferred and why. Programmes fail quietly through deferral, not through bad numbers, and a standing item that records what was traded away is the cheapest honest control in the whole reporting stack.

Common questions

Why is the number of open vulnerabilities a bad metric?: Because it responds to your tooling rather than your risk. The count rises when a vendor adds rules, when advisory databases update, and when you extend scanning to more repositories, and it falls when you narrow coverage or disable a check. That creates an incentive to scan less. A metric that moves for reasons outside your control cannot support accountability and should not be reported upwards.
What does a CVSS base score not tell you?: Anything about your environment. The base score describes the worst plausible impact for any user of the affected component, assuming the vulnerable interface is reachable. It does not know whether your service exposes that interface, whether the code path is ever executed, what authentication sits in front of it, or what an attacker would reach after succeeding. Those environmental facts usually change the priority more than the score does.
What is EPSS and how is it different from CVSS?: EPSS, maintained by FIRST, estimates the probability that a vulnerability will be exploited in the near term, so it is a forecast. CVSS describes the potential impact if exploitation happens, so it is a severity rating. They answer different questions, and using them together separates the small set of issues attackers actually pursue from the much larger set that scores highly and is never touched in practice. Neither replaces knowing whether the affected code is reachable in your own deployment.
What is the escape rate in application security?: The proportion of security defects first discovered after release rather than before it. It is one of the few metrics that directly measures whether your controls work, since a control catching nothing produces a high escape rate regardless of how busy its dashboard looks. Collect it by recording where each defect was first observed: pipeline, staging, penetration test, bug bounty, customer report or incident.
How do you measure security scanning coverage properly?: Build the denominator from an independent source. Take your deployment system or cloud asset inventory, list every running service, repository and image, then check which of them appear in the security tooling at all. Reading coverage from the scanner's own project list is circular and always looks close to complete. For dynamic testing, apply the same logic at route level by comparing requested URLs against your framework's route table. This reconciliation regularly surfaces live applications that no security tool has ever examined.
What security metrics should go in a board report?: Time to remediate for exploitable findings, escape rate to production, and the share of the estate with no coverage. Add one line naming security work that was deferred and why. If a volume figure is required, report only findings that are both known-exploited and reachable in your environment, and state that filter, because a small defined number withstands questioning while a large undefined one does not.