Where security fits in a delivery pipeline
Not everywhere, and the instinct to run every check at every stage is what makes pipelines slow enough that teams start working around them. Each check has a cost in minutes and a point at which its result is still cheap to act on, and those two properties decide placement. Get them wrong and you either learn about a problem too late to fix it easily, or you pay for the same answer four times per merge.
Why not just run everything on every commit?
Because pipeline duration is a control in its own right, and spending it on security checks that could have run later degrades everything else. A pull request pipeline that takes twenty-five minutes changes developer behaviour: people batch changes, stop running it locally, and merge without waiting. Every one of those effects makes your codebase less secure, which means an over-eager security pipeline can produce a net loss.
The second reason is that most checks have nothing new to say on most commits. Dependency analysis returns the same answer until the lockfile changes, and a base image scan until the image is rebuilt. Running them every time spends minutes reproducing a result you already hold, and buries the occasions when the answer genuinely changed.
A useful budget to hold yourself to is that everything on the pull request path finishes in under ten minutes including tests. Anything that cannot meet that runs on merge, on a schedule, or against the artefact rather than the change.
Which checks belong before the commit exists?
Only the ones that are fast, local and about the change itself: secrets detection, formatting-level static rules, and any linter that runs in under a second or two. Pre-commit is the only place where a mistake can be prevented rather than recorded, which matters enormously for secrets and hardly at all for anything else.
The constraint is that pre-commit hooks are advisory by nature. They run on a machine you do not control, they can be skipped with a flag, and a developer under pressure will skip them. So a pre-commit check is a convenience for the author, never a control for the organisation, and every pre-commit check that matters needs an equivalent server-side check behind it.
That pairing is the design rule: prevent locally, verify centrally. With only the local half you have no assurance; with only the central half the developer finds out after pushing, which is where the cost of fixing multiplies.
What should run on a pull request?
Checks whose result depends on the diff and whose output the author can act on immediately. Static analysis limited to changed files, dependency analysis when a manifest or lockfile changed, infrastructure policy evaluation when a definition changed, and the secrets backstop. All of these are diff-scoped, which is what keeps them inside the ten minute budget.
Diff scoping is the most valuable technique here and the one most often skipped. Running static analysis across a whole mature repository on every pull request is slow and reports thousands of pre-existing findings the author did not cause, which trains everybody to ignore the tool. Reporting only what the change introduced makes the tool a reviewer instead of a nag.
The test to apply is whether a finding at this stage names a line the author touched. If it does not, the check is in the wrong place. Move it to the scheduled path and let it feed the backlog rather than the review.
| Stage | Checks that belong here | Time budget | What it is for |
|---|---|---|---|
| Pre-commit, local | Secrets detection, fast lint rules | Under 5 seconds | Prevention, never assurance |
| Pull request | Diff-scoped static analysis, dependency analysis on manifest change, IaC policy | Under 10 minutes with tests | Feedback the author can act on now |
| Build and package | Image scan of what you just built, secrets in layers, SBOM generation, signing | Under 15 minutes | Facts about the artefact you are about to store |
| Pre-deploy | Signature and provenance verification, admission policy, drift check | Under 2 minutes | Refusing to run what you did not build |
| Scheduled, nightly or weekly | Full-repository static analysis, dynamic testing, registry-wide rescan, base image rebuild | Hours, nobody waiting | Backlog and trend, not gating |
| Runtime | Configuration and workload monitoring, anomalous behaviour | Continuous | The findings no build-time tool can produce |
Why does container scanning belong at build and again on a schedule?
Because the two runs answer different questions. Scanning the image you have just built tells you whether this change introduced something, which is the only part the developer can act on. Rescanning images already in the registry tells you whether the world changed under an artefact you shipped last month, which is where most real container exposure comes from.
The second run is the one teams omit, and its absence is why container programmes go quiet. Vulnerability databases are updated constantly, so an image that scanned clean in June can have a critical finding in August without a single line of your code changing. If nothing rescans, you learn this from an external source or not at all.
What makes the scheduled run useful rather than depressing is an automated rebuild on a current base image. Most operating system findings are already fixed upstream and present only because the image is old, so rebuilding clears them in bulk. Pair a weekly rescan with a weekly rebuild and the scan output becomes a short list of real decisions.
What can only be checked at deploy or runtime?
Whether the artefact about to run is the one your pipeline produced. That is a deploy-time check by definition, implemented as signature and provenance verification at admission, and it is the control that makes everything upstream meaningful. A pipeline that scans thoroughly and then permits any image to be deployed has verified something irrelevant.
Runtime also owns the categories that need a live process: which packages are actually loaded, whether a workload is making network connections nobody designed, whether privileges granted in a manifest are being used. Build-time tooling can see that a vulnerable library is present, not that it is on a reachable path with an open listener in front of it, and that distinction is what turns a backlog of thousands into a shortlist.
Configuration drift belongs here too. Infrastructure policy evaluated against a plan tells you what the change intended, not what the environment currently is, and the gap between those two grows every time somebody fixes an incident by hand at three in the morning.
How do you sequence this without a rewrite?
Add checks in the order of decreasing certainty about what to do with the result. Secrets detection first, because a hit is unambiguous and the response is defined. Then infrastructure policy on new definitions, because misconfiguration is where real incidents come from and the rules are objective. Then dependency analysis, then container scanning, then static analysis, which produces the most debate per finding and should not be first.
Run each new check in reporting mode for two to four weeks before it can block anything. You are measuring the false positive rate on your own code rather than the vendor's benchmark, and you need that number before you can defend a gate to the teams it will interrupt.
One thing to do this week: list the current pipeline stages, and against each write the minutes it costs and the last time one of its findings changed anybody's actions. Stages with cost and no effect are candidates for the scheduled path, and moving them is usually worth more than the next tool purchase.
Common questions
- Where should security checks run in a CI/CD pipeline?
- Place each check where its cost in minutes is affordable and its result is still cheap to act on. Secrets detection runs pre-commit and again in CI. Diff-scoped static analysis, dependency analysis and infrastructure policy run on the pull request. Image scanning, SBOM generation and signing run at build. Signature and provenance verification run at deploy. Full-repository analysis, dynamic testing and registry rescans run on a schedule where nobody is waiting.
- How long should a security check take in CI?
- Everything on the pull request path should finish within about ten minutes including tests. Beyond that, developers start batching changes, stop waiting for results and merge without reading them, all of which make the codebase less secure. Any check that cannot meet the budget belongs on the merge path, on a schedule, or against the built artefact rather than the change.
- Are pre-commit hooks a real security control?
- No. They run on a machine you do not control and can be skipped with a flag, so they are a convenience for the author rather than assurance for the organisation. They are still worth having for secrets detection, because pre-commit is the only stage that can prevent a mistake rather than record it. Every pre-commit check that matters needs a server-side equivalent behind it.
- Why do container images need rescanning after they are built?
- Because vulnerability databases change while the image does not. An image that scanned clean in one month can carry a critical finding weeks later with no change to your code. A scheduled rescan of the registry catches that, and pairing it with an automated rebuild on a current base image resolves most operating system findings in bulk, since they are usually already fixed upstream.
- Should static analysis scan the whole repository or just the diff?
- Scope it to the diff on the pull request path and run the full repository on a schedule. A full scan of a mature codebase is too slow for review and reports large numbers of pre-existing findings the author did not introduce, which teaches everyone to ignore the tool. A finding shown during review should name a line the author touched.
- What order should you add security checks in?
- In decreasing order of certainty about the response. Secrets detection first, since a hit is unambiguous. Then infrastructure configuration policy, because the rules are objective and misconfiguration causes real incidents. Then dependency analysis, then container image scanning, then static analysis, which generates the most argument per finding. Run each in reporting mode for two to four weeks to measure its false positive rate on your code before it blocks anything.
