What a penetration test cannot find
A clean penetration test report is evidence that a competent person did not find a way in during a defined number of days against a defined scope. It is not a statement that no way in exists, and treating it as one is the most expensive misreading available. Whole categories of risk sit permanently outside what testing can reach: things that change after the tester leaves, things that require no vulnerability at all, and things that depend on whether you would notice. Each has a control that covers it, and none of those controls is another test.
Why is a clean report not an assurance of security?
Because the finding set is bounded by three things: the scope you defined, the time purchased, and the specific expertise of the individual assigned. Change any one and the results change. A tester strong in web application logic may not probe your cloud identity configuration deeply; a network specialist may not spot a subtle authorisation flaw. This is not a criticism of testers, it is arithmetic, and it is why the same system tested by two firms produces two different reports.
The scope boundary is the largest factor. Everything you excluded is untested by definition, including the things excluded because nobody remembered they existed. Shadow systems, a forgotten subdomain still resolving to an old host, an internal tool built by a team outside the conversation: these are common sources of real incidents and structurally absent from a scoped engagement.
Time bounds the rest. Some vulnerability classes take longer to find than an engagement lasts, particularly race conditions, flaws requiring specific state to be built up over many steps, and anything needing sustained reverse engineering. A tester with five days makes a judgement about where to spend them, and that judgement is a filter on your results.
What is structurally out of reach?
Four categories. Anything that changes after the test, which is everything, since a report begins ageing the day it is issued and your systems do not stop deploying. Anything requiring no vulnerability, such as an employee with legitimate access using it improperly, or credentials stolen through phishing and used exactly as designed. Anything upstream of you, in a dependency, a base image or a supplier's environment. And your ability to detect and respond, which a standard test does not measure and often deliberately avoids provoking.
Novel vulnerabilities belong here too, with a caveat. A test cannot tell you about a weakness in a component that nobody has discovered yet, which sounds obvious and matters because it defines what you should be building instead: the capacity to find out quickly and patch fast when the disclosure arrives. Resilience against unknown vulnerabilities is an operational capability, not a testing outcome.
The useful reframing is that testing measures the state of a system at a point in time, and most of your actual risk is a function of rate: how quickly you change, how quickly you find out, how quickly you respond. Point-in-time assurance cannot report on a rate.
What covers each gap?
Each blind spot has a control that addresses it directly, and the mapping matters because the instinctive response to an uncomfortable report is to buy more testing, which usually addresses none of these.
| Gap | Why testing misses it | What covers it |
|---|---|---|
| Changes after the test | A report describes a fixed point in time | Continuous scanning, pipeline gates, configuration drift alerts |
| Runtime behaviour of containers | Testing sees a snapshot, not a workload over weeks | Runtime monitoring of process, network and file activity |
| Compromised dependency or base image | The flaw is upstream and may not exist yet | Dependency inventory, signed artefacts, rapid rebuild capability |
| Misuse of legitimate access | No vulnerability is involved | Least privilege, access review, logging of privileged actions |
| Stolen credentials used normally | The system behaves exactly as designed | Phishing-resistant authentication, anomaly detection |
| Whether you would notice an intrusion | Standard tests avoid provoking your response | Red or purple team exercises, detection engineering, tabletops |
| Assets nobody remembered | Excluded from scope because unknown | Attack surface discovery run continuously, not annually |
Why does container security need monitoring rather than testing?
Because a container estate changes faster than any testing cycle and the interesting signals are behavioural. A test can establish that a pod's service account is over-permissioned and that an escape path exists. It cannot tell you that three weeks later a workload started making outbound connections it never made before, or that a shell was spawned inside a container that has no reason to run one, or that a new image appeared in your registry from an unexpected source.
Those are runtime questions, and they need a runtime answer: monitoring of process execution, network connections and file writes inside workloads, plus alerting on the small set of events that are always suspicious in a containerised system. Unexpected shell execution, a package manager running in production, a connection to the instance metadata endpoint from an application container, and a workload mounting a host path are each rare enough that alerting on them is practical rather than noisy.
This is also where container security programmes most often stall. Teams deploy image scanning, generate a very large finding count, then have no runtime view at all, which means they can enumerate theoretical vulnerabilities and cannot tell whether anything is happening. Getting the runtime signals in place tends to be a smaller project than clearing the scanner backlog and produces more security per hour spent.
What about supply chain and insider risk?
Supply chain exposure is not primarily a testing problem, it is an inventory and provenance problem. What you need is knowledge of what your software is made of, so that when a component is disclosed as compromised you can answer whether you use it in minutes rather than in days; a way to verify that artefacts came from your build system rather than somewhere else; and a pipeline capable of rebuilding and redeploying quickly. A dependency inventory that is current, machine-readable and actually queried is worth more than any amount of testing against the same risk.
Insider risk sits outside testing almost entirely, because there is no vulnerability to find. The controls are structural: privileges granted narrowly and reviewed on a schedule, separation between who can change code and who can deploy it, logging of privileged actions to somewhere the privileged user cannot alter, and a leavers process that actually removes access. A test can confirm a specific control exists, and cannot assess whether the model as a whole is sound.
One question worth putting to your own team: if a developer with production access decided to extract customer data today, which record would show it and who would read that record. If the answer involves logs nobody monitors, that is a gap no penetration test will ever report.
What should you buy after your first test?
Whatever the report implies about your process rather than about your code. If the findings were mostly missing patches, the answer is a patch cadence and automated rebuilds rather than another test. If they were mostly authorisation flaws, the answer is a secure development practice and code review that specifically looks at authorisation. If the tester reached your data from an external position quickly, the answer is architecture work. A second test measures the same things again.
Then buy detection, in the specific sense of knowing whether the attack paths that were found would generate a record you would read. This is the cheapest exercise available and almost nobody runs it: take the timeline of the test, with its exact timestamps and source addresses, and ask your own team to reconstruct it from your logs alone. What they cannot reconstruct is a gap you now know about precisely, and the exercise costs a day.
The sequence that works is test, remediate, then build the capability the test showed you were missing, then test again on a trigger rather than a date. Organisations that repeat the test without the middle step get a similar report every year and eventually conclude that testing does not work, when what they demonstrated is that testing does not remediate.
Common questions
- Does a clean penetration test mean you are secure?
- It means a competent person did not find a way in during a set number of days against a defined scope, using their particular expertise. Change the scope, the duration or the individual and the results change, which is why the same system tested by two firms yields two different reports. Everything excluded from scope is untested by definition, including assets nobody remembered existed, and those are a common source of real incidents.
- What can a penetration test not detect?
- Four categories: anything that changes after the test, since a report ages from the day it is issued; anything requiring no vulnerability, such as misuse of legitimate access or stolen credentials used exactly as designed; anything upstream in a dependency, base image or supplier environment; and whether you would detect and respond to an intrusion, which standard tests deliberately avoid provoking. Undisclosed vulnerabilities also sit outside its reach by definition.
- What covers the gaps a penetration test leaves?
- Continuous scanning and pipeline gates for changes after the test, runtime monitoring for container behaviour, a current dependency inventory plus artefact signing and fast rebuild capability for supply chain risk, least privilege and privileged action logging for insider misuse, phishing-resistant authentication for credential theft, and red or purple team exercises for detection. None of these is more testing, which is the instinctive but usually wrong response to an uncomfortable report.
- Why does container security need runtime monitoring?
- Because a container estate changes faster than any testing cycle and the significant signals are behavioural. Testing can prove an escape path exists; it cannot tell you that a workload began making new outbound connections, that a shell was spawned in a container with no reason to run one, or that a connection reached the instance metadata endpoint from an application container. Those events are rare enough that alerting on them is practical rather than noisy.
- How do you test whether you would notice an attack?
- Use the penetration test itself. Take the tester's timeline, with exact timestamps and source addresses, and ask your own team to reconstruct the activity from your logs alone, without help from the report. Whatever they cannot reconstruct is a precisely defined detection gap, and the exercise costs about a day. If your logs cannot rebuild a known sequence you have full details for, they will not rebuild an unknown one.
- What should you buy after your first penetration test?
- Whatever the findings imply about your process rather than your code. Mostly missing patches points to a patch cadence and automated rebuilds. Mostly authorisation flaws points to development practice and targeted code review. A quick route from outside to your data points to architecture work. Then build detection capability, and test again on a change trigger rather than a date. Repeating the test without the middle step reproduces the same report annually.
