Penetration testing for compliance versus for security
A test can be fully compliant and nearly worthless. If the scope was drawn around the systems an auditor asks about, run at the cheapest acceptable depth, timed to land before a certificate date, and delivered with findings that arrive too late to fix, then you hold valid evidence and know almost nothing new. That is a rational purchase when the requirement is contractual. It becomes dangerous when the organisation mistakes the certificate for an assessment of its actual exposure.
Why can a compliant test prove very little?
Because compliance defines a boundary and testing follows it. A scheme concerned with cardholder data will ask you to test the environment that stores, processes or transmits it, and a test scoped to exactly that boundary is correct under the scheme while saying nothing about the internal application that holds your entire customer list. The scope was drawn by a regulation's concern rather than by yours.
Depth is the second gap. Most schemes require that testing occurs, performed by a suitably qualified party, following a recognised methodology. Few specify how many days, how much of the work must be manual, or that findings must be exploited rather than identified. A supplier competing on price against that specification will deliver the minimum that satisfies it, and the specification permits quite a lot of minimum.
Timing completes the picture. Tests bought for a deadline tend to be scheduled as late as the deadline allows, which leaves no room to remediate. The organisation then either accepts findings it intended to fix, or writes a remediation plan that the assessment accepts in place of a fix. Both are legitimate under most schemes and neither reduces risk before the next cycle.
Where do the two goals actually conflict?
In predictable places, and each has a resolution that costs less than people assume. The conflicts below are worth settling before the scope is written, because afterwards they become arguments about change requests.
| Conflict | What compliance pushes towards | What security needs | Resolution |
|---|---|---|---|
| Scope boundary | The regulated environment only | Whatever holds data or grants privilege | Test the regulated scope, then extend to adjacent systems |
| Timing | Just before the certificate date | Early enough to fix and retest | Work backwards from the deadline through remediation |
| Depth | Enough to evidence that testing happened | Manual chaining and exploitation | Specify manual days in the statement of work |
| Findings you will not fix | A closed or accepted status in the report | A recorded owner, control and review date | Keep a risk register separate from the report |
| Environment | A representative copy is often acceptable | The environment that is actually exposed | Test production where possible, record differences |
| Reporting | A certificate or attestation letter | Reproduction detail engineers can act on | Ask for both as separate deliverables |
What do the common schemes actually require?
Less specificity than most buyers expect, which is why the market varies so widely in price. PCI DSS is the most prescriptive of the widely encountered schemes: it requires internal and external penetration testing on a defined periodic basis and after significant change, with additional testing of segmentation controls where segmentation is relied on to reduce scope, and the cadence for service providers differs from that for merchants. Confirm the current wording and intervals with your assessor rather than working from a previous cycle.
ISO 27001 does not name penetration testing as a mandatory control. It requires management of technical vulnerabilities, and testing is one of the ordinary ways organisations evidence that, which means the auditor's expectation rather than the standard's text determines what you need. SOC 2 is similar: the trust services criteria do not prescribe a test, but many auditors and most enterprise customers expect one, and it is the customer expectation that usually drives the purchase.
In the EU, DORA introduces threat-led penetration testing obligations for certain financial entities, drawing on the TIBER-EU approach, and the UK financial sector has an equivalent in the CBEST framework. These are substantial exercises, not standard tests, and whether they apply to you is a question for counsel and your regulator rather than for a testing supplier. Note also that Cyber Essentials Plus in the UK is a technical verification audit rather than a penetration test, and presenting one as the other causes problems in procurement.
How do you buy one test that serves both purposes?
Scope the regulated environment properly, then add the systems you would actually worry about, and ask the supplier to report against both in one document with the compliance-relevant portion clearly identifiable. Assessors need to see that the required scope was covered; they do not object to additional coverage. The extra cost is usually a fraction of the total because the mobilisation, access and reporting overheads are already paid.
Specify the things the scheme leaves open. Manual testing days as a number, the methodology by name, retesting included with a stated window, and a requirement that findings include reproduction steps. These four lines in a statement of work move a supplier from delivering the compliance minimum to delivering something useful, and they are all free to ask for at procurement.
Then schedule it early in the compliance window rather than late. A test three months before the deadline lets you fix, retest and present a clean report. A test three weeks before produces an accepted risk list, and there is no scheme where that reads better than the alternative.
What does an auditor actually look at?
Coverage against the required scope, the qualifications of whoever performed the test, the methodology followed, the date, and evidence that findings were tracked to a decision. They are checking that a competent process happened and that the organisation did something with the output. Most auditors are not evaluating the technical quality of the testing, which is precisely why quality varies.
The item that fails most often is the last one. A report with unremediated high severity findings and no register, no owners and no dates is a control failure even when the testing itself was excellent, because the scheme cares that you manage what you find. A modest report with a clean, dated remediation trail passes comfortably.
So keep the artefacts an auditor wants separate from the artefacts your engineers want. One is a scope statement, a signed report, a remediation register with owners and dates, and a retest confirmation. The other is reproduction detail and code-level advice. Trying to make one document do both jobs produces something too long for the auditor and too vague for the developers.
Where does container security fall between the two?
Mostly into the gap, which is why it is worth calling out. Compliance schemes were largely written before container orchestration became standard, so their questions land on patching, segmentation and access control in terms that map awkwardly onto ephemeral workloads and cluster-level identity. An assessor may accept image scanning evidence as vulnerability management and never ask whether a compromised pod can reach the cluster API.
Meanwhile the security question in a container estate is nearly always about identity and blast radius: what a workload's service account can do, whether the instance metadata endpoint is reachable, whether network policy actually restricts anything, and whether the pipeline that deploys can be subverted. None of that appears on a typical audit checklist, and all of it appears in real incidents.
Practical approach: use a recognised configuration benchmark for the cluster as your compliance-facing evidence, and buy an assumed breach test for the security answer. The benchmark gives an assessor something structured to review. The test gives you the thing the benchmark cannot express, which is whether the combination of your configurations leaves a path from one container to your data.
Common questions
- Is a compliance penetration test enough for security?
- Not by itself. Compliance defines a boundary and testing follows it, so a test scoped to a regulated environment can be fully valid while saying nothing about systems outside that boundary which hold data or grant privilege. Schemes also rarely specify depth, manual effort or exploitation, so a supplier competing on price delivers the permitted minimum. Extend the scope beyond the regulated environment and specify manual days explicitly.
- Does ISO 27001 require a penetration test?
- The standard does not name penetration testing as a mandatory control. It requires management of technical vulnerabilities, and testing is one common way organisations evidence that, so what you need is determined by your auditor's expectation rather than by the text. SOC 2 is comparable: the criteria do not prescribe a test, but auditors and enterprise customers commonly expect one. Confirm the expectation with your certification body before scoping.
- What does PCI DSS require for penetration testing?
- It is the most prescriptive of the widely encountered schemes, requiring internal and external penetration testing on a defined periodic basis and after significant change, plus testing of segmentation controls where segmentation is used to reduce scope, with intervals that differ between service providers and merchants. Because the wording and cadences are revised between versions, confirm the current requirement with your assessor rather than reusing a previous cycle's plan.
- Is Cyber Essentials Plus a penetration test?
- No. Cyber Essentials Plus is a technical verification audit that checks whether a defined set of basic controls is in place, using sampled hands-on checks. It does not involve a tester working towards an objective, chaining weaknesses or assessing your authorisation logic. Presenting it as a penetration test in a procurement questionnaire causes problems, and buyers who require a test will not accept it as a substitute.
- What do auditors check in a penetration test report?
- Coverage against the required scope, the tester's qualifications, the methodology followed, the date, and evidence that each finding was tracked to a decision with an owner. They are usually not evaluating technical quality, which is why quality varies so much across the market. The item that fails most often is the last: unremediated high severity findings with no register, owners or dates read as a control failure even when the testing was excellent.
- Do compliance schemes cover container security?
- Poorly, because most were written before container orchestration became standard, so their questions on patching and segmentation map awkwardly onto ephemeral workloads and cluster-level identity. An assessor may accept image scanning as vulnerability management without asking whether a compromised pod can reach the cluster API. Use a recognised cluster configuration benchmark as audit evidence and buy an assumed breach test to answer the actual security question.
