Retesting, and what fixed should actually mean

Why does a retest pass mean less than it appears to?

Because a retest is scoped to the finding as written, and findings are written as single examples. If the report says the account identifier in the path is not checked against the session at one endpoint, the developer will fix that endpoint, the retester will confirm that endpoint, and the report will close. Whether the other endpoints in the same controller carry the identical flaw is outside the scope of both activities unless somebody asks for it explicitly.

There is a second, quieter problem. Fixes are frequently applied at the wrong layer. Blocking a payload with a web application firewall rule, filtering a specific string, or rejecting one character makes the proof of concept fail while leaving the underlying defect in place. A retest that only replays the original request will pass. A retest that varies the encoding, changes the parameter or approaches the same sink from another route will not.

The test to apply: read the developer's fix description and ask whether it names the defect or the exploit. Escaping the input on this page is the exploit. Using parameterised queries in this data access layer is the defect. Retests against the first kind of fix are worth much less than retests against the second.

What should a retest actually cover?

Three things beyond replaying the original request. Variant testing, meaning the same flaw approached differently, since the point is the class rather than the instance. Pattern testing, meaning a check of a sample of comparable components for the same mistake, because a defect found once in a codebase is rarely unique. And a regression check that the fix has not opened something else, which happens most often with authorisation changes and with rate limiting.

Retests should also verify the fix in the environment that matters. A fix confirmed in staging and not yet deployed is not remediated, and the gap between the two is where findings quietly age. If the retest ran against a copy, the report should say so, and the closure should remain provisional until the change is live.

None of this requires a large budget. Variant and pattern checks on a fixed set of findings are hours of work, not days, and they are the difference between a closed list and a closed problem.

What should the contract say about retesting?

Retesting terms are usually thin, and thin terms consistently cost money later. Each row below is a clause worth negotiating before signature, when it is free, rather than after the report, when it is a change request.

Who decides that a finding is closed?

The tester verifies, and your organisation accepts, and these should be separate acts recorded separately. A supplier saying the fix works is a technical statement about a moment in time. Closing the finding is a business decision about whether you accept the residual risk, and it should sit with a named owner rather than with whoever last touched the ticket.

This matters most for the findings that will not be fixed. Every remediation list has items that are too expensive, blocked on a vendor, or dependent on a platform migration scheduled for next year. Those are legitimate outcomes provided they are recorded as accepted rather than closed, with the accepting person named, the reason stated, the compensating controls listed and a review date set. Acceptance without an expiry becomes invisible within a year.

Keep the original report immutable. Amending it so findings disappear destroys the evidence trail an auditor, an insurer or an acquirer will later want, and it removes your own ability to see how long a defect class kept recurring. Track status in a separate register that references the report.

What does a fix need to prove in a container estate?

That the change is in the image that runs, not only in the repository. Container remediation fails in a specific and common way: the base image is updated, the application image is rebuilt, and one deployment still references a stale tag or a digest pinned months ago. The fix is genuinely present in the build and genuinely absent from production, and nothing in a typical retest catches that unless the retester looks at what is actually running.

So retest terms for containerised systems should require verification against a running workload, with the image digest recorded. A digest is the only unambiguous identifier here, because tags move and can be overwritten. Recording it also gives you a way to answer, months later, whether the version carrying the fix was ever deployed.

Infrastructure findings need the equivalent discipline. A permission narrowed by hand in a console and not in the Terraform that manages the account will be reverted at the next apply. Ask the retester to confirm the fix exists in the declarative source as well as in the live environment, because those two states diverging is the normal condition of most cloud estates rather than an unusual one.

How long should you have to remediate?

Set the clock by severity band and by whether a fix exists, and write the bands into your own policy rather than accepting whatever the report suggests. Something demonstrated to expose customer data from the internet is a days problem. An internal privilege escalation requiring an existing foothold is a weeks problem. A hardening recommendation with no demonstrated impact belongs in the backlog with everything else and should not be dressed as urgent.

Where a compliance scheme imposes deadlines, they are usually the floor rather than a sensible target, and they vary by scheme and jurisdiction. Confirm the current requirement with whoever handles your assessment rather than working from what applied at the last cycle.

The most useful metric is not how many findings you closed but how long the highest severity ones stayed open. Median time to remediate by band, tracked across engagements, tells you whether the programme is improving. A count of closed findings mostly tells you how many were easy.