Why nobody can produce an accurate data map

Why does the interview approach fail?

Because the people who know the flows are not the people asked, and the flows outnumber the memory available. A privacy lead sends a template to department heads, who describe the systems they think about rather than the systems their data reaches. Nobody names the error tracker that captures request payloads, the session replay tool on the marketing site, the analytics vendor added by a growth experiment, or the support tool that indexes every attachment a customer ever sent.

The second problem is granularity. A questionnaire asks about systems, and personal data moves in fields. Knowing that the CRM holds customer records tells you nothing about whether a date of birth field is populated, which is the difference between an ordinary record and one that needs a stricter retention rule.

The result is a record of processing that is not wrong so much as incomplete in a specific direction: it lists the systems the organisation bought deliberately and omits the ones it accumulated. Those are the ones that appear in breaches and subject access requests.

Which artefacts can you derive a map from?

Ones you already generate and rarely read. Infrastructure as code state gives you every datastore that exists, including the ones nobody remembers provisioning. Egress logs and service mesh telemetry give you the real list of external destinations your workloads talk to. Your identity provider's application catalogue gives you the SaaS estate that people actually log in to, and card statements give you the part that bypassed procurement.

Each source has a bias, which is why you use several. The table sets out what each one finds and what it will silently miss, and the pairing matters more than the individual source.

How do you find data leaving a container platform?

Turn the network into the inventory. In a Kubernetes estate the cheapest accurate discovery is a default deny egress policy applied in a non-enforcing or audit posture, or the equivalent telemetry from a service mesh or CNI that logs connections, then reading the destination list after a week of normal traffic. Every external hostname your workloads reached is on it, whether or not a contract exists.

This finds the things questionnaires never do: a model provider called from one service, an email delivery API, a geolocation lookup, a monitoring vendor receiving traces with request attributes attached. It also finds the ones that stopped mattering, which lets you close egress paths rather than document them.

Once the list is stable, enforce it. A default deny egress policy with an explicit allowlist converts vendor discovery from an annual exercise into a change request, which means the next tool an engineer wants to try becomes visible before the data moves rather than afterwards. That is the only mechanism we have seen keep a processor list accurate over years.

What about personal data inside free text and images?

Assume it is there and stop pretending the schema tells you. Support ticket bodies, internal notes fields, uploaded attachments, chat exports, call recordings and screenshots pasted into issue trackers hold personal data that no column name announces. In practice these stores are where the awkward material lives, because people write things into a notes field that they would never put in a structured field.

Container platforms add their own version of this. Crash dumps and core files can contain whatever was in process memory, including plaintext identifiers and tokens, and they are frequently written to a shared bucket with a long lifecycle. Image layers are another: a dataset copied into an image and deleted in a later instruction is still present in the earlier layer, and so is anything else that was ever added.

The realistic treatment is classification by container rather than by field. Declare the whole store as holding personal data of an unknown shape, apply the strictest retention that the business can accept, restrict access, and stop trying to enumerate what is inside it.

What does a map need to record to be useful?

Enough to answer the four questions that other obligations depend on. For each store: what identifier links a record to a person, what the purpose and lawful basis are, who or what receives copies, and how long it is kept and by what mechanism. Article 30 asks for a similar set, but the difference between a compliance record and a useful one is the identifier column, because that is the field a deletion or access request has to travel along.

Recording the mechanism rather than the intended period is the other change worth making. A retention entry that reads twenty four months is a plan. One that reads S3 lifecycle rule, expires objects at 730 days is a fact, and it can be verified by anybody in a minute.

Keep it in the repository next to the code, in a text format, reviewed in pull requests. A map that lives in a spreadsheet is maintained by whoever remembers to, and a map that lives next to the schema is maintained by the person changing the schema.

What test tells you how bad the gap is?

Pick one real person, ideally a colleague who consents, and try to enumerate every system holding data about them in a single working day. Not a policy exercise: actual queries against actual stores. Write down the systems as you find them and compare that list with the record of processing at the end.

The finding is always the same in shape and different in detail: two or three stores that appear in reality and not in the register, usually a log platform, an analytics or warehouse copy, and a SaaS tool bought by one team. Those three are also, not coincidentally, the ones that break a deletion request.

Run the same exercise against a customer who was deleted three months ago and it becomes a retention test as well. If their identifier still appears anywhere, the deletion path is incomplete, and you now know exactly where before a regulator or a customer asks.