What happens when you send company data to an AI provider?
Less than most security teams fear and more than most vendors volunteer. The headline concern, that your prompts become training data, is generally not true on business tiers and is contractual rather than technical. The risks that survive scrutiny are different ones, and they are worth understanding properly because the answer determines your architecture, and rearchitecting later is expensive.
Do prompts get used to train the model?
The commitment not to train on customer content attaches to a paid agreement: business plans, enterprise plans and paid API plans, where it sits in the terms rather than resting on trust. It does not attach to API access as such, and on consumer tiers it has historically gone the other way by default.
The line is paid against unpaid, not consumer against API, and that distinction catches people out because it cuts across the assumption that anything reached through an API is safe. Google's Gemini API terms separate Paid Services from Unpaid Services: only the paid side carries the commitment not to use prompts and responses to improve the product, while the free tier is used to develop Google's services and may be read by human reviewers. OpenAI does not train on API data by default, and Anthropic does not train on retained data without express permission, so the two of them match the general rule as stated.
The practical consequence is specific, and it is the most common mistake in early projects: a team pilots on a free API key or a personal account, believing that reaching the model through an API is itself the protection, and is instead feeding company data into a training set. Nobody notices until security reviews it, by which point the pilot has been running for weeks on real documents.
So the first thing to establish is not whether AI providers train on data in general. It is which specific tier and agreement your integration will run under, and to read that document rather than a blog summary of it. Check which tier the key belongs to, not merely which company issued it.
How long is data retained?
Typically a short window for abuse monitoring, commonly around thirty days, after which it is deleted. Several providers offer zero-retention arrangements for eligible customers, where request content is not stored at all.
This matters for a specific and underappreciated reason: retention is what makes a provider breach your problem. If content is retained for thirty days, a compromise at the provider exposes thirty days of your prompts. If it is not retained, the exposure is limited to data in flight.
Ask directly whether zero retention is available on your agreement, what qualifies you for it, and whether it applies to every endpoint you plan to use. The answer is frequently yes and frequently not mentioned unless asked.
Where do the real risks sit?
Not usually with the provider. Three others are more likely to cause an incident, and all three are yours to control.
The first is over-collection at the source. Systems commonly send far more than the task needs, because sending the whole record was easier than selecting fields. A summarisation feature rarely needs the customer's payment details, and the cheapest privacy control available is not sending what is not required.
The second is logging. Prompts and responses get written to your own application logs, error trackers and observability tools, which are frequently accessible to more people and retained far longer than anything at the provider. Teams that carefully negotiate zero retention with a vendor then keep the same content in a log aggregator for a year.
The third is the output. A model asked to answer from company documents will answer from whichever documents it can reach, so a retrieval system indexing a shared drive will happily surface salary information to whoever asks. Permissions have to be enforced at retrieval time, per user, not assumed from the interface.
| Concern | Actual exposure | Who controls it |
|---|---|---|
| Training on your prompts | Low on paid agreements, contractually excluded | Your choice of agreement |
| Provider retention | Bounded, often removable entirely | Your agreement |
| Sending more than needed | High and routine | You |
| Prompts in your own logs | High, longest-lived, most overlooked | You |
| Retrieval ignoring permissions | High, and it leaks internally | You |
| Cross-border transfer | Depends on regime, and not every provider offers an EU region | Your provider and architecture |
What about GDPR and data residency?
Sending personal data to a model provider is processing, so it needs a lawful basis, a processing agreement, and clarity on where it happens. This is well-trodden ground rather than novel legal territory, and every major provider offers a data processing agreement. Regional endpoints are the part that varies, and the variation is wider than most architecture reviews assume.
OpenAI runs a genuine European processing endpoint at eu.api.openai.com, subject to approval and a Modified Retention amendment. Anthropic does not: as at August 2026 neither the first-party API at api.anthropic.com nor Claude Enterprise offers EU data residency, and requests default to US infrastructure. EU-resident inference for Claude means routing through AWS Bedrock in an EU region or Google Cloud Vertex AI in an EU region, where the cloud provider rather than Anthropic becomes your processor and a different processing agreement applies. EU support in Microsoft Foundry was still listed as coming through 2026. Google's position has the same shape: region pinning is a Vertex AI feature, not something the direct Gemini API offers.
Settle that before you standardise on a provider. A team that picks Anthropic on the assumption that a European endpoint exists has to rearchitect onto a hyperscaler to get one, which is exactly the expensive late discovery this page exists to prevent.
A regional endpoint also narrows the transfer question rather than closing it. It settles where processing and storage happen, but where the provider is a US-headquartered group, remote access by the US entity for support, incident response or abuse review is still a transfer under EDPB Guidelines 05/2021. What closes it is the processing agreement together with a transfer mechanism: the Standard Contractual Clauses, or the provider's certification under the EU-US Data Privacy Framework. The framework is in force, the General Court having dismissed the Latombe challenge on 3 September 2025, but an appeal is pending before the Court of Justice as Case C-703/25 P, so it is not a settled foundation to build on unexamined.
The area that genuinely deserves caution is special-category data: health, biometrics, and anything covered by sector-specific rules. There the question is not only where it goes but whether it should be processed this way at all, and that is a conversation for your own counsel rather than for a supplier's assurance.
When is self-hosting the right answer?
When something concrete forbids the alternative, rather than when it feels safer. Three situations qualify reliably: a contractual prohibition on sub-processors, which is common in defence, critical national infrastructure and parts of the public sector; a regulator or client requiring data never to leave a specific network; or a workload whose volume makes hosted inference genuinely more expensive at your scale.
Outside those, self-hosting usually trades a contractual risk you can assess for an operational one you must staff. Running open-weight models well means GPU capacity, model updates, evaluation and someone on call, and the security posture of a self-hosted deployment maintained by a busy team is not automatically better than a provider's.
The decision is worth making explicitly and early, because it is the one architectural choice that is expensive to reverse in either direction.
What should you settle before writing code?
Which tier and agreement the integration runs under, and whether zero retention is available on it. What the minimum field set is for the task, decided deliberately rather than inherited from whatever the existing API returns. Whether prompts and responses will be written to your logs, and if so with what retention and what redaction. How retrieval enforces per-user permissions. And which region processes the requests.
Those five answers form the security section of the design, and having them written down before implementation is the difference between a review that takes a week and one that sends the project back to the start. They also cost nothing to establish at the beginning, which is not true of any of them later.
Common questions
- Do AI providers train on the data you send them?
- The major providers commit contractually not to train on customer content on paid agreements: business plans, enterprise plans and paid API plans. The commitment attaches to the paid agreement rather than to API access as such, and consumer tiers have historically differed. The dividing line is paid against unpaid rather than consumer against API, which matters because free API tiers can carry consumer-style terms: Google's Gemini API uses content submitted on its unpaid tier to develop its services, while only the paid tier excludes it. Since the distinction is contractual rather than technical, which tier and agreement your integration runs under matters more than which company issued the key, and piloting on a free or personal account is a common and costly mistake.
- How long do AI providers keep your prompts?
- Typically a short abuse-monitoring window, often around thirty days, after which content is deleted, and several providers offer zero-retention arrangements where request content is not stored at all. Retention length matters because it defines how much of your data a breach at the provider would expose. Ask whether zero retention is available on your agreement and whether it covers every endpoint you intend to use.
- What is the biggest privacy risk in an AI integration?
- Usually not the provider. Sending more data than the task requires, writing prompts and responses into your own application logs where they are retained far longer and seen by more people, and retrieval systems that ignore per-user permissions and surface documents someone should not see, together account for most real incidents. All three are within your control.
- Can you use AI providers under GDPR?
- Yes, with a lawful basis, a data processing agreement and clarity on where processing happens. Every major provider offers a DPA, but regional endpoints are not universal: OpenAI runs an EU processing endpoint at eu.api.openai.com subject to approval, whereas Anthropic offers no EU residency on its own API or on Claude Enterprise, so EU-resident inference for Claude means AWS Bedrock or Google Cloud Vertex AI in an EU region, with the cloud provider as your processor under a different agreement. Google's region pinning is likewise a Vertex AI feature rather than a direct Gemini API one. A regional endpoint narrows the transfer question rather than resolving it, because remote access from a US parent is still a transfer; the DPA plus a transfer mechanism, the Standard Contractual Clauses or Data Privacy Framework certification, is what closes it. Special-category data such as health or biometric information warrants separate advice from your own counsel rather than a supplier's assurance.
- When should you self-host an AI model instead of using an API?
- When something concrete requires it: a contractual prohibition on sub-processors, a regulatory or client requirement that data never leaves a specific network, or a volume at which hosted inference is genuinely more expensive. Otherwise self-hosting exchanges a contractual risk you can assess for an operational burden you have to staff, including GPU capacity, model updates and on-call coverage.
