Procurement questions for third-party AI
Standard AI supplier questionnaires are close to worthless, because every supplier answers yes to every question and nobody checks. Asking whether a vendor has an AI ethics policy, tests for bias and handles data securely produces three affirmatives and no information. Useful procurement questions have a property in common: the honest answer is sometimes inconvenient, and a supplier who has not done the work cannot fake it without writing something specific and checkable.
Which single question matters most?
Who trained the model you are selling me, and can I have their documentation. A large proportion of AI products are a workflow wrapped around someone else's model, which is a perfectly legitimate way to build software and a serious problem if the supplier will not say so. If your vendor cannot name the underlying model and version, they cannot tell you when it changes, cannot pass through the documentation you need, and cannot answer a regulator's question about training data because they do not know the answer either.
The follow-up establishes whether you are talking to someone who has thought about this: what happens to my system when that model is deprecated. Hosted models are retired on the supplier's schedule, and a product built on one inherits that schedule. A vendor with a good answer will describe version pinning, a notice period and a revalidation process. A vendor with no answer has not yet had a model retired under them.
This question also determines who carries the obligations. Where you deploy a supplier's product under their name and their intended purpose, they hold the provider duties. Where you brand it as your own, materially modify it, or use it for a purpose that makes it high risk, those duties move to you regardless of what the contract says about it.
What should you ask about data?
Five specific questions, none of which is answered by a security certification. Is my data used to train or improve your models, and is that a default I have to switch off or a contractual commitment. Is it a per-account setting or a per-workspace setting, since the difference determines whether one employee can undo it. How long is my input retained, and specifically, is there a retention window for abuse monitoring that persists even when training is disabled. Which subprocessors see it, in which regions, and how much notice do I get when that list changes. And on termination, what is deleted, from where, and within what period.
The abuse monitoring window is the one that surprises people. It is common for inputs to be retained for a period so that misuse can be investigated, entirely separately from any training question, and a supplier answering that they do not train on your data has not answered whether they store it. Ask for the retention figure explicitly, and ask whether zero retention is available and at what tier.
Then ask the question that tests all the previous answers: can you show me where in the contract each of these commitments appears. Marketing pages and trust centres change without notice. If a commitment lives only there, you have a statement of intent rather than a term you can enforce.
What should you ask about performance and evidence?
Ask for evaluation results on inputs that look like yours, and expect the answer to be that they do not have any. That is the correct answer, and what matters is what the supplier proposes next. Benchmark figures from public datasets say nothing about performance on your document types, your customer language or your edge cases, and a supplier who offers them as evidence for your use case is either inexperienced or hoping you will not notice.
The productive version of this conversation is to run a paid pilot against an evaluation set you build yourself, before signing anything long. Take a hundred real cases with known correct answers, hold them back, and measure. This is the single highest-value hour in AI procurement and it is skipped constantly, because it requires someone internal to define what correct means, which is uncomfortable and revealing.
Ask also for the failure modes. A supplier who has run their product at scale can tell you what it gets wrong, in what circumstances, and what customers do about it. A supplier who says it does not really get things wrong has either not deployed it widely or is not listening to the customers who have. That answer alone is a reliable sorting mechanism.
Which contract terms matter more than the questionnaire?
Six, and they are worth more negotiating attention than the entire security annex, because they determine what happens when something goes wrong rather than describing what is supposed to happen when it does not.
| Term | Ask for | Why it matters | Expect pushback on |
|---|---|---|---|
| Model change notice | Named model and version, with notice before any change to it | Behaviour can change without your deployment changing at all | Notice periods; many suppliers reserve the right to change silently |
| Training and retention | A contractual commitment, not a setting, plus the abuse monitoring window in writing | Defaults change, settings get reset, and pages are not enforceable | Zero retention, which is often restricted to higher tiers |
| Output indemnity | Cover for intellectual property claims arising from outputs, and the conditions that void it | Indemnities commonly lapse if you disable filters or modify prompts in certain ways | Scope, caps, and whether training data claims are covered at all |
| Incident notification | A defined window in hours, covering model failure and not only security breach | Your own reporting duties start when you know, and can be short | Defining a model producing wrong output as an incident at all |
| Information for compliance | An obligation to supply what you need to meet your own regulatory duties | Your duties are unmeetable if the supplier will not share documentation | Anything framed as proprietary |
| Exit and portability | Export of prompts, configurations, fine-tuned adapters, embeddings and logs | Derived artefacts are what make switching expensive, and they are rarely covered | Ownership of anything created by training on your data |
What about liability for what the AI produces?
Read the indemnity conditions rather than the indemnity. Several major suppliers offer to defend customers against intellectual property claims arising from model output, which sounds comprehensive and is bounded by conditions that are easy to breach in normal use: keeping the supplier's safety filters enabled, not deliberately prompting for infringing material, using specified model versions, and passing through certain notices. A deployment that disables a filter for a legitimate operational reason may have stepped outside the cover without anyone realising.
Ask separately about claims relating to the training data itself, which is a different exposure from claims about a specific output and is often excluded. Ask what the liability cap is, and compare it with the harm your use case could actually cause. Where a supplier's cap is a multiple of annual fees and your exposure is regulatory penalties or a class of affected individuals, the cap tells you that the residual risk sits with you.
Finally, note what almost no AI contract contains: a quality commitment. Availability is covered by uptime terms. Accuracy is not. If the model becomes worse, is slower to answer, or starts refusing a category of request, most contracts provide no remedy at all. That is not necessarily unreasonable given the state of the field, but it should be a conscious acceptance rather than a discovery made later.
How do you read the answers?
Look for specificity as the primary signal. A supplier who has done the work answers with a number, a document name, a date or a clause reference. A supplier who has not answers with a category: robust processes, industry-leading security, comprehensive testing. The distinction is easier to apply than any scoring matrix, and it survives the fact that you often cannot verify the underlying claim.
Treat certification carefully. An information security certificate says the supplier manages information security, which is necessary and tells you nothing about the model. An AI management system certificate says a management system exists and is followed, which again says nothing about whether the model works. Neither is a substitute for evaluation evidence, and a supplier leading with certificates in answer to a performance question has changed the subject.
Then run one final check that costs nothing: ask for a customer reference in your sector who has been live for more than a year, and ask that reference what surprised them. The interesting answers are always operational, concerning volume, cost changes, model updates that shifted behaviour, or the amount of internal work needed after go-live, and none of them appear in any questionnaire.
Common questions
- What should you ask an AI vendor before buying?
- Start with who trained the underlying model and which version is in use, because a large share of AI products wrap someone else's model and a supplier who cannot answer cannot tell you when behaviour changes or pass through documentation you need. Then ask what happens when that model is deprecated, whether your data trains it, how long inputs are retained including for abuse monitoring, and whether each commitment appears in the contract rather than on a web page.
- Do AI vendors train on customer data?
- It varies by supplier and by tier, and the distinction that matters is whether not training is a default setting or a contractual commitment. Settings can be changed, reset or applied per account rather than per organisation. Ask separately about retention for abuse monitoring, which commonly persists for a defined window even where training is disabled, so a supplier confirming they do not train has not yet answered whether they store your inputs.
- What is an AI output indemnity and what does it cover?
- A supplier commitment to defend the customer against intellectual property claims arising from what the model produces. The conditions matter more than the promise: cover typically requires keeping the supplier's safety filters enabled, not prompting deliberately for infringing content, and using specified model versions. Claims about the training data itself are a separate exposure and are often excluded, and liability caps are frequently a multiple of fees rather than related to actual harm.
- Should you run a pilot before buying an AI product?
- Yes, and against an evaluation set you build rather than a demo the supplier runs. Take around a hundred real cases with known correct answers, hold them back, and measure the product on them. Public benchmark figures say nothing about your document types, your customers' language or your edge cases. The exercise is skipped mainly because it requires someone internal to define what a correct answer is, which is uncomfortable and worth doing anyway.
- Does an ISO certificate mean an AI product is safe?
- No. An information security certificate confirms the supplier manages information security. An AI management system certificate confirms that a management system for AI exists and is followed. Neither says anything about whether a particular model is accurate, fair or suitable for your use. Certificates answer procurement questions about organisational maturity; only evaluation evidence on representative inputs answers questions about the system itself.
- What happens if a vendor changes the AI model behind their product?
- Behaviour can change without anything in your deployment changing, which means your testing, your documentation and your operator guidance can all become wrong overnight. Ask for the named model and version in the contract, a notice period before any change, and the ability to pin a version. Many suppliers reserve the right to change silently, and where that cannot be negotiated it should be recorded as a known limitation and monitored for.
