Most AI buying calls follow the same script. Someone asks about SOC 2, someone asks whether the data trains the model, someone asks about pricing tiers, and everyone signs.
Those questions are fine. They are also the ones every model vendor has answered four hundred times and has a slide for. The questions that decide what happens to you in eighteen months are the ones nobody asks, and this piece is nine of them.
Each one comes with the same three notes: why it matters, what a good model vendor answer sounds like, and what a bad one sounds like. Take them into the call word for word.
Key Takeaways
- A retention promise is not a retention guarantee. A US court order in May 2025 forced OpenAI to preserve output logs it would normally have deleted, including standard API traffic. Zero-retention and certain enterprise agreements were carved out.
- Notice periods vary by an order of magnitude. Anthropic commits to at least 60 days before retiring a public model. OpenAI’s published floors run to six months for generally available models and about two weeks for previews.
- Aliases float. A model name without a dated snapshot suffix points at whatever the vendor ships next, which means a silent behavior change you never approved.
- Indemnities cover copyright, not correctness. Every major shield has conditions, and disabling or circumventing safety features voids most of them.
- Your model vendor’s subprocessor list is the real contract. No-training clauses matter only if they flow down to the foundation lab underneath.
- EU buyers can demand documentation by law. Article 53(1)(b) of the AI Act entitles downstream providers to an Annex XII package, and most integrators never ask for it.
Quick Navigation
- Why the Standard Questionnaire Misses Your Model Vendor
- Round One: What Your Model Vendor Does With Your Data
- Round Two: How Long Your Model Vendor Keeps the Model Alive
- Round Three: What Your Model Vendor Owes You in Writing
- Round Four: Where the Model Vendor Indemnity Actually Stops
- Round Five: What Happens When You Leave the Model Vendor
- Scoring the Answers: A Model Vendor Walk-Away Test
- The Verdict: A Model Vendor Is a Dependency, Not a Purchase
- Frequently Asked Questions
Why the Standard Questionnaire Misses Your Model Vendor
Standard software buying asks about uptime, security and price. Those questions assume the thing you bought stays the thing you bought, which is exactly what a model vendor cannot promise.
A model does not stay put. It gets retired, re-pointed, re-tuned and re-priced, and the behavior you tested in March may not survive to September.
So the useful model vendor questions target change rather than state. Not “is it secure today” but “what happens when it changes, and who tells me”.
The nine below sit in five rounds. Ask them in order, because each round narrows what your model vendor can plausibly claim in the next one.
Round One: What Your Model Vendor Does With Your Data
Question 1: What happens to our retention promise under a litigation hold?
Why it matters. A retention policy is a promise between you and your model vendor. A court is not part of that deal.
In May 2025, a federal judge told OpenAI to keep and set aside output log data it would normally delete, as part of the New York Times copyright case. The order was two pages long and it reached standard API traffic that had a 30-day deletion policy.
The order was lifted on 26 September 2025 and normal deletion resumed. Data from the April to September window stayed in secure storage, and in November 2025 the court ordered production of 20 million de-identified logs to the plaintiffs.
Crucially, OpenAI stated that zero-data-retention endpoints and certain enterprise agreements sat outside the hold. That is the real lesson for model vendor selection: architecture protected customers where policy did not.
A good answer sounds like. “We are subject to litigation X. ZDR endpoints are excluded. Here is who at the model vendor calls you if a hold ever touches your tenant.”
A bad answer sounds like. “Our policy is 30 days.” That is the policy. You asked about the exception.
Question 2: Is the foundation lab a subprocessor of your model vendor?
Why it matters. Most AI products are a wrapper. Your no-training clause is worthless if it stops at your model vendor and the lab underneath has different terms.
Ask your model vendor for the subprocessor list. Then ask whether the training ban, the retention window and the regional processing promise each flow down to every name on it.
A good answer sounds like. A named list, a link to a change-notice page, and proof the flow-down is in the contract rather than just words.
A bad answer sounds like. “We use a leading foundation model provider.” A vague model vendor answer here is almost always a gap, not a security habit.
Round Two: How Long Your Model Vendor Keeps the Model Alive
Question 3: How much notice does the model vendor give before retirement?
Why it matters. This is the most underasked model vendor question in AI buying, and the spread between providers is huge.
Anthropic publishes a lifecycle table with four states: active, legacy, deprecated and retired. It promises at least 60 days of notice before it retires a public model. A third-party tracker computing the real gap across 19 Anthropic models with both dates found a median of 63 days, ranging from 60 to 189.
The same tracker puts OpenAI’s published floors at roughly six months for general models, three months for special versions and about two weeks for previews. Mistral’s median lands near 91 days.
Sixty days is a real constraint. It barely covers re-running an eval suite, re-tuning prompts and shipping a tested migration, and it fails outright if your model vendor’s notice lands during a code freeze.
One more detail catches teams out. Retirement dates on Anthropic-operated platforms do not govern Amazon Bedrock or Google Cloud, which set their own schedules, so your model vendor answer depends on which door you came through.
A good answer sounds like. A minimum notice period in the contract, a public lifecycle page, and a named migration contact at the model vendor.
A bad answer sounds like. “We’ll let you know.” Get a number into the agreement, and ask for twelve months if the workload is regulated.
Question 4: Does our model identifier pin to a snapshot, or float?
Why it matters. Model vendors publish both dated snapshots and friendly aliases, and the alias points at whatever ships next.
That is convenient for demos and dangerous for production. If your integration calls the alias, your model vendor can change the behavior under you without breaching a single term.
Ask your model vendor three things: whether you are pinned, how long a pinned snapshot stays servable, and whether change notes ship with each new snapshot.
The ground has shifted here. In November 2025 Anthropic published Commitments on Model Deprecation and Preservation, pledging to keep the weights of every public model for at least the life of the company, and to run a set interview with each model before it retires.
Preserved weights are not the same as continued access. Even so, it is the first public commitment of its kind, and a fair bar to hold any model vendor against.
A good answer sounds like. “You are pinned to a dated snapshot. Snapshots stay live for N months. Change notes ship with every release.”
A bad answer sounds like. “We always use the latest and greatest.” That is a silent update policy dressed as a feature.
Round Three: What Your Model Vendor Owes You in Writing
Question 5: Will you deliver Annex XII documentation on every major update?
Why it matters. If you sell into the EU, this is a legal right against your model vendor that your team has probably never used.
Article 53(1)(b) of the EU AI Act tells providers of general-purpose AI models to give downstream providers the information listed in Annex XII: intended tasks, acceptable use, what the model can and cannot do, how it is built and how to plug it in. Those duties took effect on 2 August 2025, and the Digital Omnibus did not move them.
Signatories to the GPAI Code of Practice also promise to answer fair follow-up requests from downstream providers within 14 calendar days.
Here is the practical trap. Teams building on top cannot finish their own technical file without that package. Most never ask for it, and almost none write delivery-on-update into the model vendor contract.
We mapped the wider GPAI picture in our piece on the four gaps between GPAI obligations and the US patchwork, and the evidence classes regulators actually ask for sit alongside it.
A good answer sounds like. “Here is our current model documentation form, and yes, we will agree in writing to reissue it on every major version.”
A bad answer sounds like. “That is covered in our model card.” A model card is marketing-adjacent. Annex XII is a defined field list, and your model vendor knows the difference.
Round Four: Where the Model Vendor Indemnity Actually Stops
Question 6: What voids your model vendor’s copyright indemnity?
Why it matters. Almost every major model vendor offers some form of copyright shield, and almost every one is conditional in ways buyers skim past.
Microsoft’s Customer Copyright Commitment, for example, stacks conditions onto the base agreement. The customer must not tamper with safety systems, must hold rights to the input, must not use output it knew or should have known was infringing, and on Azure OpenAI must turn on the mitigations the docs require. Trademark claims are cut out entirely.
OpenAI’s Copyright Shield covers Enterprise and API customers, not free or Plus tiers, and it drops away where safety or citation features were switched off or ignored. Anthropic sets IP indemnity terms in its enterprise agreement rather than through a standard public scheme.
Notice the shared shape. Turn off a content filter for a sound engineering reason, and you may have quietly voided your model vendor coverage.
A good answer sounds like. A written list of the exact settings that must stay on, plus proof the shield survives your fine-tuning plans.
A bad answer sounds like. “We fully indemnify our customers.” Ask which conditions apply, then watch the pause.
Question 7: Who pays when the model is simply wrong?
Why it matters. A copyright shield is not an accuracy shield, and this is where the model vendor liability chain usually breaks.
Provider terms usually deny any promise about the accuracy of outputs, and cap total liability at the fees you paid in the past twelve months. A wrong dose, a wrong figure in a filing, a wrong sign-off: none of that sits inside an IP carve-out.
So the answer is almost always “you do”. Ask anyway, so it is explicit before your own customer contract promises something your model vendor never did.
A good answer sounds like. A plain statement of the cap, plus a talk about insurance and where a human must review.
A bad answer sounds like. Anything that implies the model vendor absorbs downstream harm.
Round Five: What Happens When You Leave the Model Vendor
Question 8: What do we get back, and in what format?
Why it matters. Model vendor exit terms tend to cover data export and stop there. The valuable assets sit elsewhere.
Ask for your fine-tuned weights or adapters, your eval sets and scores, your prompt and tool settings, and your full request logs with timestamps. Ask what format each one arrives in, and how long you have to collect it after you leave.
Ask one more thing: what deletion evidence your model vendor provides, and whether it covers backups and subprocessors.
A good answer sounds like. Named files, named formats, a stated window and a deletion certificate.
A bad answer sounds like. “You can export your data through the dashboard.”
Question 9: What capacity and rate limits are actually committed?
Why it matters. Most model vendor SLAs cover whether the endpoint is up, not how much you can push through it. Those are different promises, and only one protects a launch.
Ask whether your rate limits sit in the contract or in their discretion, what your model vendor does with them in a crunch, and whether pricing is locked for the term.
Then ask for twelve months of status page history, including slowdowns rather than only full outages.
If the answers here are soft, the honest comparison is against running the model yourself — a calculation we walked through in the hidden fees in a self-hosted LLM bill.
A good answer sounds like. Committed throughput, a written escalation path and notice before any price change.
A bad answer sounds like. “Limits are generous.” Generous is not a number.
Scoring the Answers: A Model Vendor Walk-Away Test
You will rarely get nine clean answers from any model vendor, and you should not expect to. What matters is which ones come back vague.
Use a simple rule. Any model vendor answer that offers a marketing phrase instead of a number, a name or a clause reference counts as a fail.
| Fails | What it means |
|---|---|
| 0–1 | Normal. Close the gap in the contract and proceed. |
| 2–3 | Negotiate. Push the weak answers into written terms before signing. |
| 4+ | Walk, or pilot only. The model vendor has not thought about your risk. |
One caveat, stated plainly. A startup that says “we do not have that yet, here is our plan” is a better partner than a big vendor that answers smoothly and promises nothing.
The Verdict: A Model Vendor Is a Dependency, Not a Purchase
Software buying asks what you get. Model vendor buying should ask what you depend on, because the thing you tested will change while you are still using it.
Every question above targets that difference: retention under legal pressure, notice before retirement, pinning versus floating, documentation on update, the conditions that void a shield, and what you carry out the door.
None of them are exotic. They are simply the questions a model vendor does not volunteer, because the honest answers are complicated and the deal closes faster without them.
Ask them anyway. The cost of asking is one uncomfortable call, and the cost of not asking arrives with a 60-day migration notice during your busiest quarter.
Frequently Asked Questions
What questions should we ask an AI model vendor before signing?
Beyond security and pricing, ask your model vendor about retention under litigation holds, subprocessor flow-down, minimum retirement notice, snapshot pinning, Annex XII documentation on update, indemnity voiding conditions, liability caps for wrong outputs, exit artefacts, and committed rate limits. Those nine surface the risks a standard questionnaire misses.
Can a court override our model vendor’s data retention policy?
Yes. A May 2025 order in the New York Times case required OpenAI to keep output logs it would normally have deleted, including standard API traffic on a 30-day policy. Zero-data-retention endpoints and some enterprise agreements were excluded, which is why architecture matters more than policy wording here.
How much notice do AI providers give before retiring a model?
It varies widely by model vendor. Anthropic commits to at least 60 days for publicly released models, with an observed median around 63 days. OpenAI’s published floors are roughly six months for generally available models, three months for specialized variants and about two weeks for previews. Partner platforms such as Bedrock and Vertex set separate schedules.
What is snapshot pinning and why does it matter?
A dated snapshot ID locks you to one model version, while an alias points to whatever the vendor ships next. Call an alias in production and your model vendor can change how it behaves without breaking any term, so pin in production and test new snapshots on purpose.
Does an AI copyright indemnity cover hallucinations?
No. Copyright shields cover third-party intellectual property claims arising from outputs, subject to conditions. They do not cover factual errors, and model vendor terms usually disclaim output accuracy while capping liability at the fees paid in the previous twelve months.
Keep reading
Here are the latest posts from the blog.
Model Vendor Risk: 7 Dangerous Gaps in Your AI Contract
Benchmark Scores Fail: 5 Proven Reasons to Build Your Own
Agent Incident Response: 6 Proven Steps When the Log Lies
