In short
- When the model is part of the product, its price, terms and location become part of your contract.
- Bring your own key, so usage and data terms are between you and the provider.
- Test the product on a mid-tier model, and test switching models during the pilot.
The AI demo was the best part of the evaluation. The assistant read a programme of sixty projects and wrote a sharp summary of what was slipping and why. The contract was signed in the spring.
By the following spring, three things have changed. The model provider has changed its prices. The organisation's security team has approved a different provider, with an endpoint in the region. And a newer model, cheaper and good enough, has come out. The programme office asks the vendor to switch. The answer is that the product is built on the original model, the prompts are tuned for it, and a change is on the roadmap.
The organisation bought programme software. It also bought an AI model, without meaning to.
What goes wrong
AI models change faster than procurement cycles. A contract signed for three years will see several generations of models, several price changes and, in many organisations, a change in what the security team allows. Programme software that cannot follow those changes passes every one of them on to the client.
The model is part of the product. When a vendor builds on one provider, the model is a design decision rather than a setting. The client cannot point the product at another model, and certainly not at one running inside its own network.
The prompts are tuned to one model. Even where switching is possible in principle, the product's instructions were written and tested against one model. Moved to another, it behaves differently, and nobody has measured how.
The cost is set by the largest model. A product tested only on the most capable and most expensive model needs that model to work well. The client pays for it, often through a markup inside the licence that nobody can see.
The key belongs to the vendor. When AI usage runs on the vendor's account, the data terms are the vendor's terms with its provider. The client cannot see the usage, set its own retention terms, or move the account.
Sovereignty is decided by the vendor's choice of provider. If the provider's endpoint is abroad, every AI request leaves the country, whatever the hosting contract says.
What good looks like
Model-agnostic AI means the client chooses the model and can change it, without changing the product. Five things to ask for, and to test.
Ask for the model as a setting
The product should accept any capable model the client chooses: an approved provider, a provider's endpoint in region, or a model running inside the client's own network. Ask the vendor to show the setting, and which models have been run through it.
Bring your own key
With your own API key, the AI usage runs on your account with the provider. You see the usage, the data terms are yours, and moving to another provider is a change of key rather than a renegotiation with the software vendor.
Ask for a commitment on a mid-tier model
The most useful single question is which tier of model the vendor commits to. A product that performs well on a mid-tier model costs less to run, and it is more likely to work on a model your security team can approve or host. Run your own questions on that model during the evaluation, on your own data.
Put a cap on the spend
AI usage is a running cost, and it grows with adoption. Ask for a daily cap that an administrator can set, for the whole platform and for each part of the organisation, with a clear message when it is reached. A cap turns an open-ended cost into a budget line finance can approve.
Test the switch before you sign
During the pilot, change the model once. Note what changes in the answers, and how long the change took. Then write the right to change models into the contract, without a change request or a fee. The wider data questions for a tender are in data sovereignty and PMO software.
Expect three answers, and what to ask next
"Our model is the best, so you will not want to switch." It may be the best today. Ask what happens when it is not, and when your security team prefers another.
"You can switch, but results may vary." That is honest, and it means the switch has not been tested. Ask which models the vendor has run its product on, and with what results.
"The AI cost is included in the licence." Then the cost of the model sits inside a price you cannot see or control. Ask what the licence would be with your own key.
Lodestar is model-agnostic: Polaris works with any model through the client's own API key, or with a model hosted inside the client's own network. It runs on our own proprietary harness, tested across leading models, and we commit to its performance on mid-tier models. Administrators set daily AI spending limits for the platform and for each workspace, and client data is never used for training.
Questions for your next review
- Which AI model does our programme software use, and who chose it?
- Does the AI usage run on our own key, or on the vendor's account?
- On which tier of model is the vendor committed to performance, and have we tested it?
- What is our daily cap on AI spend, and who is told when it is reached?
AI models will keep changing for the life of any contract signed today. The software worth buying is the software that lets the client change with them.
