In short
- An AI pilot with no baseline can be called a success by anyone and proved by no one.
- The benefit owner runs the work the pilot changes, outside the team that built it.
- Write the stop condition when the pilot is funded, and let the gate apply it.
The executive committee asks the programme office for a view of all the AI work in the organisation. Three weeks later the office has a list of pilots from six departments. Some were paid from an innovation budget, some from a supplier's free trial, some from a team's own operating spend. Each has a sponsor and a demo. None has a number for how the work was done before the pilot started.
The committee's next question is which ones to scale. The office can say which pilots are running and roughly what they cost. It cannot say which of them changed anything, because nobody wrote down what "anything" was.
Being asked to govern work that was set up to sit outside governance is an uncomfortable seat. In my view the way out is to stop treating AI as a special category, and to govern it the way the organisation already governs money.
What goes wrong
An AI pilot is usually framed as an experiment, and the word does a lot of work. An experiment is allowed to fail, which is reasonable. It is also excused from everything an investment owes: a business case, a baseline, an owner of the result, a date to decide. That second part is where the trouble starts.
No baseline, so no comparison. A pilot that drafts contract summaries "saves time". Against what? If nobody timed the work before go-live, the saving is whatever the sponsor says it is. Once the tool is in daily use the old way of working is gone, and it cannot be measured afterwards. A baseline is taken before, or it is never taken.
No owner of the benefit. The team that built the pilot owns the model. The benefit lives in a department that has to work differently: fewer hours on a task, a faster turnaround, less rework. If nobody in that department has signed up to the number, the benefit has no home. The builders report usage, because usage is what they can see.
Usage stands in for value. Logins, prompts and active users are easy to count, and they rise as soon as a tool is available. They show that people tried it. They say nothing about whether the work became cheaper, faster or better.
Nothing is ever stopped. An experiment with no end date and no stop condition is only ever extended. Each extension is small, so none reaches the committee. A year on, the list is longer, every pilot is still running and still paid for, and none has been scaled.
The cost is spread too thin to see. Licences sit in one budget, the people time in several others, the data work in IT's run costs. No single line is large enough to attract a question, so the total is never added up.
The PMO then inherits a portfolio it cannot judge, because every number it would need was never captured.
What good looks like
None of this is new. It is the discipline the organisation already applies to a new building or a new finance system, applied to a smaller and faster kind of investment. Five moves.
Put every AI initiative on one register
Use the register that holds the rest of the portfolio, not a separate list for AI. Each entry carries a sponsor, the process it changes, the stage it is in and its full cost: licences, the people time of the team using it, and the data work behind it. A pilot running on a free trial still spends people's weeks. The rule is simple: work that is not on the register does not get people or money.
Measure the work before go-live
Pick two or three measures of the process the pilot will change: time per case, turnaround, rework, cost per transaction. Measure them on the current process for long enough to see its normal variation, by the people who will own the measure afterwards, and write down how it was taken. The baseline belongs to the process, so it still means something if the tool is replaced.
Name a benefit owner in the business
The benefit owner runs the work the pilot changes. They agree the target, sign the baseline and report the measure after go-live. Their signature on the baseline is what turns a number into a baseline. The team that built the model can report usage, cost and reliability, and that is useful, but it is a different report. The wider case for an owner outside the project is in Delivered, and no benefit.
Make each gate ask for evidence
The gate into a pilot asks three things: is the baseline measured, is the owner named, and is the stop condition written. The gate into scale asks for the measure against the baseline, from the owner's reporting, and for the risk and control questions to be answered. Usage figures can be attached. They do not pass a gate on their own.
Write the stop decision when the pilot is funded
State what result, by what date, would stop the pilot. When the date comes, the gate applies it. A stop is recorded as a decision, with what was learned and what happens to the licences and the data. Extending a pilot is a decision too, with a new date and a reason in the minute, never the default for a pilot that ran out of time.
Expect three kinds of special pleading
"It is too early to measure." Then it is too early to fund at scale. The baseline costs nothing to take before go-live and cannot be taken after it.
"The value is strategic." Strategic value still shows up somewhere: a capability the organisation did not have, a decision made faster, a risk reduced. Name where, and who would notice.
"Stopping wastes what we spent." The money is spent either way. A stop on evidence is the gate doing its job, and it releases people for the next pilot. For how to write a case that survives that conversation, see A business case without invented savings.
Give AI work the same route as every other project
The discipline holds only if AI work travels the same route as the rest of the portfolio, so nobody has to remember a separate rule.
In Lodestar an AI initiative is a project like any other, on the same lifecycle stages and gates. The gate into delivery can require a charter whose benefits table becomes the project's benefits register, each benefit with a target, a realised value and a linked KPI that carries its own baseline and owner. Benefit values count once a governed review accepts them.

Questions for your next review
- How many AI initiatives are running today, and how many of them are on the same register as our other projects?
- For each pilot, what measure of the work was taken before go-live, and who signed it?
- Who in the business owns each benefit, and have they agreed the target?
- What result, by what date, would make us stop each pilot?
- Which pilot did we last stop, and what did we record about why?
AI work is new, and the questions it has to answer are old. A portfolio that asks them before the money is spent can tell its committee what changed.
