↑↓ to moveEnter to openAll results →

    AI will polish a watermelon unless it reads the evidence

    A watermelon project is green outside and red inside. An AI summary written from owners' self-assessments makes the skin shine. Point the model at the evidence instead, keep the claim and the proof apart, and leave the rating with an approver who signs it.

    EVIDENCE AND ASSURANCEPolishing awatermelon

    In short

    • A model that summarises self-assessments writes the same optimism more fluently.
    • Give the model the files and the plan, and withhold the owner's claim until it has read them.
    • The model can say what the evidence supports. A named approver owns the rating.

    The monthly pack now opens with a summary page, written by an AI model from the owners' updates. It reads well. "The programme remains broadly on track. Minor slippage in procurement is being actively managed, and mitigations are in place for the two dependencies flagged last month." Every sentence came from something an owner wrote. None of it is false.

    Underneath, the "minor slippage" is the third re-issue of the same tender. The mitigation for one dependency is a meeting that has not been held. Both facts were in the owners' updates, in softer words. The summary softened them again, and the committee moved on.

    Programme people call this a watermelon: green outside, red inside. The old worry was that owners would hide the red. The new one is that the model will hide it for them, in better prose than any owner could write.

    What goes wrong

    Status leans towards optimism before any model touches it. I wrote about why in Green, and wrong: the owner grades their own work, and green is the cheaper answer. A summary written from those grades adds four effects of its own, all pushing the same way.

    Summaries compress the exception first. A model asked to condense forty updates into a page keeps what most of them have in common. Thirty-five greens are the common case. The red that one owner buried in a subordinate clause gets, at best, a subordinate clause of its own.

    Hedges survive and details do not. Owners write "being actively managed", "minor" and "on track subject to approvals". Those phrases are short and sound finished, so they pass straight into fluent prose. The awkward specifics, the tender number, the third re-issue, the date of the meeting that never happened, read as clutter and are the first thing cut. The hedge was the owner's signal, and the summary drops the very thing it was hedging.

    The model reasons towards the claim it is given. Tell a model that a milestone is 80% complete and ask it to summarise the evidence, and it will explain why 80% is plausible. The claim becomes the frame. Everything after it is read as support.

    Fluency borrows authority. A paragraph typed by a project manager reads as one person's view, and readers discount it. A clean summary at the front of the pack reads as the programme's view. It gets more credit than any of its inputs earned, and it carries no name.

    That last effect is the dangerous one. When the summary proposes the colour and the PMO accepts it, nobody rated the project. There is no one to ask how they knew.

    Summary of self-assessments Owner's self-assessment AI summary Green board pack fed in smoothed The claim, restated in better prose. Evidence read first Evidence files Model reads files Claim vs evidence Approver rates signs a frozen copy Owner's claim held back until the read is done
    The same model, two inputs. Only the lower route lets the evidence speak before the claim does, and leaves a named person to rate it.

    What good looks like

    The fix is to change what the model reads, and who signs what it produces. These moves work with any model and any tool.

    Give the model the evidence, and withhold the claim

    Feed it the files filed against the milestone, the plan, the gate's stated criteria and the definition of the KPI. Leave out the owner's percentage and colour. Ask it what the evidence shows has been done. Only then bring in the claim and compare the two. A reading made without the claim cannot be pulled towards it.

    Keep the claim and the proof in separate columns

    Every line of a status summary should show three things side by side: what the owner says, what the evidence shows, and the gap between them. Merged into one sentence, they read as agreement. Kept apart, the gap is the first thing a reader sees.

    ItemOwner's claimWhat the evidence showsFor the approver
    Systems integrator tenderMinor slippage, actively managedThird re-issue notice; no evaluation report filedAsk for the evaluation date
    Data migration, phase one80% completeTest report covers two of five data setsSupports two of five
    Land survey dependencyMitigation in placeMeeting scheduled, not held; no minuteNo mitigation yet

    Make every sentence point to a file

    Require each statement in the summary to cite the document and the passage it rests on. "No evidence found" is a valid result, and it should be printed in those words instead of being smoothed into "on track". A summary that cannot point to a file for a claim has told you something useful about the claim.

    Replace the hedge words

    Keep a short list of phrases that may not appear in a summary on their own: "broadly on track", "actively managed", "minor slippage", "mitigation in place". Each must be replaced by a date, a number or a document. The list will feel pedantic for a month. After that, owners start writing the date first.

    Leave the rating with a named approver

    The model can say what the evidence supports, what it does not, and what to ask. The rating belongs to the person your delegation of authority names, who is never the person who made the claim. If the approver agrees with the model, the rating is still theirs, with their name and the date, against a copy of the evidence that cannot change afterwards.

    Let the model read the proof. Let a person own the colour.

    That is how Polaris is used on evidence in Lodestar. When a completion or a KPI reading is submitted, it reads the attached files and works out the value without seeing the value claimed. It then tells the approver what supports the claim and what does not. It never approves. The approver signs a frozen, SHA-256-hashed copy of what was submitted, so the rating and its evidence stay together.

    Approvals queue with evidence items and their approval steps
    Approvals. Every governed change in one queue, with its route.

    Questions for your next review

    1. What does the summary at the front of our pack read: the owners' updates, or the evidence filed against the plan?
    2. For the last three items the summary called on track, which document shows each one?
    3. Which phrases in this month's summary could be replaced by a date, a number or a file?
    4. Who decided the colour of each project in this pack, and is their name on it?
    5. When the evidence and the owner disagree, where in the pack does a reader see it?

    AI makes a status report cheaper to write and easier to believe. The second part is safe only when the model has read the proof and a person has signed the rating.

    Hany Amer
    Written byHany AmerFounder and CEO, Lodestar

    Hany is a strategy and transformation executive with a technical background in software and AI. He has advised C-suites at a top-three strategy consulting firm (MBB) and independently, and has run transformation programmes of more than USD 1 billion.

    Connect on LinkedIn

    See it on your own programme.

    Send us a slice of your portfolio and we load it before the session, from strategy down to task.