Has your AI actually added value?

You're reading The AI Directive - practical AI intelligence for business leaders. [Subscribe here](https://theaidirective.co.uk/subscribe) if someone forwarded this.

Over the last two issues, I have written about what 90 days of practical AI work has taught me, and what I would do differently if I were starting again.

There is one more question to put at the end of that arc:

Has the AI actually added value?

That is not the same as asking whether people are using it.

It is not the same as asking whether the tool runs reliably.

It is not the same as asking whether it produces plausible outputs.

Those things matter, but they do not prove benefit.

This is where the wider business conversation is moving. McKinsey's 2025 State of AI work says AI use is now widespread, yet only 39 percent of organisations report EBIT impact at enterprise level. Deloitte's 2025 AI ROI research says 85 percent of organisations increased AI investment in the previous year and 91 percent planned to increase it again, while ROI remains harder to prove. Gartner makes the same point in a more operational way: GenAI ROI needs a comprehensive measurement approach, including total cost of ownership, not just productivity claims.

That is the right conversation for leaders.

Not "are we doing AI?"

Not "how many prompts have we written?"

Not "did the automation complete?"

The better question is whether AI is changing business performance in a way worth the investment, risk and management attention.

AI does not need a special exemption from normal business discipline.

In any serious project, you define the intended benefit before you start. You set a baseline. You decide how the benefit will be measured. You track cost. Then you ask whether the benefit has actually been realised.

AI should be treated the same way.

Before a team starts using AI in a workflow, the business should be able to finish a sentence like this:

We expect AI to improve [business outcome] from [current baseline] to [target result] by [review point].

That sentence is deliberately plain.

It forces the team to say what benefit they expect, not just what technology they are excited about.

It also stops AI work being judged only by process measures. A workflow running on time is useful operational evidence. It is not the benefits case.

The benefits case is whether the business is better off.

One reason AI ROI becomes confused is that leaders mix different levels of measurement together.

Technical performance gets treated as value.

Usage gets treated as value.

Productivity anecdotes get treated as value.

They may all be useful signals, but they are not the same thing.

McKinsey's recent measurement work is useful here because it separates AI value into layers: financial impact, strategic outcomes, operational KPIs, user adoption and technical performance.

That is a better frame than a single "hours saved" conversation.

I would adapt it like this.

This is the top layer because it is the hardest one to argue with.

Did AI help grow revenue?

Did it reduce cost to serve?

Did it improve margin?

Did it improve cash or working capital?

Did it avoid spend that would otherwise have been needed?

Did the total cost of ownership still make sense once licences, model usage, integration, governance, review and management time were included?

This does not mean every early AI use case needs a perfect P&L calculation. Some benefits will begin as leading indicators. But leaders should still know where the financial value is supposed to land.

If the value is "productivity", ask what happens to that productivity.

Does it reduce cost?

Does it increase capacity without hiring?

Does it improve sales conversion?

Does it allow faster delivery?

Does it free senior people from low-value work so they can spend more time on commercial decisions?

If the productivity claim cannot be connected to any economic outcome, it is not yet a benefits case. It is a hope with a nicer dashboard.

Not all value shows up immediately as cost saving.

Some AI value sits in strategic outcomes:

This is why a narrow efficiency lens can miss the point.

McKinsey's State of AI survey notes that the organisations seeing the most value often pursue growth and innovation as well as efficiency. Deloitte also points out that AI ROI leaders are more likely to define critical wins in strategic terms, including revenue growth opportunities and business model reimagination.

That matters.

If leaders only ask "how many hours did it save?", they may undercount the work that improves customer experience, decision speed or commercial focus.

The question is broader:

Which business outcome are we trying to move, and did AI help move it?

Operational measures still matter. They just belong in the right place.

This is where cycle time, throughput, rework, defect rate, first-contact resolution, cost per case, response time, backlog age and exception rate belong.

These are not low-level measures if they are tied to a business outcome.

Reducing elapsed time from ten hours to one hour matters if it shortens a decision cycle, improves service, releases capacity or reduces cost.

Reducing rework matters if it improves quality, customer experience or management confidence.

Reducing exception leakage matters if it avoids risk, missed revenue or operational drag.

The mistake is measuring operational activity in isolation.

"The AI produced the report" is not enough.

"The reporting cycle moved from three days to same day, with fewer corrections and no loss of quality" is a benefits statement.

That is the standard.

AI does not create value if it stays in a pilot corner.

This is the adoption layer, but it should not be treated as vanity usage.

The question is not "how many people logged in?"

The question is whether the right people are using AI in the workflow where the benefit is meant to appear.

Useful measures might include:

This is also where change management belongs.

Deloitte's work is useful here because it points to human-centred implementation, AI fluency and change management as part of ROI. That is right. A technically capable AI workflow that people do not trust, understand or use will not realise value.

Adoption is not the end goal.

But without real adoption in the work, the value probably will not arrive.

Technical measures are necessary, but they sit at the bottom of the value chain.

Leaders still need them:

But these are health measures, not the final business case.

A model can be fast, cheap and reliable while still being pointed at the wrong problem.

Equally, a workflow can be commercially valuable but too expensive, risky or fragile to scale.

This is where Gartner's warning on total cost matters. The cost of AI is not just the visible subscription. It includes compliance review, retraining, internal overhead, governance, data preparation, integration, support and the cost of failed experiments.

If those costs are not in the ledger, ROI will look better than it is.

One more point is worth making.

Not all AI work should be measured on the same timeframe.

Deloitte's research says leading organisations use different frameworks or timeframes for generative and agentic AI. That makes sense.

Generative AI used for drafting, summarising or comparison may show value quickly through productivity, quality and faster decisions.

Agentic AI, where systems take on multi-step workflows with more autonomy, may need a longer benefits window because the work involves process redesign, change management, risk control, data foundations and operating model change.

If you use the wrong measurement window, you get bad decisions.

You may kill strategic work too early because it does not show immediate savings.

Or you may keep weak automation alive because it looks busy in the short term.

The measurement has to match the ambition.

For a leadership team, I would keep the first scorecard short but serious.

1. Intended benefit

What business outcome should improve?

2. Value layer

Is this financial impact, strategic outcome, operational performance, adoption, or technical/control improvement?

3. Baseline

What is the current performance before AI?

4. Target

What would count as meaningful improvement?

5. Full cost

What will this cost in licences, usage, implementation, review, governance, training and management attention?

6. Evidence

After a sensible review period, what changed?

7. Decision

Keep it, fix it, scale it, or stop it.

That is a better conversation than "we are using AI".

It is also a more uncomfortable one, which is usually a sign that it is closer to the truth.

1. AI adoption is not AI value.

2. Define the expected benefit before the workflow starts.

3. Measure value at the right level: financial, strategic, operational, adoption or technical.

4. Include total cost of ownership, not just tool subscription.

5. Match the measurement window to the type of AI work.

Before approving another AI pilot, ask for this:

What business outcome will improve, how will we measure it, and what full cost are we accepting to get there?

If the answer is vague, the project is not ready.

Not because AI is wrong.

Because the benefits case is.

The question is not whether AI is impressive.

The question is whether it has made the business faster, better, cheaper, safer or more commercially effective.

That is the test leaders should hold it to.

See you Tuesday.

[@TobySmithFirth on YouTube](https://youtube.com/@TobySmithFirth)

These are tools I use personally. Affiliate links marked - I earn a small commission if you sign up, at no extra cost to you.

Some links in this issue are affiliate links. I only recommend tools I actually use.