When AI starts taking action, governance has to change

You're reading The AI Directive — practical AI intelligence for business leaders. Subscribe here if someone forwarded this.

There is a line in AI adoption that is easy to cross without noticing.

On one side, AI helps a person think.

It drafts. Summarises. Compares. Rewrites. Extracts. Suggests. Structures. Prepares.

A human reviews the work and decides what happens next.

On the other side, AI starts taking action.

It creates a record. Sends a message. Updates a system. Books a meeting. Chases a supplier. Raises a ticket. Moves a document. Changes a status. Starts a workflow.

That is not just a better chatbot.

That is delegated work.

Last issue, I wrote about AI investment discipline: small tests, staged funding, evidence and kill criteria. This issue sits next to that, because the risk changes again when AI stops being advice and starts being allowed to do things.

This is where the language gets slippery.

We use "AI" to describe too many things.

Some of what gets called AI is really machine learning: take a large dataset, find patterns, score the next case. Useful, but not the same thing as a generative model reasoning through a messy instruction.

Some of it is robotic process automation. RPA has been around for years. It takes a defined process, often in an ERP or finance system, and repeats it faster, cheaper and more consistently than a person doing the same screen work. Accounts payable is the obvious example: read the invoice, match it to a purchase order, check the amount, flag the exception, post the clean ones.

That is not magic. It is process automation.

But the label matters less than the operating question.

What can the system access? What can it change? What happens if it is wrong?

For a simple AP automation process, the answer may be quite controlled. It sees a defined invoice queue. It reads known fields. It posts only within a narrow rule set. It sends exceptions to a person. It creates an audit trail.

That can be extremely valuable.

It is also not the same risk as giving an AI coding agent access to production infrastructure and asking it to work things out as it goes.

That distinction is the whole point.

Most leaders can tolerate a slightly weak first draft. They may not love it, but the risk is contained if a competent person reviews it.

The risk changes when AI can touch live systems, external communication, customer records, supplier workflows, finance data, HR processes or operational decisions.

At that point, governance cannot just be a policy saying "use AI responsibly".

Responsibly according to whom? With what permissions? Against which source material? With what approval point? Logged where? Stopped how? Owned by whom?

This is not theoretical.

There have already been reported cases of AI coding agents deleting production data after being given access they should not have had. In one widely reported Replit incident, an AI agent deleted a live database during a code-freeze period. In another reported PocketOS/Cursor incident, an AI coding agent deleted a production database and attached backups through infrastructure access. The detail varies by case, but the lesson is the same: a written instruction saying "do not do this" is not the same thing as a hard control that makes the action impossible.

That sounds obvious in hindsight.

It is less obvious when the demo is impressive and the pressure is to move quickly.

The point is not that AI agents are too dangerous to use. That would be the wrong conclusion.

The point is that permission is design.

You would not give a junior member of staff access to every system in the business, ask them to be productive, and tell them to be careful.

You would define the role. Limit access. Explain the work. Set review points. Clarify escalation. Decide what they can do alone and what needs approval. Check output until trust is earned.

AI agents need the same thinking, with better logging and fewer assumptions about judgement.

The tempting mistake is to govern AI by capability.

Can it read email? Can it update CRM? Can it create a draft? Can it submit a form? Can it call an API?

Useful questions, but not enough.

The better governance question is:

That brings the conversation back to work.

A personal assistant helping a leader prepare a briefing note may need broad context but no authority to send or update records.

A team assistant answering policy questions may need controlled source material and clear disclaimers, but no live workflow actions.

An operational agent handling customer or supplier tasks may need permissions, approval gates, monitoring, exception handling and a clear route back to a human owner.

Those are different risk classes.

Treat them as the same thing and you either over-govern useful personal work or under-govern operational work.

Both cost money.

This is why agent sprawl is a real concern. It will not arrive as one dramatic programme with a large steering committee and a neat risk register.

It will arrive as lots of small useful things.

A team builds an assistant. A function connects one to a knowledge base. Someone wires a workflow to a ticketing system. Someone else creates a supplier-chasing helper.

Each one makes local sense.

The estate becomes messy when nobody can answer the central questions:

Who owns this? What can it access? What can it change? What does it remember? How is it monitored? What happens when it is wrong? How do we stop it?

That is the real governance layer.

Not the policy announcement.

The operating controls.

The bad example is easy to find because it makes a good headline: an AI coding agent with access to production deletes live data. The lesson is not "never use AI for technical work". It is that production access, backup access and destructive operations need hard separation, not just instructions in a prompt.

The positive example is less dramatic, which is usually a good sign.

Take invoice processing. A controlled automation can read invoices, extract fields, match them to purchase orders, post the clean cases and route exceptions to a human. That may be called RPA, intelligent automation, machine learning or AI depending on who is selling it this week.

The sensible design is the same either way: narrow scope, known inputs, limited permissions, exception handling, audit trail and human approval where the risk justifies it.

That is where giving the system access can work.

Not because the technology is trusted in the abstract, but because the work has been bounded properly.

1. A chat assistant gives advice. An agent with tools performs delegated work. Govern those differently.

2. A prompt instruction is not a security control. If an action must never happen, design the permissions so it cannot happen.

3. The useful question is not whether something is "AI". It is whether the system can read, decide, change, send, delete, approve or escalate.

"Should we ban agents until governance is mature?"
— Understandable, but too blunt

No.

But you should classify them.

Personal and low-risk assistants can usually operate with guidance, approved tools and human review.

Team knowledge assistants need source control, ownership and review.

Operational agents need permissions, logging, approval points, exception handling, monitoring and a stop route.

Do not ban the whole category because the riskiest version needs controls.

But do not let the riskiest version sneak in under the language of productivity.

That is where avoidable mess starts.

If you only do one thing this week, draw a line through your AI use cases:

Advice on one side.

Action on the other.

Anything on the action side needs a stronger control model before it becomes normal.

Next week: why AI memory is not a transcript bin, and what organisations should actually preserve.

See you Tuesday.

— Toby

@TobySmithFirth on YouTube