Just because AI can do it, does that mean it should?

Imagine an AI agent working on a customer request. It reads the message, figures out what the customer needs, finds information across several systems and decides what to do next. When information is missing, it looks for it. When something unexpected happens, it adapts.

So far, so good. Now let it make a final decision. Perhaps it decides whether the customer qualifies for something, whether an asylum seeker is granted asylum or whether a benefit is awarded.

Still comfortable?

Maybe. Maybe not. What changed was the life-altering potential of the decision we asked AI to make.

That distinction interests me because agentic AI naturally draws attention to what is technically possible. Architecture has to go further and ask where that possibility is useful.

From instructions to objectives

Much of traditional automation starts with instructions. We define the process, encode the rules, and tell the software what to do when particular conditions occur. An agent works differently. Give it an objective and the right tools, and it can decide how to pursue it, choose actions, interpret context and adapt as circumstances change.

This behavior helps with a problem that has frustrated automation efforts for years: the real world sometimes refuses to behave like a flowchart. Cases arrive with missing information. People express the same thing in different ways. Documents vary, and exceptions appear that nobody anticipated in the original specification.

Conventional software handles predictable processes very well. Traditionally, handling the unpredictable parts has required humans, so giving software more freedom is obviously appealing. But this freedom to reason is itself an architectural choice.

What kind of decision are we making?

For a moment, think about the work, not about the technology. A customer sends an email describing a complicated problem and attaches several documents. Working out what has happened requires interpretation. Several reasonable routes forward may exist. An agent can understand the material and could retrieve additional information. Using this additional information to decide what to examine next could be genuinely useful.

Later in the process, suppose we reach a governed calculation using an approved formula. Creative interpretation adds no value and isn't appropriate. Given the same validated inputs, we expect the formula to produce the same answer every time.

This distinction is where the conversation about autonomy becomes more interesting. A single business process can contain areas where we actively want the system to reason and others where consistency and approved formulas matter more.

So I wouldn't start an agentic AI project by asking how much of the process we can give to an agent. I would ask where the process benefits from autonomous reasoning, and what happens when that reasoning extends into parts of the process where the answer should already be known.

The pilot will not answer everything

These distinctions may not be clear at first in a pilot. We choose a use case, give an agent some tools and see whether it can complete the task. If it succeeds, we have learned something useful: the technology can perform the work under those conditions.

Production raises a different set of questions. What will the agent do across a million cases rather than a hundred? How often will it call another AI model or AI tool? What happens when a tool returns an unexpected result? How do we reconstruct the agent's actions when something goes wrong or when an auditor asks for it?

Then there is a deceptively simple question. If the agent receives the same facts tomorrow, do we expect the same answer?

For some tasks, perhaps we do not. Interpretation has room for variation. For others, we absolutely do! The answer tells us how much autonomy that point in the process has.

Saying no to AI can be a sound architectural decision

With a powerful new technology, we naturally look for places to use it. 

I understand the enthusiasm. I share much of it, but sometimes we need to think a little further.

Architecture is partly the discipline of choosing boundaries. An agent might have considerable freedom to investigate an unfamiliar case, understand unstructured material and determine which capabilities it needs. When the process reaches established business logic, the agent could hand the work to software designed to execute that logic consistently. We call this deterministic.

In another use case, we may give the agent much more freedom to reason. But there will also be situations where ordinary software solves the problem just as well. The hard part is recognizing the difference.

Saying no to AI at one point in a process leaves room to use it where autonomy adds value.

As agents become more capable, we will keep discovering things they can do – and things they can’t. Good architecture requires us to decide where to apply agentic AI.

 

We’re looking forward to our sponsorship at Adopt AI 2026, December 3-4, in Paris. Each Thursday leading up to the event, Agile & Digital Enterprise Architect Peter Kalmijn, a member of the Future Makers Research Community (FMRC) at Atos, will explain why Good AI architecture starts by saying no.