How to Design AI Agents

When I teach Nielsen's heuristics, part of the room reacts as if I were stating the obvious. A system should show its status. A person should be able to undo an action. An error message should explain what happened. All of it feels natural if you grew up with products where these principles were already the norm.

For me, at the time, they were a discovery. And I don't think the heuristics are out of date. If anything, they shaped digital products so thoroughly that we stopped noticing them. But lately I keep asking a different question: are these rules enough for an interface that doesn't just respond to a person, but can plan and act on their behalf?

Imagine an agent you asked to organise a work trip. It found flights, checked your calendar, suggested a hotel and started filling in the booking. While everything goes to plan, it feels almost like magic. But what if the agent decides to change the dates? What if the booking can't be cancelled for free? What if it messages a colleague without asking you? And at what point should you find out that something went wrong?

This is no longer a question of where a button goes. It's a question of the limits of autonomy, responsibility, predictability and control. And these decisions, I think, belong in a product designer's job. We no longer design only how a person drives an interface. We also design how the system acts without them, and when it should hand control back.

Seven decisions a designer makes about an agent

I wouldn't start with a chat interface, an avatar or a library of ready-made AI patterns. I'd start with seven decisions. They shape how the system behaves long before anyone picks a component.

These seven don't claim to be a new universal standard. They are my working synthesis of research from Microsoft, Google PAIR, Apple and Anthropic. What they're good for is turning an abstract "do agentic UX" into concrete product decisions you can discuss with engineers, managers and future users.

First decide whether autonomy is needed

Not every feature that uses an LLM has to become an agent. If a short report has to be pulled from three known sources at 8:30 every morning, a workflow with fixed steps is often enough. If a user asks "figure out why we didn't get paid and suggest what to do next", the system may have to decide for itself what to search, which tools to use and what to ask. Anthropic's Building effective agents draws exactly this line: workflows follow predefined paths, while agents dynamically direct their own process.

I like the model-first approach, where a team first explores what a model can and can't do and only then shapes the product idea. It stops you copying familiar competitor screens just because they exist. But model-first doesn't replace user-centred design. That a model can send a message on its own is not reason enough to let it.

Autonomy should depend on the action, not on the agent's name

The most useful model for starting this conversation with a team, I think, is risk × reversibility. Checking calendar availability, drafting an email, sending it and making a payment are very different levels of responsibility. They shouldn't get the same approval rule just because one agent does all four.

One caveat: even "easy to undo" doesn't always mean "safe". You can delete a sent email from your own mailbox, but you can't reliably take it back out of someone else's inbox or memory. A designer has to weigh the real consequences, not just whether an Undo button exists.

Permission, explanation and recovery are designed together

You can make an agent ask before every action, but then why delegate the work at all? Or you can let it do everything automatically, and the user loses the chance to catch a mistake in time. The balance depends on context. I'd separate three things: agreeing to the plan, agreeing to a specific risky action, and being able to step in while it runs. These are three different mechanisms, not one screen with a Confirm button.

The same goes for explanations. People rarely need a log of every click and tool call. It's far more useful to know that the agent changed the agreed plan, found conflicting data or couldn't finish an important part of the task. So the main view should show progress and consequences, and the detailed log should stay one tap away. When the agent makes a mistake, the product should offer the next safe step, not just "Something went wrong".

What research shows, beyond our own assumptions

In 2019 Microsoft Research published 18 Guidelines for Human-AI Interaction, tested with 49 design practitioners against 20 AI products. They cover four phases: the first encounter, regular use, errors, and how the system changes over time. They came before the current wave of LLM agents, but they explain well why a product should be honest about its abilities, support correction and never change its behaviour quietly.

In 2026 an Apple team studied the UX of agents that operate interfaces directly. The researchers first mapped existing designs and interviewed eight UX and AI practitioners, then ran a Wizard-of-Oz study with 20 participants, in which a researcher played the agent. People wanted to see progress and understand the reasons behind important decisions, but not to supervise every click. They also expected to be able to step in, especially when the task turned risky or the agent strayed from the agreed plan. It's a small qualitative study, not universal statistics for all agents, but it helps form UX hypotheses you can test.

Probably the most interesting numbers come from Anthropic's Measuring AI Agent Autonomy in Practice (2026), which analysed real use of Claude Code. Among newer users (under 50 sessions), full auto-approve was on in roughly 20% of sessions; by 750 sessions it was over 40%. At the same time, interruptions went up: new users (around 10 sessions) interrupted the agent in about 5% of turns, experienced users in about 9%. So experienced users don't simply "trust more". They change how they supervise: they approve less up front and step in more when it matters. These numbers are about Claude Code, so mostly programming, and they don't prove people would behave the same way with a finance or health agent.

The same study has another finding that matters for designers: on the hardest tasks, Claude Code stops to ask for clarification more than twice as often as people interrupt it. So "ask the person" is not only a button in the interface. It's also model behaviour that can be designed and trained, which is why "when to stop" is its own decision among the seven.

There is a more cautious view too. The position paper AI Agents Push Humans Out of the Loop (NeurIPS 2026) argues that current agent design doesn't support effective oversight, and that long use erodes the very skills oversight needs. A routine "yes" to every request is not control. Anthropic reached a similar conclusion from practice: when there are dozens of actions, people stop reading approvals, which is why Claude Code got Plan Mode, where a person reviews and edits the plan up front instead of every step. Good control doesn't mean the most confirmations. It means a person can understand the state of the work and step in exactly when it's needed.

This research helps us revisit the classic heuristics, not replace them. Visibility of status now means not just a loading indicator but readable progress on a long task. User control includes access rights, intervention and recovery. Error prevention covers not only field validation but the limits of what a model can do at all. And consistency matters even more when the system changes its own plan.

Where the guidelines agree, and where they part

Put Microsoft, Google PAIR, Apple and Anthropic side by side and the core is shared. The differences start where AI stops suggesting and starts acting on its own.

Shared

• Be honest about what AI can and can't do. For Microsoft these are the first two guidelines, for PAIR a chapter on mental models, for Apple a category of its own, for Anthropic the principle of transparency.
• Control stays with the person. Stop, correct, decline, take over: the 2019 guidelines and the 2026 agent research say the same.
• Errors will happen, so recovery is designed in advance. Microsoft has a whole phase for when AI is wrong, PAIR a chapter on what to do when things go wrong, Apple a set of findings on agent errors.
• Trust should be calibrated, not maximised. A person should trust the system exactly as much as it deserves.
• Explain why the system did something. Not the whole inner workings, just the reason for the decision.
• Behaviour changes over time, and people should be told. Microsoft writes about updating behaviour, Apple about replanning in the middle of a task.

Where they differ

• How firm the rules are. Microsoft gives 18 concrete guidelines. PAIR gives patterns around the questions a team should ask. Apple deliberately gives no prescriptions, only a map and a vocabulary, because the authors think the field is still too young.
• Who acts. Microsoft and PAIR wrote for AI that suggests while a person decides. Apple, Anthropic and Microsoft's Magentic-UI write about an agent that acts. That's why irreversible actions, such as payments or emails to people, barely appear in the early guidelines.
• Where control sits. Magentic-UI and the participants in Apple's study favour permission before a specific risky action. Anthropic finds that approving every step wears people out and moves control to the level of the plan. Feng, McDonald and Zhang (2025) suggest choosing the level of autonomy as a decision in itself, from an operator who drives every step to an observer who can only stop the agent.
• How much to show. The early guidelines lean towards explaining more. In Apple's study people had no intention of watching every step, and one participant suggested progressive disclosure on their own.
• Who stops. Almost everyone writes about the person interrupting the agent. Anthropic adds the other side: the agent should stop and ask by itself.
• Whether AI is needed at all. PAIR starts with exactly this question. Microsoft and Apple assume the product already has AI.
• What the conclusions rest on. Microsoft tested its guidelines with practitioners on 2019 products, PAIR draws on Google's experience, Apple on a study with 20 people, Anthropic on millions of real interactions, but only in its own products.

The shared core can be learned as a foundation today. The differences are the open design questions, and each team answers them for its own product.

How to test agentic UX before launch

In classic products we often test whether a person can complete a task. With agents I'd add another question: can a person understand what the system has already done, what it will do next and how to stop unwanted consequences? So a happy path isn't enough. You have to study uncertainty, errors and changes of plan.

For a trip-planning agent, for example, I'd test situations like these: it found a cheaper flight but with an awkward layover; a non-refundable booking has to change; the calendar and a colleague's message contradict each other; the agent loses access to email halfway through. For each one, check whether the person noticed the risk, understood the consequences and could take back control.

You can start before the agent exists. In Wizard-of-Oz testing a researcher plays the system by hand while the participant uses the prototype as if the agent were working on its own. That lets you test approval policies, error responses and expectations before the hard engineering. Five early interviews can surface problems, but they can't prove statistically that one design beats another.

I'd suggest a team measure not only task success rate but also the share of risky actions taken with proper approval, time to intervention (how quickly a person can stop an unwanted action), recovery success after an error, and unnecessary approvals per completed task. For background agents, it's also worth checking whether people learn about significant changes of plan in time. These are candidates for product metrics, not ready-made industry benchmarks.

Where designers can learn this

I wouldn't look for the one course that lets you call yourself an AI agent designer. The field moves fast, and what you need sits where human-computer interaction, product thinking, behavioural research and a technical grasp of how models and tools work meet. It's more useful to build your own curriculum on three levels: principles, patterns, and how an agent works inside.

The best complement to reading, I think, is building a small agent of your own. It doesn't have to be a full product. A scenario where the agent reads some information, makes a plan and asks permission for one important action is enough. Once you watch it make mistakes, replan and ask for clarification, it becomes clear very quickly why agentic UX is not the design of a chat window.

Who to follow

• Emily Campbell (The Shape of AI): the most complete catalogue of AI patterns.
• Vitaly Friedman (Smashing Magazine): AI patterns across hundreds of real examples; runs the Design Patterns For AI Interfaces workshop.
• Saleema Amershi (Microsoft): co-author of the Human-AI Interaction guidelines and of the Magentic-UI agent.
• Ruijia Cheng and Jeffrey Nichols (Apple): empirical research on agent UX.
• Kevin Feng (University of Washington): levels of agent autonomy.
• Jakob Nielsen (UX Tigers): predictions about AI and UX that are worth arguing with.

What stays the same

Nielsen's heuristics haven't lost their meaning. People still need to understand the state of the system, fix mistakes, recognise familiar patterns and stay in control. And human behaviour stays at the centre: designers spent years learning to understand it, and that skill is now needed for a new question, how people behave with AI, what they expect, when they trust and when they check. That's why the chat is so hard to replace, even after years of attempts: it's the most familiar form of conversation, and people come to AI with ready expectations of it. The question isn't chat or no chat, but what surrounds it: the plan, the status, the permissions, the control. What changes with agents is the scale of design decisions. It's no longer enough to predict what happens after a button is pressed. You have to design how the system behaves when there may be no next button at all.

Does it need to be an agent? What is it allowed to do? When should it stop? What does the person see while it works? How do they learn about a mistake and take back control? If a team can't answer these, the problem isn't missing AI components in the design system. The team hasn't yet decided how the product itself should work.

“

I think the future of agentic UX isn't about making a system as autonomous as possible. It's about making its autonomy understandable, appropriate and controllable.

What would prove me wrong

This argument would weaken if agents with full autonomy and no approvals consistently earned more trust and made fewer mistakes than agents with well-designed stopping points. Or if people who work with agents for years stopped intervening altogether, and nothing bad happened. Or if designers trained only on the classic heuristics designed agents no worse than those who learned from the new guidelines. If you have data like that, I'd really like to see it.

In short

• Nielsen's heuristics still hold, but they were written for an interface that waits for a command. An agent plans and acts on its own, so the old rules face new questions.
• Start with seven decisions, not screens: whether you need an agent, access, level of autonomy, when to stop, what's visible, how to undo, how the agent talks about itself.
• Autonomy is chosen per action: the riskier and less reversible the action, the earlier the agent asks. And "can be deleted" doesn't always mean "safe".
• With experience, people give an agent more freedom but also step in more often. Design a trajectory of trust, not one level of control.
• Microsoft, Google PAIR, Apple and Anthropic agree on the core: honesty about abilities, control, recovery, calibrated trust. They differ on where control sits and how much to show.
• Test error and change-of-plan scenarios, not only the happy path. Wizard-of-Oz lets you do it before the agent exists.
• You can learn this for free, but the best thing is to build a small agent yourself and watch how it fails.

If you're designing an AI agent and want to check how it behaves in risky scenarios, write to me.