Promoted, Demoted, Trusted

How AI agents earn trust when they work among people

Built for Early Adopters

Most agents today are built for early adopters: people who like to experiment, tolerate risk, forgive rough edges and are happy to hand over access just to see what happens. It’s why an agent asking for access to your mailbox, your calendar and even your credit card information finds traction. It’s also why that traction almost surely stalls.

Geoffrey Moore called the gap between early adopters and everyone else the chasm. What gets you across it is confidence: that the thing will behave, and that there’s a way out if it doesn’t.

For agents, that confidence is trust. Trust is earned in private spaces, like a one-on-one interaction between a person and an agent. More crucially, it’s earned in shared spaces, where it depends on how the agent behaves, how it presents itself and how it represents its principal, the person it acts for.

In this essay I share my thinking on how trust gets built, what it buys, and what to do when it breaks.


From Tool to Persona

Components of trust

Before trust, a more basic question: what is an AI agent? Two schools of thought have formed around it:

  1. One says it’s a tool. A powerful one, but still an instrument.
  2. The other says it’s an agent in the full sense, closer to a colleague. It acts, it decides, and it should be treated as a participant in the work.

My take is that both are right, just at different points in time.

Every agent starts as a tool.

As its owner extends its agency, it picks up qualities we usually reserve for colleagues, and at some point it stops reading as a tool and starts reading as a persona. I don’t mean a person, but an abstract presence with a recognizable shape.

If I had to define a persona in this context, I’d say it has a point of view, agency, the judgment to ask the right questions, and continuity. The table below shows the range of each of these trust variables, from Tool to Persona.

Agent as a Tool Agent as a Persona
Point of view It reflects your view back to you It brings an independent opinion
The ability to push back Sycophancy gets in the way It disagrees when it has a real counterargument
Agency It mostly talks and needs hand-holding for actions It acts without being walked through each step
Judgment about permissions It asks permission for every step of an action Not asking less, but knowing what’s worth asking about. That’s the mark of a senior colleague too
Continuity Its context and memory don’t persist, and may differ depending on where it’s invoked It’s the same presence tomorrow, with a history of working with you or the team
Addressability It has a generic identity, and only the person using it invokes it It has a unique identity and a principal, and people other than its principal can address it
A tone of voice Generic Unique and consistent in its manner, values and tone

So if every agent starts as a tool, its principal can tune each of these variables, and the further right they go, the further the agent moves into persona territory.

In a shared space, that promotion happens on two layers at once:

  1. Institutional. Say a team lead lets the agent send routine replies to clients. The permissions variable has been promoted to persona level, and the agent has moved up a rung for everyone on the team. No matter who uses it, that quality now adds to the agent’s personification at the institutional level.
  2. Personal. Say one team member shares context with the agent, such as the working files she’s looking into. For her, the agent has moved further right: permission to reply to clients, plus access to her context, makes it much more like a valuable colleague. Another team member may choose not to share anything. To him, the agent is a tool with an unusually large permission set.

The same agent will have a different working relationship with every team member. For some it will be a tool, for others a persona, depending on how far the variables have been tuned to the right.


Promotion to trust

Trust grows in the order of reversibility

So how does a tool move to the right? A while ago I handed an agent the admin of building a speaking practice: finding topics and calls for talks, drafting abstracts, tracking what I’d submitted where, chasing follow-ups. This was work I didn’t have much skin in, and admin I didn’t want to spend time on.

What I started noticing was that the agent came back to me every so often, adding small tasks to my list. Record a two-minute intro by Thursday so I can finish this submission. Approve this abstract. That experience grew my trust in the agent, because by design it never sent anything out without my sign-off. Nothing it touched carried my identity, so my reputation stayed intact.

Trust between people and agents gets built the way it always has between people: in the order of reversibility.

  1. We start by interacting with an agent privately: asking questions, having conversations.
  2. With time, we get comfortable handing it work that’s easy to undo: drafting replies, preparing documents.
  3. Then work shared with others: posting a project update for the team, scheduling a meeting across several calendars.
  4. Then work that goes out in my name: replying to my kids’ teacher, RSVPing for us both.
  5. Finally, money and anything that can’t be taken back: solving my problem and paying for the solution when a purchase is needed.

Each rung is earned on the one below it, and mistakes get more costly as trust increases.


Visible until trusted

What trust buys

If an agent can earn its way from tool to persona, what does that trust actually buy it? The answer is invisibility in a shared space.

Here’s an example. J. Chen, a fictional character, receives an email from a customer. J. Chen’s project agent pulls the context together and drafts an answer in a minute. That answer can reach the customer in four ways, and each one is a rung on the trust ladder:

  1. The agent drafts, and J. Chen reviews it and sends it themselves.

  2. The agent sends it under its own name after J. Chen reviews it, so it’s marked as reviewed by J. Chen.

  3. The agent sends it under its own name without J. Chen’s review, because permission was given beforehand.

  4. The agent sends it as J. Chen.

As you can guess, the more trust J. Chen has built with the agent, the higher the rung it operates on. But on rungs 2 and 3, when someone’s agent sends a response without its principal’s direct participation, the receiver will have questions:

  • Did its principal actually sanction this, especially if the message doesn’t say it was reviewed?
  • Did they not think the customer was worth a direct message?
  • Did the agent translate the message correctly?
  • And why should the customer care about this response anyway?

The customer would likely ignore it.

Rung 4 is where trust is truly built. J. Chen lends the agent their name, but not their accountability. If the answer is wrong, it’s still their answer. That’s where the agent becomes invisible.

So the agent should be visible by default, and earn its invisibility over time through integrity, reliability and good behavior.

In shared spaces, invisibility has another payoff: less noise.


Demotion from trust

  • Trust feeds more trust: Trust leads to more delegation, which gives you time back, which makes the agent more valuable, which builds more trust.
  • Drift: As trust grows, you check less. Errors start getting through unseen, and they stay unseen until one of them shows up somewhere public.

What happens next depends on the type of violation:

  • A competence failure, like a wrong fact that embarrasses the principal in public, doesn’t end the relationship. It demotes the agent. The principal pulls it down a rung or two, checks its work again for a while, and carries on, because it’s still valuable.
  • An integrity failure. is different. If the agent leaks something private, or exposes the principal to an attack, there’s no demotion. The human just leaves.

Competence can be coached. Integrity can’t be patched.

Persona may misbehave

How you manage trust

The more an agent becomes a Persona, the more we trust it with. Which raises the question every principal may eventually face: what if the Persona misbehaves?

This isn’t a new problem. Think about ambassadors. Their job is to represent their country, bring its point of view, negotiate on its behalf, commit and make decisions, all without constantly calling home. They arrive with letters of credence, a formal document from their head of state that says this person speaks for us.

And if an ambassador has ill motivations?

Well, there are levers. They can lose their title, their salary, their reputation and their standing back home. You can’t change an ambassador’s inner motivations, but if they misbehave, there are real consequences.

An agent is the reverse. If it drifts (or has ill motivations, or the wrong loyalties), you can change its instructions, correct its memory and teach it what you meant. What you can’t do is make it worry about consequences. There’s no real way to punish an agent the way you’d punish a person. An agent can’t be held accountable, only constrained.

Here’s a framework for how people can manage the trust they develop with their agents:

What you’d do to an ambassador What you can do to an agent
Take away the title Revoke permissions
Recall them home Pause it, or confine it to private work
Coach and correct them Change its instructions and memory
Rely on their ties to home Rely on the maker’s reputation, contracts and liability
Watch them Keep an audit trail; the consequence is being caught
Dismiss them Replace it; switching is cheap

Crossing the chasm

Early adopters are more forgiving than the mainstream.

What gets an agent across is everything above:

  • it starts as a tool and earns its way to persona,
  • it climbs in the order of reversibility,
  • it stays visible until it’s trusted,
  • and it can be demoted when it slips.

Time saved is what early adopters are looking for.

Time-to-trust is what mainstream will pay for.

If you’re building agents that work alongside people, one-to-one or in groups, here’s what to do:

  • Start on the bottom rung. Ask for the least access that still does useful work, and let the rest be earned.
  • Make demotion a feature. Give people ways to constrain the agent: narrow its reach, pause it, take work back. A customer who can demote your agent is a customer who’ll promote it.
  • Make the agent visible by default. Give it credentials: show whose agent it is and who reviewed the work. Let invisibility be earned.
  • Show the track record. Trust built on evidence travels further than trust built on a demo.
  • Treat integrity as the line. People will forgive a wrong answer. They won’t forgive a leak.

We’ve been sending agents to act in our name for as long as there have been names worth acting in. What’s new is that this one has nothing to lose, and it’s our job to design around it.

Subscribe on Substack →