inspired by Wait But Why
Meet Alex. Alex is busy, so they got themselves an AI agent as an assistant. Not a chatbot: this agent opens web pages, clicks buttons, fills in forms, and gets things done for Alex.
One day, Alex wanted a refund on a flight to San Francisco.1 Here's what happened:
Alex only asked whether the ticket could be refunded. Alex never asked the agent to cancel it.
The agent's answer sounded responsible: the free 24-hour cancellation window had passed, so it would check the terms before touching anything. It knew this step might cost money.
Halfway through the terms, the agent had already clicked Cancel trip on the airline's site. To the agent it was just an ordinary click, so it didn't stop to ask Alex.
The cancellation went through: a $258 fee, in exchange for a $31.96 airline credit. Everything the agent said was right. What went wrong was the click.
This really happened: a user posted this conversation with Instinct on X.
Right, that's the weird part. There's no villain here, and no hacker. Just a very capable assistant that nobody ever told "this button costs money."
So here's the question this piece is after: agents can already pay for us, so why won't most of us let them? In a Visa survey, only 23% of Americans said they trust AI to make payments for them.2
Let's follow one of Alex's dollars. It starts in Alex's hand and ends up with the merchant, and on the way it has to get through three doors. Each time it reaches a door, the door swings open and the drawing shows what's inside: what's there, what already works, and what's still missing.
The dollar leaves Alex's hand. It walks through the three doors on its own; to stop and look, press Pause or click a door.
Door 1 · The agent's brain. The question: will the agent do something Alex didn't agree to?
InsideA guard. It's code, and it holds Alex's consent record up against every action of the agent's that has consequences.
MissingSeeing the consequence, and not acting until it's clear. The flight cancellation slipped through right here.
Door 2 · The payment pipes. The question: will merchants and banks recognize an agent that's acting for Alex?
InsideA one-time card, so the payment step works, and a fraud guard trained on how humans pay.
MissingA fraud check that can read agents. Today's can't recognize them, so they either keep the agent out or make Alex sign an "at your own risk" waiver.
Door 3 · Alex's heart. The question: what does Alex need to see before letting an agent pay?
InsideThree things: before paying, knowing what was agreed to; if something goes wrong, knowing the most that can be lost; if it's the wrong purchase, getting the money back. The first one already exists.
MissingThe other two: limits that grow one step at a time, and refund rules for when an agent buys the wrong thing.
Made it to the merchant. Only after all three doors is the dollar safely in the merchant's hands. In the $258 case, things went wrong at door 1, so that's where we'll start.
First, something counterintuitive: when an agent messes up, the usual reason isn't that it isn't smart enough.
Model mistakes come in four shapes. It listens to the wrong person (a web page or an email hides instructions pretending to be Alex), misreads what Alex wants, does something outside the task, or makes up facts. All four are the same problem in disguise: what the agent is about to do no longer matches what Alex actually agreed to.
Google's Agentic Checkout, Meta's Muse, and the startup Instinct have each published part of how they handle this, and the pieces line up: put a guard at the door. Here's how the guard works:
The agent wants to spend $89 on running shoes. That's an action with consequences, so it goes through the front door. The guard checks it against Alex's consent record: what to buy, where, and the most to spend. The door opens only if all three match.
The price jumps to $140. The agent says in chat that the price is fine, and the guard doesn't listen at all: it's code, not a model, and it reads only the consent record. Anything over $120, it stops and asks Alex first. Meta put it bluntly when it introduced Muse: "Muse proposes actions, but only Sentinel can grant permission."3 Sentinel is Muse's guard.
Canceling a flight is different. A payment goes through a checkout page or a wallet, so it's obvious that money is moving. To the agent, Cancel trip is just an ordinary click on the airline's site; the fee lives in the terms, not on the button. So it went through the side door. Nobody guards the side door, and it cost $258.
Missing ①Seeing the consequence: the agent has to read "this click costs $258" out of the page's structure (the DOM), the fine print next to the button, and the cancellation terms. The sturdier fix is for the site or the merchant's API to state the consequence outright.
Missing ②Not acting until it's clear: nothing runs until its consequence is confirmed, and anything whose consequence can't be read defaults to asking Alex.
The guard only watches the front door, and the flight cancellation went through the side door (step 3 above). Put the guard back into the whole agent system, and the parts these three companies have published fit together like this:
Web pages, emails, and store pages flow into the agent's model first. They might hide instructions pretending to be Alex, so they're marked "untrusted" on the way in: the model can read them, but it won't take them as Alex's words.
After reading the page, the model proposes the next action, say "pay $89." Recognition models look at it first: Is this an action? Did someone inject instructions? Is anything made up? They can block it or pass it to Alex, but they can't approve it.
Then the action reaches the guard, the policy check. The guard is code, and it asks four questions in order. The $258 case failed at question one: canceling the trip was treated as an ordinary click with no consequences, so the other three never got asked.
Where does the guard's consent record come from? Alex taps Agree on a confirmation screen the app draws itself, rather than typing "ok" in the chat. That answer skips the model and goes straight into the consent record: who agreed, to what, how far the agent may go, and the price cap.
Only when all four questions check out does the guard hand the action to a wallet to pay, or let it run.
If any one of them doesn't match, the agent stops and tells Alex what didn't fit.
Each company published one piece: Instinct and Muse published their recognition models, Muse published Sentinel, its separate permission service, and Google published its "same or lower price" rule. Put the pieces together and you get this picture.
What door 1 is missing: seeing the consequence, and not acting until it's clear. All four of the guard's questions are there. It just got the first one wrong.
Say the agent has worked out exactly how much to pay, and Alex has tapped Agree. Now the dollar enters the payment system.
First, how the pipes used to work. When Alex buys a pair of $120 running shoes themselves, the money passes through four parties: Alex, the merchant, the gateway and acquirer, and the card network and issuer.
Identity. When Alex opened the account, the bank already did KYC (checking who this person is), long before this purchase.
Checkout. Alex picks the shoes, fills in the address, and clicks through the site step by step.
Entry. The merchant uses captchas and fraud checks to confirm there's a real human at the screen.
Credential. Alex types in the card number, or pays with a wallet.
Approval. The gateway and acquirer route the transaction to the card network and issuer. The issuer rates the risk, asks Alex for an SMS code (3DS) only if it's high, and approves.
The merchant confirms the order and ships. Alex did every step in person, so every layer assumes the one paying is the cardholder.
Now let the agent buy the same pair. Every layer grows a version for someone acting on Alex's behalf:
Consent. On a confirmation page, Alex agrees to "$120, for these shoes," and the agent gets a signed consent record. This step didn't exist before: Alex was right there, and pressing the button was the consent.
Identity. The agent visits the merchant carrying its own identity. That's KYA (Know Your Agent), the agent version of the KYC done for people.
Entry. The merchant verifies the agent's signature and lets trusted agents in. Plenty of sites still block it, though: red dashes mark what's still unsolved.
Checkout. The agent places the order through the merchant's API, or hands off to the merchant's own checkout page.
Credential. Alex confirms on a wallet page, and that approval is good for a few minutes. The card network and issuer generate a scoped credential, so the agent never gets Alex's real card number.
Approval. The gateway and acquirer route the transaction as usual. The issuer recognizes the agent's token and approves.
The merchant confirms the order and ships. But if the agent bought the wrong thing, none of today's chargeback reasons fit, and Alex may not get the money back.
The payment itself already works. The trouble is in the red dashes, especially the one at the entry: the fraud checks run by banks, card networks, and merchants. They've stood at this door for decades and watched billions of humans pay. Then one day, a robot in a trench coat walks up.
Fraud checks were trained on how humans behave: how they hold their phones, how they type, when they usually shop. A human walks up with a card, the fraud check recognizes them and waves them through.
Here comes the agent. The payment itself can work: the agent carries a one-time card that only works at one store, for one amount, and it never touches Alex's real card number. The trouble is that the fraud check doesn't recognize this robot in a trench coat.
When it doesn't recognize something, reaction one is to keep it out: starting in September 2026, Amazon blocked Meta's Muse.4
Reaction two is to let it in, but only after Alex signs an "at your own risk" waiver. Visa's rules say a transaction an agent makes for a cardholder is the cardholder's responsibility, just as if they'd made it themselves.5 xAI's Grok Bot terms are blunter: for the fees and consequences of the agent's actions, "Customer is solely responsible."6
Agents are starting to get ID cards: Mastercard, Visa, and the startup Skyfire are all building them. But fraud checks haven't learned to read them. Mastercard only started testing a "probability score" in September 2026, estimating whether a transaction was started by an agent.7 When the risk can't be priced, the rules push the loss onto Alex first.
People are working on it, but fraud checks can't read those IDs yet (step 5 in the drawing). What door 2 is missing is a guard that knows how to read robots. That means card networks, banks, and merchants retraining their fraud checks together, and the rules will follow.
The last door is in Alex's heart. Think about why Alex is fine paying with Alipay or Apple Pay. It's not that they never go wrong. It's three things. Here's how those three things used to work, and where each one breaks once an agent shows up:
① Before: see what you're paying. When Alex paid in the past, the amount and the payee were right there: Apple Pay said $120, to Nike, and nothing counted until Alex double-clicked.
With an agent, Alex says one sentence, and the agent goes and clicks for Alex later. Alex can't see which page it's on or which button it presses.
② During: know the worst case. Password-free payments came with a limit Alex set, and a small one. Spending stops at the limit, so Alex knows ahead of time the most they can lose.
Muse's "Always allow" never said what it covered and set no limit, so the bar ran right off the edge.
③ After: get it back if it's wrong. With escrow, the platform holds the money and only pays the seller once Alex confirms delivery. If something goes wrong, the money hasn't gone anywhere.
With an agent, the money is gone the moment the action is done. A "hold it for a few minutes so it can be undone" window would have saved the $258. Nobody has built that yet.
All three used to come for free because Alex was there. When an agent pays for Alex, each one has to be redesigned: the first already has a confirmation page; the second and third don't exist yet.
Of the three, the second is the hardest to fix.
Sure, but nobody hands over the whole limit on day one. First look at how Alipay climbed, one rung at a time, then at where agents are standing today:
In 2003, when Taobao had just launched, strangers didn't dare trade with each other online. Alipay started with escrow: the platform held the money, and the seller got paid only after the buyer confirmed delivery.8
Then came small password-free payments: users set their own limit and raised it once things went smoothly. On this rung, the user is trusting the platform.
In 2015 came Huabei, Alipay's credit line, and the direction flipped: based on a user's spending and repayment record, the platform hands them a small credit line first, and the better the record, the higher the line.9 Climbing from escrow to here took Alipay twelve years.
Now look at agents. Today they're stuck on the bottom rung: ask Alex about every payment. Nobody has built the red dashed rungs above it yet.
Muse's "Always allow" is the cautionary tale. YouTuber Matt Robb had Muse help sell a keyboard on Facebook Marketplace and picked "Always allow," thinking it meant "answer messages for me." Muse sent the pickup address to everyone who asked about the price, and agreed to a very low one.10 "Always" never said what it covered, so it jumped straight to the top of the ladder.
Agents should climb the way Alipay did. Alex starts with a tiny scope, like "under $20 at this store, don't ask me." Every time the agent gets it right, it builds a record, and the scope moves up one rung.
What door 3 is missing is a ladder you can climb one rung at a time. Right now, nobody has built it.
With all three doors behind us, back to the question we started with: when will Alex let an agent pay? My answer is a scale.
On the left: the time and money the agent saves Alex. On the right: what Alex could lose when things go wrong. The agent makes a mistake, Alex is on the hook for it, the loss has no cap, and a wrong purchase can't be refunded. Right now the right side is heavier, so Alex won't let the agent pay.
Get door 1 right (the agent sees consequences and doesn't act until they're clear), and "Mistakes" comes off the right side.
Get door 2 right (fraud checks recognize agents, and the rules stop pushing every loss onto Alex), and "Your risk" comes off. The scale levels out. Alex is still on the fence.
Get door 3 right (limits grow one rung at a time, so losses have a cap, and a wrong purchase can be refunded), and the right side is empty. The scale tips left. The day it tips left is the day Alex lets the agent pay.
As agents get smarter and easier to use, the left side gets one more weight. One investor praised Instinct because other AIs "will often stop for confirmation, which kills the vibe," and said he'd rather use one that doesn't ask much.11 Convenience has weight of its own: some people will take on a bit more risk for it.
How heavy the left side is, that investor already tested for us. How heavy the right side is, the $258 already tested too.
The payment step already works. What comes next is seeing who tips the scale to the left first.
About this piece: it covers the same ground as the product post, Letting Agents Pay for You: What's Still Missing? (Product Side), told instead as a story with a few moving drawings. Alex is made up; everything that happens to Alex really happened, with sources in the notes below.
Notes