How Do You Get to the Point Where You Trust an AI Agent Nobody Is Watching?

Suppose you actually have one now — an agent that takes a job end to end. Pulls the data, writes the code, runs the tests, opens the PR, updates the docs. It doesn’t need you feeding it prompts line by line. You hand it the task at night and go to sleep.

The real question isn’t whether it finishes. After a couple of years spent trying to read everything I can on frontline agent practice — production write-ups, the eval and benchmark ecosystem, primary sources, papers — I’m increasingly sure of one thing: capability stopped being the bottleneck a while ago. The thing that actually has everyone stuck is a single question —

The next morning, do you dare use what it produced?

If yes, you’ve freed up an entire person. If no, it’s just a fancier autocomplete: you review every line, fix every line, and hand back all the time you thought you’d saved. So “handing work to an AI agent nobody is watching” was never a capability problem. It’s a trust problem. And trust happens to be the least mature layer in the entire agent stack.

Between the demo and production sits an eval gap

I’ve watched a lot of stunning agent demos. In the screen recording it glides through, gets it right on the first try, and the room claps. Then the thing hits production and starts leaking.

The reason isn’t mysterious. A demo is one carefully selected success. Production is ten thousand consecutive runs that nobody gets to select from. The first only needs to be right once. The second needs to be able to fail affordably, get caught, and roll back. The thing sitting between them is what I call the eval gap.

Plenty of teams already sense the gap is there, so they pile on observability: traces, token accounting, a log line for every tool call, alerts on failed retries. All of that is correct — but be clear about what it actually solves.

Observability tells you what happened. Evals tell you whether it was right.

Those are different things. Observability can tell you the agent made 47 tool calls last night, spent forty-five cents, and retried twice at step 31. It cannot tell you whether the report those 47 calls produced reaches the wrong conclusion. However complete your logs, they only record the accident in high fidelity. They don’t prevent it.

From what I’ve tracked, the mismatch shows up plainly in the numbers: among teams running production-grade agents, roughly nine in ten have observability, but only around half have real evals. In other words, most people have installed a dashcam and no brakes. We’re a little too fond of watching what it’s doing, and we avoid the harder, more expensive, more important question — whether what it did is right.

This is what I kept hammering on in The Super Individual’s Intelligence System : observability is the ticket in, evals are the moat.

The trust stack for unattended agents

So how do you actually get to “comfortable”? My answer is unglamorous and engineering-shaped: trust isn’t asked for, it’s built. Three parts.

Part one: guardrails that authorize before the action fires.

Most people’s mental model of a guardrail is still “filter the output” — wait for the agent to finish talking, then run a classifier over it to check for anything nasty. That’s far too late. For an agent that acts, the danger was never in what it said. It’s in what it did.

Guardrails have to move forward to the tool execution layer. You’re not intercepting its speech, you’re intercepting its hands. Before delete, transfer, deploy, send actually fire, run an authorization check: is this action on the allowlist? How large is the blast radius? Is it reversible? Does the amount or the scope exceed a threshold? The harness decides whether to let it through, before execution — you do not rely on the model having thought it through first. I’ll come back to this, because it’s the foundation the whole system stands on.

As for how to tier that authorization, there’s a four-level table further down that you can copy straight into a config file. It’s the most immediately usable thing in this piece.

Part two: regression evals.

Guardrails prevent bad actions. Evals answer whether the work was right. The key word is regression — you need a fixed set you can re-run, and every time you change a prompt, swap a model, or add a tool, you run it again and see whether the numbers went up or down.

There’s a counterintuitive and very important point here: passing a benchmark doesn’t mean it’s usable. However high you climb on a public leaderboard, all that proves is it does well on problems someone else picked. Your business, your data, your edge cases — you have to write those problems and grade them yourself. I’ve even noticed a new ranking style appearing in the industry, one that weights human preference together with factuality rather than scoring a single dimension of right and wrong. Which tells me people are finally catching on: “looks right” and “is right” are two things you have to test separately.

Part three: the morning HITL review.

HITL — human in the loop. This isn’t regressing to manual work. It’s moving your attention precisely, from “watching the whole run” to “pressing a button at the points that matter.”

Overnight the agent runs on its own, guardrails hold back the irreversible actions, and evals stamp a confidence score on every output. In the morning you sit down with coffee and review exactly two categories: whatever the evals flagged red, and whatever the action is irreversible. Anything rollbackable — editing a draft, running tests, generating a report — let it rip. Anything where one mistake sinks the whole thing — moving money, dropping a database, publishing externally, deploying to production — you press that button yourself.

The order of the three matters: guardrails first (don’t cause harm), evals in the middle (judge right from wrong), HITL last (carry final responsibility). Drop any one and “unattended” is just a word.

Compounding error: the math you don’t get around

Why am I this cautious about “unattended”? Because there’s a piece of math, cold and indifferent, that nobody gets around.

Agents work in steps. Plan, call a tool, read the result, plan again. A serious task runs fifteen or twenty steps easily. Every step has a probability of error, and errors multiply along the chain. They don’t add.

Let’s do the arithmetic. Say a given step is 95% accurate — sounds pretty reliable, right? Run twenty of them:

0.95^20 ≈ 0.36

36% left. An agent that’s 95% right at every step, run across a 20-step task, gets the whole thing right about a third of the time. You thought you hired a straight-A student; on a long chain what you actually have is a kid hovering around the pass mark. This is compounding error — the mathematical core of agent reliability, and the root cause of “stunning demo, miserable production.” I ran the same numbers in The Super Individual’s Intelligence System , and I’m repeating them here because everyone should have them memorized.

One number is easy to dismiss as a special case. Spread it into a table and it starts to scare you properly. Steps across the top, per-step accuracy down the side, probability the whole chain comes out right in the cells:

Per-step accuracy →   90%     95%     98%     99%    99.5%
Steps ↓
   5 steps           59%     77%     90%     95%     98%
  10 steps           35%     60%     82%     90%     95%
  20 steps           12%     36%     67%     82%     90%
  50 steps            0.5%    8%     36%     61%     78%
 100 steps            0.003%  0.6%   13%     37%     61%

Three things in this table are worth staring at for a while.

First, read across: the right-hand columns are absurdly expensive. Going from 95% to 99% moves the 20-step row from 36% to 82%. But “improve per-step accuracy by 4 points” is a weightless sentence whose actual meaning is: cut the error rate by four fifths. That’s a new model generation, a full prompt rewrite, and a rebuild of your tool interfaces. And it gets harder the further right you go — 99% to 99.5% means halving the error rate again. This is why that one percent is both so expensive and so valuable.

Second, read down: long chains are a meat grinder. Take that same “sounds pretty good” 95%. At 5 steps you still have 77%. At 50 steps you have 8%. Chain length hurts reliability an order of magnitude more than your intuition says. A lot of people’s first instinct when building an agent is “let it think a few more steps, give it a few more tools,” without noticing that every added step shoves another factor smaller than one into the product.

Third — and this is the useful one — read along the diagonal. 90% accuracy over 5 steps (59%) and 98% accuracy over 20 steps (67%) are the same order of reliability. Meaning: shortening the chain and improving the model are two mathematically interchangeable moves. And one of them you can do this afternoon; the other one you wait for a model generation.

So what do you do? Two roads.

One is grinding per-step accuracy upward. That road is real and it works, but its price tag is written on the right side of that table — and it isn’t in your hands. It’s in the model vendors'.

The other is more engineering-shaped and more realistic: shorten the chain, add checkpoints, allow rollback. This one deserves working through, because a lot of people think “add checkpoints” is a psychological comfort. It isn’t. It literally rewrites the multiplication.

Why do checkpoints change the math? Because 0.95^20 holds only under an implicit premise: once an error happens, it rides all the way to the end. Step 3 goes off, and the next 17 steps run forward on top of the deviation, so whatever comes out is guaranteed wrong. What a checkpoint does is knock that premise out: put a check at the end of step 5, and an error gets caught right there, rolled back, and that short stretch re-runs — instead of continuing downstream and contaminating everything.

So the 20-step chain stops being one 20-way product and becomes four short products of five steps each:

No checkpoints:  [1 → 2 → ... → 20]                overall 0.95^20 ≈ 36%
                 one long product; any step wrong = everything wrong

Checkpoints:     [1..5] ✓ [6..10] ✓ [11..15] ✓ [16..20] ✓
                   ↑check   ↑check    ↑check     ↑check
                 each segment 0.95^5 ≈ 77%; on failure only that segment re-runs
                 allow one retry per segment → each ≈ 1-(1-0.77)² ≈ 95%
                 four segments overall ≈ 0.95^4 ≈ 81%

36% → 81%, and not from a better model — from better scaffolding. Notice that 81% is nearly identical to the 82% you’d get by pushing per-step accuracy to 99%. You just got a full model generation’s worth of improvement for free, through engineering. That’s the one piece of arithmetic in this article I most want you to keep.

Two honest premises here, and leaving them out would be dishonest. One: the check itself has to be reliable, and cheaper than the generation. Which is why checkpoints belong where correctness has an objective standard — did the tests pass, is the schema valid, do the numbers reconcile, does the file exist. Having one model judge whether another model’s work is “good” just relocates the uncertainty, and relocates it somewhere more expensive. Two: the retry has to be a genuinely independent attempt. If the failure is that the task exceeds the model’s ability (it simply doesn’t know how to use that API), then ten retries are ten identical failures and that (1-p)² term evaporates.

So the real order of operations is: find the places where checks are cheap and verdicts are objective, and nail your checkpoints there, so the chain breaks where breaking is useful. Not a cut every five steps, evenly spaced.

One last honest caveat: getting per-step accuracy to 99% is enormously hard, and plenty of real tasks never touch a stable 95%. So for the vast majority of teams, the second road is the one you can actually build today — you can’t stop the model making mistakes, but you can make its mistakes hit a gate and stop before compounding amplifies them. This is also the other face of that “hidden cost behind cheap” arithmetic from the third article : cheap tokens make you brave enough to run longer chains, and the longer the chain, the further down and to the left this table drags you. Cheap buys you attempts, not reliability. Reliability is something you nail into the chain yourself, one checkpoint at a time.

“Runs at night, reviewed in the morning”: the whole pipeline in one diagram

Put all of that together and an unattended agent you can genuinely relax about looks like this:

        NIGHT (unattended)                     MORNING (HITL)
  ┌─────────────────────────────┐    ┌───────────────────────────┐
  │                             │    │                           │
  │  task → plan → [tool call]  │    │   human arrives           │
  │           │                 │    │      │                    │
  │           ▼                 │    │      ▼                    │
  │   ┌──────────────┐          │    │  review only two things:  │
  │   │ guardrail    │──over────┼────┼─▶ ① eval-flagged red      │
  │   │ (pre-exec)   │ threshold│    │   ② irreversible action   │
  │   │ reversible?  │  → queue │    │      │                    │
  │   │ blast radius?│          │    │   ┌──┴───────┐            │
  │   └──────┬───────┘          │    │   │ approve  │──▶ execute │
  │     pass │                  │    │   │ reject   │──▶ rollback│
  │          ▼                  │    │   └──────────┘            │
  │   execute → eval → persist  │    │                           │
  │          │(rollback point)  │    │   Any action where one    │
  │          ▼                  │    │   mistake sinks it: a     │
  │   next step / close the loop│    │   human presses it.       │
  │                             │    │                           │
  └─────────────────────────────┘    └───────────────────────────┘

  reversible actions   → let the agent run them
  irreversible actions → queue at the HITL gate, wait for a human

Read that diagram and you have my whole argument: an agent’s autonomy should be allocated strictly by the reversibility of the action. Editing drafts, running tests, generating first passes — let it loose. Moving money, dropping databases, publishing externally — everything queues. Autonomy isn’t better when it’s higher. It’s better when it’s high where it should be high and locked where it should be locked.

Guardrail tiers: a four-level table you can copy

“Allocate autonomy by reversibility” sounds elegant. What you actually need at the tool execution layer is a table you can write into a config file. This is the split I use — four tiers, and one question decides the tier: if this action is wrong, what does it cost to undo?

Four traps I’ve fallen into using this table, passed on directly:

One: you tier actions, not tools. The same db.execute tool is L0 running a SELECT, L2 running an UPDATE, L3 running a DROP. Authorize at tool granularity and you’ve granted the tool’s maximum danger level. Authorization granularity has to reach the parameter level.

Two: L2 thresholds must be set on cumulative volume, not just per action. A $500 cap per transfer sounds safe — until the agent makes 200 transfers in an hour. Every cap needs both a per-action limit and a windowed cumulative limit, otherwise it’s a fake guardrail you can walk around with a loop.

Three: when unsure, tier up. The cost of a tiering mistake is wildly asymmetric: mistake an L3 for an L2 and you might lose a production database; mistake an L2 for an L3 and you press one extra button in the morning. That asymmetry is far too large to justify optimizing for fewer button presses.

Four, and the easiest to miss: watch for composition. Each step reads L1 on its own; strung together they’re L3. The agent writes some internal data to a temp file (L1), uploads the temp file to object storage (L1), then sets that bucket to public-read (looks like a config operation) — three unalarming steps that compose into an irreversible data leak. So on top of action-level guardrails you need a small number of trajectory-level rules: something like “any chain that has touched data marked sensitive has all downstream outbound actions promoted to L3.” Rules like that are annoying to write, and they catch exactly the accidents that per-action review can never see.

The value of this table isn’t that it’s clever — it isn’t clever at all. It’s that it converts a vague psychological question, “how much do I actually trust this agent,” into a config file that can be reviewed, diffed, and argued about in code review. Once trust can be version-controlled, it stops being a feeling and becomes engineering.

Eval sets aren’t designed. They grow out of incidents.

There isn’t much resistance to adding guardrails — everybody’s afraid of accidents. What actually blocks most teams is part two: where does the eval set come from?

I’ve watched a lot of teams stall right here, and they stall for the same reason every time: they want to design a complete eval set. Book a meeting, open a doc, try to enumerate every scenario worth testing. The meeting ends, three pages get written, and then nothing — because you cannot finish this from the front. You will never think of them all.

Don’t design it. Grow it. Five steps:

Step one: run it, don’t test it. Let the agent actually run inside a low-risk scope for a week or two. What you’re short on right now isn’t evals, it’s samples — you don’t yet know how it fails. The overlap between the failure modes you imagine and its actual failure modes is surprisingly low.

Step two: every time something goes wrong, freeze it into a case immediately. This is the central move. Last night the agent parsed a date wrong and skewed the framing of an entire report — don’t just patch the prompt and move on. Save that run’s input, that step’s intermediate state, and the correct output, verbatim, as an eval case. Every incident is a free exam question handed to you by the universe, and it’s a question your users will actually ask.

Step three: want the wrong samples before the right ones. Counterintuitive but important: an eval set with only positive cases carries almost no information — the model was probably going to get those right anyway. All of an eval set’s value density lives in the negatives. So during cold start, prioritize sweeping in every failure, every human rejection, every output that made you frown during the morning review.

Step four: grade with objective checks first; don’t reach for a model judge on day one. If exact match works, use exact match. If schema validation works, use schema validation. If you can run tests, run tests. Only where no objective test exists (“is this summary any good?”) do you bring in a model as judge — and you need to remember that a model judge is itself a model that makes mistakes, and it needs its own evals. Don’t validate one thing with another thing you haven’t validated.

Step five: wire it into CI, run it on every line changed. An eval set that isn’t wired into the workflow will rot within three weeks. Change a prompt, swap a model, add a tool — run it every time, look at the numbers. An eval set with no regression protection is just a pile of JSON sitting in a repo.

The shape of the whole thing is roughly:

   a failure in production
        │
        ▼
   ┌────────────────────┐
   │ capture: input +   │  ← the incident's value is captured here,
   │ intermediate state │    not after you've patched the prompt
   │ + correct output   │
   └────────┬───────────┘
            ▼
   ┌────────────────────┐
   │ freeze as eval case│
   │ annotate: why wrong│
   └────────┬───────────┘
            ▼
   ┌────────────────────┐      ┌────────────────────┐
   │  regression suite  │◀─────│ new attack samples │
   │  (only grows)      │      │ from continuous    │
   └────────┬───────────┘      │ red teaming        │
            ▼                  └────────────────────┘
   every prompt / model / tool change
            │
            ▼
   run it → numbers up? down?
            │
            ▼
   down means don't ship ← this line is the whole meaning of "regression"

Note the inlet on the right: whatever continuous red teaming produces should feed straight back into the same eval set. These aren’t two systems, they’re one. Red team’s job is to find new ways to be wrong; the eval set’s job is to guarantee that particular wrongness never comes back. That interlock is the technical foundation of the forecast I’ll get to below.

One thing from my own experience: the value of this eval set shows up suddenly, around month three. For the first two months it feels like overhead — every incident now costs you an extra twenty minutes of capture work. Then in month three you swap in a new model, run the suite, and it tells you flatly that the new model regressed on the seven cases you care about most — and in that moment you’ll be so grateful you could cry. It’s the kind of investment with brutally delayed returns that you can never go back from once it pays. And precisely because the returns are delayed, it’s the thing most teams keep putting off — which is exactly why it gets so valuable in the second half.

The morning checklist, and a few mistakes to skip

Guardrails and evals are written for machines. This section is written for you — what, specifically, you look at when you sit down tomorrow morning.

The morning review checklist

Work it in this order, from “most likely to be catastrophic” to “most likely to be ignored”:

• The gate queue first (the L3 items waiting on you). This is the only place that genuinely needs your judgment, so give it your best attention while you still have it. Ask each one: do I recognize this action? Is the blast radius what I think it is? If it’s wrong, can I absorb it?
• Then the eval reds. Not that it’s red — why it’s red. Did the model regress, or is the case itself out of date? The latter is more common than you’d think, and it’s the main source of eval rot.
• Then the chains that stopped halfway. Tasks that died mid-run are often more informative than the ones that finished. Where it stopped is either your guardrail doing its job, or your guardrail with a badly set threshold. A guardrail that never fires and a guardrail that fires daily are both misconfigured.
• Spot-check one all-green chain. The most counterintuitive item, and the most important. All green doesn’t mean all correct. It means nothing you already knew to check got tripped. Pick one chain at random every day and walk it end to end by hand, and you’ll periodically find new failure modes — and every one of those is the next eval case. This is your only detector for unknown unknowns.
• Bill and duration last. Anomalies in those two numbers are often the first signal of behavioral drift: last night suddenly cost three times as much, which usually means something is spinning in a retry loop, not that it got industrious.

How long should this take? From what I’ve seen, a healthy system settles at fifteen or twenty minutes of morning review. If it takes two hours every day, your guardrails are too loose and too much is leaking through to you. If it takes two minutes and there’s never anything to look at, that probably isn’t reliability — it’s that your evals aren’t testing anything meaningful. Drift in either direction needs fixing.

Anti-patterns

The most common and most expensive ways to get this wrong, ordered by damage:

• Observability without evals. The most universal one, as above: dashcam installed, brakes not installed.
• Guardrails written into the prompt. You put “please do not delete production data” in the system prompt and feel safe. A prompt is a suggestion, not a constraint. Guardrails have to be hard-coded at the tool execution layer, where the model can’t touch them, change them, or route around them. Any guardrail that can be talked out of its position with a sentence was never a guardrail.
• Shipping on benchmark scores. A public leaderboard proves it does well on problems someone else picked. It has nothing to do with your business or your edge cases.
• A model judge that was never evaluated. You use one model to grade another and then trust the grade completely. You’ve relocated the problem, and relocated it somewhere less visible.
• HITL degraded into a rubber stamp. The most insidious and most dangerous one. A person who clicks “approve” 80 times a day stopped reviewing somewhere around click 30. They’re just clicking. The moment there’s too much for a human to review, human review quality goes to zero — and what you have isn’t HITL, it’s a performance of HITL. So tune the thresholds until the L3 queue is single digits per day. HITL’s value is inversely proportional to its volume.
• Patching the prompt after an incident and keeping no case. Every time you do this you throw away a free lesson and guarantee the same mistake happens again.

The first two decide whether you’ll have an accident. The last two decide whether you’ll have the same one twice.

Why guardrails are the least mature layer in the stack

I keep coming back to guardrails, so let me say the unflattering part plainly: guardrails are the least mature layer in today’s agent stack.

Models have a pile of comparable SOTA numbers. Orchestration frameworks have the LangGraphs of the world fighting it out. Observability has a set of mature tools. Vector stores, RAG, tool-calling protocols are all converging fast. Guardrails alone have no dominant framework, no mature paradigm, no best practice everyone defaults to copying. Every team is out there with their own if-else, their own regexes, their own thresholds pulled from the air, rebuilding the same leaky wheel.

Which is the irony: the layer unattended agents depend on most is the layer that’s least mature. In 98.4% Is Scaffolding, 1.6% Is Judgment I laid out the division of labor in harness engineering — in an agent system, 98.4% is scaffolding and 1.6% is judgment. Guardrails, evals, HITL gates all live in that 98.4%. The model contributes the critical 1.6% of judgment, but whether you dare let that 1.6% out of the building depends entirely on how solidly the 98.4% is built.

In the end: “information is worthless, what’s valuable is the ability to process it” — and in the agent era I’d add a line. The ability to process information is getting cheap too. What’s actually valuable is the trust to use the processed result directly, and that trust is held up by scaffolding.

Second-half forecast: the bottleneck moves from building agents to trusting them

This is a forecasting column, so let me commit. Two clear predictions about the second half of 2026 and the agent trust layer.

Prediction one: evals and observability become the next M&A battleground.

In the first half, the money and the attention went to building agents — stronger models, smoother frameworks, flashier demos. In the second half the bottleneck will visibly migrate from building agents to trusting agents. When every company holds a pile of working agents and nobody dares actually let one run unattended, the market will pay enormous sums for anything that makes them trustworthy.

Eval platforms, observability tools, guardrail middleware — three categories previously filed under “ops miscellany” — become contested ground overnight. The big players will buy their way into this layer, because building a trustworthy eval and guardrail system from scratch is far slower than acquiring a team that already has one. Whoever controls the ability to make agents trustworthy controls the actual gate on agent commercialization. Remember this call: in the second half, money flows from “smarter” to “more trustworthy.”

This is the same coin as the blue-ocean argument in the fifth article . That one says the red ocean is full and the blue ocean is vertical, regulated, and end-to-end — but on what basis does an agent in a regulated field dare take end-to-end responsibility for a customer? On exactly the machinery in this piece: guardrails prove it won’t go off the rails, evals prove it does the work right, HITL proves a human is accountable at the decisive moments. The trust layer isn’t just a business, it’s the entry requirement for every blue-ocean business. Without that foundation, “we take end-to-end responsibility” is a sentence your legal team will never let you say.

Prediction two: continuous red teaming becomes table stakes.

One-shot pre-launch security testing gets retired. The reason is simple: an agent is alive. Swap a model version, add a tool, edit a prompt, and its behavioral boundary quietly drifts. Safe last month doesn’t mean safe this month.

So red teaming can’t be a one-off. It has to become standing, automated, continuously running adversarial testing: a red team (quite possibly made of agents itself) attacking your production agent around the clock, inducing privilege escalation, probing guardrail boundaries, and feeding every new hole straight back into the eval set. Continuous red teaming is to agent safety what CI/CD is to software quality — from “test once before release” to “always testing.” By the end of this year I expect it to go from a luxury at frontier companies to standard equipment for anyone deploying agents seriously.

A caveat I have to state

Forecasts are forecasts. Let me put this plainly so you don’t read the piece as blind optimism.

Safety lives in the harness, not in the model’s good sense.

I want to say that with maximum weight. You can never, ever hand safety to “the model will probably think it through.” The model has no good sense. It has a probability distribution. That it thought it through once doesn’t mean it won’t, on some edge you never tested, drop the production database with complete composure. What actually stops it was never its conscience. It’s the guardrail you hard-coded at the tool execution layer that nothing can route around.

So there’s exactly one bottom line, bold, and carved into the SOP: irreversible actions require HITL. Moving money, deleting data, publishing externally, deploying to production, sending an unrecallable message — no matter how “mature” the agent or how high the eval scores, a human finger lands on the final confirm. This isn’t distrust of the agent. It’s basic respect for two objective facts: compounding error and irreversibility.

This bottom line and the “proactive ≠ autonomous decision” boundary from the second article are the same nail driven in two places. That one argues that “who initiates” and “who decides” are independent dimensions — an agent can be extremely proactive about initiating and must be extremely restrained about deciding. The guardrail tiers here are what that restraint looks like in practice: L0 through L2 is the range where it can be proactive; L3 is the gate where it must hold back. Proactive agents aren’t an exception to this system, they’re a layer built on top of it — they ask you to trust not just the agent’s execution but the agent’s judgment (its judgment that this thing is worth interrupting you for right now). The bar goes up, never down. Build the unattended trust foundation first, then talk about proactive. Reverse the order and what you get is an agent that proactively causes accidents.

What we’re really learning is how to let go

Writing this, I notice it looks like a technical piece but is really about something harder — learning to let go.

Handing work to an agent nobody is watching is, fundamentally, no different from handing work to a new hire. You don’t let them touch the production database on day one because their résumé was impressive. You scope their permissions first (guardrails), make everything they do traceable and reviewable (evals), and keep the decisive calls routed through you (HITL). Then, as they prove themselves over and over, you loosen your grip a little at a time.

Trust is earned through engineering, bit by bit. It isn’t granted all at once by capability.

The agent capability curve over the past two years has been frighteningly steep, but I’m less and less anxious about whether it’ll replace me, and clearer and clearer about one thing: in an era where anyone can summon a powerful agent, the scarce thing is no longer an agent that can do the work — it’s someone who can build a harness that makes an agent worth trusting. The former keeps getting cheaper. The latter keeps getting more valuable.

So, back to the question at the top — the next morning, do you dare use its output directly?

My answer: when you can answer, in your own words, what the guardrails held back, what the evals checked, and which steps still need your finger — then you dare. And nobody can build that for you. You grow it yourself, one layer at a time. Which is probably the most worthwhile job the agent era leaves each of us.