An Introduction to Agent Engineering: What 400,000 Sessions Showed About Designing How You Delegate

Hello, this is Ryuta Hamamoto from TIMEWELL.

AI can write code. So why has the market value of some engineers gone up, not down? A study has answered that with data.

On 16 June 2026 Anthropic published "Agentic coding and persistent returns to expertise". It analysed about 400,000 sessions from about 235,000 people, covering seven months from October 2025 to April 2026, in a privacy-preserving form¹. The important point is that this is real usage data, not a benchmark.

Reading it, a few premises I had been carrying around fell apart. In order, then.

The division of labour shown by 400,000 sessions

The clearest finding is this one. Their own wording:

people make about 70% of the planning decisions but only 20% of the execution decisions¹

People make about 70% of the planning decisions, and only about 20% of the execution decisions. Put the other way: about 80% of how the work gets done is decided by the AI.

What I like about these numbers is that they break the coarse story that "AI writes code in place of people." The roles were not swapped. They split by layer. People stayed on as the architect; AI took the build. That structure appeared on its own, without anyone issuing an order.

The inside of the sessions is also in the paper¹.

Use	Share
Writing, editing and testing code	about 56% (writing 25%, editing 26%)
Operating software	17%
Planning and exploration	14%
Analysis and documentation	13%

Nearly half of the work is not writing code. I did not expect that.

The change over time is more interesting still. Across the seven months, sessions used for debugging almost halved, from 33% to 19%. Meanwhile estimated value per session rose 27%. By type: building up 43%, operating up 34%, editing up 32%¹.

You can read that as a move from a tool you use to fix what broke, to a tool you use to make things.

Expertise does not vanish. It is amplified

This, I think, is the largest finding.

You often hear that once AI spreads, specialist knowledge will not be needed. This data pointed the other way.

Verified success — success with evidence you can check, a passing test, a commit — looked like this¹.

• Sessions rated novice: 15%
• Intermediate and above: 28% to 33%

Sessions rated expert reach verified success more than twice as often as those rated novice¹

Expert sessions reach verified success more than twice as often as novice ones.

Output makes the same point more plainly. About 600 words per session for novices, about 3,200 for experts. Roughly five times the difference¹. Same AI. Spoken to in what looks like the same way.

When something goes wrong the gap opens further. Novices 4%, experts 15%¹. Once the work has gone sideways, expertise is doing visible work.

Why? I think people with more expertise can cut a task cleanly and define what success actually is. AI faithfully amplifies the quality of the instruction. A vague brief becomes a vague result. A precise brief becomes a large one.

And this matters: that expertise does not have to be software-engineering experience. Verified success by occupation was 30% in software-related jobs and 26% in the others¹.

Every one of the ten largest occupations in our dataset lands within seven points of software engineers¹

Every one of the ten largest occupations in the dataset sits within seven points of software engineers. What is doing the work is not the ability to write code. It is knowing your own domain in depth.

If you want to take a reading of where your own organisation sits with AI, the AI literacy check will give you a current position.

What to measure is "how long you can leave it alone"

Model comparisons tend to collapse into benchmark scores. What you actually want to know at work is something else.

The AI-safety research group METR put the question this way. "A task that would take a human expert how many hours, completed at what success rate?" They call this the task horizon. A "50% task horizon of one hour" means the system can finish a one-hour human task with 50% probability².

METR measured this on about 230 tasks. The results²:

Model	50% task horizon
GPT-2	2 seconds
Claude 3.7 Sonnet	50 minutes
o3	about 2 hours
Opus 4.6	about 12 hours

From four seconds in 2019 to more than 16 hours in 2026. Over the long run it has been doubling about every seven months².

There is a newer wrinkle than the material I first used. The recent stretch has sped up. From 2024 into 2025 it was doubling about every four months, and if that pace holds it is projected to reach month-scale tasks by 2027².

What I like about the measure is that it changes the question. Not "can this AI write code," but "how large a piece can you hand over without watching the middle."

And the skill of designing how you hand work over compounds. Today you may only be able to leave three hours. If the doubling holds, next year that is nine hours, and after that it is measured in days. Your design skill can stay where it is; the volume of work you can park with the agent still grows. That is why I think this is the skill to pick up now.

Long tasks fail because of the environment, not the model

So should you just hand over larger pieces? It is not that simple.

Anthropic ran an experiment in which models were asked to build production-quality web apps across multiple context windows, and published the failures³.

One term. A context window is the upper limit on how much information an AI can hold at once. When the conversation and the code go past it, the oldest material drops out. On a long job you always hit that wall, so the work splits across several sessions.

The result: even frontier models, the system came apart. The cause was not the model's intelligence. It was the design of the environment. Two failure shapes.

Trying to do the whole thing at once. The agent attempts to finish the entire task in a single session and runs out of steam half-done.

Memory cuts. The agent in the new session knows nothing about what happened last time. It starts from zero, every time.

It is a shift-working team with no handover at all, and a new graduate clocking in on a blank sheet every shift.

The fix was ordinary enough to be slightly embarrassing. Make it do what human teams have been doing for a long time³.

An initialising agent understands the whole task and breaks it into features. A coding agent implements one feature at a time, runs the tests, commits, and updates a progress file. The next session reads the progress file first and continues from there.

In short: put the memory outside the agent's head. Git history, a progress note, test results. Those survive a change of context. The agent does not need to remember everything. It only needs to be able to restore state from what was written down.

Three principles that separate systems that work

The thinking at the core of the harness built in that experiment — the scaffolding and control environment around the agent — works regardless of model or tool.

Principle 1. Treat the default as a fail.

Every success criterion starts as unmet. The agent saying "I am done" is not completion. You need evidence that the criterion was met.

Without that, the agent marks its own homework, and always gives itself full marks. "The tests passed (I did not run them)" happens as a matter of course. The idea is to shut overconfidence down with structure, not with a pep talk.

Principle 2. Split the person who evaluates.

The pass-or-fail judgement is made by an evaluator in a fresh context, separate from the agent that did the work. This evaluator cannot make fixes. All it can do is return a pass or fail, and a reason.

Same logic as a human team not letting the author of the code review it. The person who wrote it is too close to the artefact to see the defects.

Anthropic's piece is blunt on this point. A harness with no evaluator fails quietly when a frontier model is producing mediocre output³. No error is thrown, so you do not notice. That is the part that should worry you.

Principle 3. Make it write its own handover notes.

Have it update the progress file on every commit. What it did, what is left, what the next session needs to know. That single habit turns a system that resets every time into a system that accumulates.

How you control what an agent is allowed to read, and who the result is handed to, is also the subject of Why Cloudflare OS is open source. Start permissions at zero. Record what was read. The thinking points the same way.

That said, do not turn everything into an agent

The finding from the other side belongs here as well.

A study called "Agentless" showed that you can get strong results at low cost without an autonomous agent at all — just a three-step pipeline of locating the problem, making the fix, and verifying the patch⁴.

What to take from that is not "agents are always better." It is match the complexity to the task. Simple problems are won by simple machinery. A complex agent starts to pay for itself only on complex, long-running problems.

And telling which of those two you are standing in is itself most of the skill. You do not assemble a robot arm to drive one nail.

There is also research that backs the weight of the environment. Princeton's SWE-agent showed that a model is a new kind of computer user, and needs an interface designed for it. Give it tools shaped around how an agent actually moves, rather than a human editor as-is, and performance rose while the model itself was left unchanged⁵.

The harness is not an accessory to the model. It is half the system.

Where to start

It has run long, so here is the short version.

People decide 70% of the plan; AI decides 80% of the execution. Expertise does not lose its value; it shows up as a fivefold gap in output. The length of task you can leave alone keeps doubling, recently every four months. And what creates the gap is not the model. It is the environment you put around it.

In practice I think the order is this.

First. Hand over one small, complete task and watch. Not "write a function" — "fix this bug, write a test that proves it is fixed, and commit both." Pass a unit that has a beginning, a middle and an end. Then look at where the instruction and the result drifted.

Second. Write down any convention you have already explained twice. The stack, the places that must not be touched, how tests are written, the house style for commits. Write it once and it is read every time. The time spent repeating the same explanation disappears.

Third. Introduce a progress file and run the same task twice. With a handover and without. Compare what happens at the start of the second run. You will not believe the effect of external memory until you have felt it.

Fourth. Split off an evaluator. No write access. No knowledge of the working history. Its only job is to read the artefact and return a pass or fail, plus a reason. Calibration is simple: run it on a task you already know fails and a task you already know succeeds. If it drops the first and passes the second, you can trust it.

Holding a codebase as structure is in What graph engineering is. The concrete features of Claude Code are in the complete guide to Claude Code.

If I am honest, the part of this research that encouraged me most was the occupation finding. All of the top ten occupations within seven points of software engineers. What is doing the work is not whether you can write code. It is whether you know your own job in depth.

A veteran in accounts handing over the month-end close, a star salesperson handing over a proposal: the same three principles decide the outcome. Check against evidence. Look with a different pair of eyes. Leave it in a form the next person can pick up. I do not think agent engineering is an engineers-only subject.

If you want to talk through how to design AI use, or how to build a way of handing work over inside the company, WARP may be a useful reference. For a conversation, start here.

---

Footnotes

1. Anthropic, "Agentic coding and persistent returns to expertise" (Zoe Hitzig, Maxim Massenkoff, Eva Lyubich, Shaoyi Zhang, Ryan Heller, Peter McCrory, published 16 June 2026). An analysis, in a privacy-preserving form, of about 400,000 conversational sessions from about 235,000 people between October 2025 and April 2026. "people make about 70% of the planning decisions but only 20% of the execution decisions"; the session mix (writing, editing and testing code about 56% (writing 25%, editing 26%), operating software 17%, planning and exploration 14%, analysis and documentation 13%); the fall in the share of debugging sessions from 33% to 19%; the 27% rise in estimated value per session, 43% for building, 34% for operating and 32% for editing by type; verified success of 15% in novice-rated sessions and 28 to 33% in intermediate and above; "Sessions rated expert reach verified success more than twice as often as those rated novice"; success when things go wrong of 4% for novices and 15% for experts; output per session of about 600 words for novices and about 3,200 for experts; verified success by occupation of 30% in software-related jobs and 26% in the others; and "Every one of the ten largest occupations in our dataset lands within seven points of software engineers" — all from that study. https://www.anthropic.com/research/claude-code-expertise ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶ ↩⁷ ↩⁸ ↩⁹ ↩¹⁰

2. METR, "Measuring AI Ability to Complete Long Software Tasks", and related updates. Task horizon is defined as the time a human expert would need, for tasks an agent can solve at a given success rate. Measured on about 230 tasks (mostly coding, some general reasoning). The 50% task horizon grew from four seconds in 2019 to more than 16 hours in 2026, doubling about every seven months over the long run. From 2024 into 2025 the doubling was about every four months, and if that pace holds it is said to be able to reach month-scale tasks by 2027. By model: GPT-2 at 2 seconds, Claude 3.7 Sonnet at 50 minutes, o3 at about 2 hours, Opus 4.6 at about 12 hours. https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ ／ https://theaidigest.org/time-horizons ↩ ↩² ↩³ ↩⁴

3. Anthropic, "Effective harnesses for long-running agents" (engineering blog). That agents have to work in discrete sessions with no memory, and that a finite context window means a complex project cannot be finished in a single window; the split between an initialising agent that sets the environment on the first run and a coding agent that makes incremental progress in each session and leaves a clear artefact for the next; and the observation that a harness with no evaluator fails quietly when a frontier model is producing mediocre output — all from that article. https://anthropic.com/engineering/effective-harnesses-for-long-running-agents ↩ ↩² ↩³

4. Xia et al., "Agentless" (arXiv:2407.01489). A study showing that a three-stage pipeline of locating the problem, making the fix and verifying the patch can produce strong results at low cost without using an autonomous agent. https://arxiv.org/abs/2407.01489 ↩

5. Yang et al., "SWE-agent" (arXiv:2405.15793). From a Princeton research team. It showed that an AI model is a new kind of computer user and needs an interface designed for it, and that giving the agent tools optimised for how it moves raises performance without changing the model itself. https://arxiv.org/abs/2405.15793 ↩