What AI Agents Already Do on Their Own and What Stays With People: A Spring 2026 Snapshot

METR was given access to internal models at Anthropic, Google, Meta and OpenAI. Agents run projects worth hours and days of human work on their own, yet fall well short of experts in judgment and reliability. We look at where the line runs.
AI capabilities are usually judged by demos and advertising. In spring 2026 a rare source of a different kind appeared: an independent organization looked at how agents work inside the companies that build them.
Where the data comes from
METR, a research organization that evaluates frontier models, ran a pilot assessment from February 16 to March 16, 2026. Anthropic, Google, Meta and OpenAI gave access to their most capable internal models and answered a detailed questionnaire on how they use AI internally. The report was published on May 19, 2026.
The purpose of the report was to assess the risks of autonomous agents. For us the by-product matters: a sober picture of what agents can do and what they cannot do yet.
What agents do on their own
Capabilities have grown a lot. By METR's measurements, agents handle projects that would take a person hours and days, and on tasks of reimplementing software from a specification they solved work that would take a person weeks.
Practice has changed too. As recently as late 2025, METR itself kept agents under close supervision: small changes, each approved by a person. By spring 2026, engineers leave an agent working for tens of minutes, sometimes hours. Anthropic told the report's authors that the work of its researchers and engineers is increasingly shifting to reviewing changes and orchestrating agents.
Where agents are weak
One of the report's six key findings: in judgment and reliability, agents fall well short of human experts.
Agents are strongest where the result can be checked cheaply and quickly and where many attempts are allowed. In open-ended tasks the picture is different.
- •METR rated an analytical write-up by the best of the models below the 20th percentile among work tests submitted by candidates for such a role.
- •Andon Labs tasked a model with running a small store in San Francisco on a budget of 100,000 dollars. The model ordered a thousand toilet seat covers for the staff bathroom and listed them for sale, and scheduling errors kept the store closed for three days in a row.
- •Redwood Research gave a model 5,000 dollars and the task of making money. In four runs the result was zero.
The authors note a persistent gap: in real work, agents are weaker than their benchmark results suggest.
What agents do and do not do
The report sums up its observations in a table. The right-hand column is METR's assessment; there is less direct confirmation from the companies for it.
| Agents often do on their own | At no company do agents do this on their own |
|---|---|
| Write complex code changes | Approve changes to critical shared code |
| Analyze data and produce charts | Set research agendas |
| Find complex vulnerabilities | Make security decisions |
| Summarize large amounts of information | Make hiring and budget decisions |
| Optimize individual performance metrics | Make final judgments about risk |
How much it delivers in practice
METR's repeat experiment with experienced developers using late-2025 agents showed a speedup of 4-20%. The authors consider the estimate too low: some developers declined to take part because working without AI no longer pays off for them. In surveys, the same people report gains of 1.6x to 4x, and the authors recall that self-reports have proved inflated before.
Two numbers on quality. As of late 2025, about half of the AI solutions that passed automated tests would have been accepted in a real code review. In long tasks, eight hours and more, at least 16% of successful runs turned out to be illegitimate on manual review: the agent worked around the conditions of the task. Checking for such workarounds, the authors say, is often the majority of the work in an evaluation.
What this means for a small company
The report describes programming and research. The principle carries over to any work with documents and calculations.
- 1Give AI what is easy to check. Reconciliation, sampling, a draft based on a template, a calculation with a control total.
- 2Keep acceptance and high-stakes decisions with a person.
- 3Treat review as separate work and budget time for it. At the companies that build these models, it takes up a noticeable share of labor.
More on how roles in a team are changing in our overview "AI and Narrow Specialization".
Related service
Express Financial Diagnostics
An independent look at your finances in 5 days: 3-5 findings and a plan for what to fix first
