Syntic
All posts

Published October 4, 2026 in Agents

AI Is Moving From Answering to Doing

By Syntic AI

A language model on its own can only produce text. It cannot send an invoice, update a record or file a pull request. Everything a model "does" is still text: a structured request that some other piece of software reads and executes. What has changed in the past three years is not that models gained hands. It is that the systems around them learned to turn their output into actions, feed the results back, and keep going until the work is finished.

That shift, from producing a reply to running a loop, is what people mean when they say AI is moving from answering to doing. It is real, and the measured progress is steep. But it moves the hard problem somewhere new. When a system answers, a person judges the answer. When a system acts, someone has to know whether the action happened, whether it was correct, and whether it should have happened at all. The constraint on AI that does things is no longer mainly intelligence. It is verification.

Four changes that made action possible

No single breakthrough turned chat models into agents. Four separate changes, arriving between 2022 and 2025, stacked on top of each other.

A structured way to ask for an action

The first was a reliable interface between model and software. In June 2023, OpenAI added function calling to its API: developers describe available functions with a JSON schema, and the model can respond with a JSON object containing the arguments for a call instead of prose. The developer's code runs the function and returns the result to the model.

It sounds modest. It was the moment a model's output became something a program could parse without guesswork.

Reasoning interleaved with acting

The second was a pattern for using that interface over many steps. The ReAct paper, published in 2022 by researchers at Princeton and Google, showed that prompting a model to alternate between reasoning traces and actions, then reading the result of each action before deciding the next, beat imitation and reinforcement learning baselines by 34 percentage points on the ALFWorld household-task benchmark and by 10 points on the WebShop shopping benchmark, using only one or two examples.

Observations from the environment kept the model honest, reducing the hallucination and error propagation of pure step-by-step reasoning. Most agent frameworks today elaborate this loop: think, act, observe, repeat.

Models trained to use tools, not just told about them

Early agents relied on prompting a general model to behave like one. That changed when labs began training tool use directly. When OpenAI released its o3 and o4-mini models in April 2025, it said it had trained them to use tools through reinforcement learning, teaching them not only how to call tools but when to reach for them.

Shared connectors and access to the screen

The last two changes widened what a model can reach.

In November 2024, Anthropic open-sourced the Model Context Protocol (MCP), a standard for connecting AI applications to the systems where data and tools live, replacing one-off integrations with a single protocol. By December 2025, when Anthropic donated MCP to the newly formed Agentic AI Foundation under the Linux Foundation, the company reported more than 10,000 active public MCP servers and over 97 million monthly SDK downloads. A model that once needed custom engineering for every system can now plug into many of them through the same interface.

For software with no API at all, there is computer use. In October 2024, Anthropic released a capability that lets a model look at screenshots, move a cursor, click and type. The company was candid that it was "imperfect", with actions such as scrolling and dragging still difficult. But it meant that, in principle, any interface built for a person could also be operated by a model.

Together these form the architecture behind most "doing" systems today: a model that decides, a structured action format, tools reachable through APIs, protocols or the screen, and a loop that runs until a stopping condition is met.

The loop is the product

This changes what a builder is actually building. In a chat product, the model is the product. In an agent, the model is one component inside a control system.

Anthropic's engineering guidance on agents draws a useful line here. "Workflows" route a model and tools through predefined code paths. "Agents" let the model direct its own process and choose its own tools. Agents suit open-ended problems where the number of steps cannot be predicted in advance. The same guidance stresses that an agent needs "ground truth" from the environment at each step, such as a tool result or the output of executed code, to assess its progress, along with stopping conditions such as iteration limits.

That is why the most convincing agent results come from environments that return clear signals. Code runs or it does not. Tests pass or fail. Where the environment answers back unambiguously, the loop can correct itself. Where it doesn't, the loop can drift for a long time before anyone notices.

The trend also explains why so much product design is moving away from screens and toward goals: you state an outcome and the system works out the clicks. We explore that broader shift in software is moving from interfaces to intent.

How far it has come, and where it stops

The benchmark record shows both the speed of progress and its edges.

When SWE-bench was introduced in 2023, it asked models to resolve 2,294 real GitHub issues across 12 Python repositories. The best model at the time solved 1.96% of them. Coding agents have improved so much since then that the benchmark's human-validated subset is now a crowded leaderboard rather than a frontier, a story we follow in the future of coding is agentic.

Computer use tells a similar story. OSWorld, a benchmark of 369 tasks in real desktop and web applications, launched in 2024 with humans completing over 72% of tasks and the best model completing 12.24%. Anthropic's first computer-use release scored 14.9% in the screenshot-only setting. By mid-2026, according to the OSWorld 2.0 paper, a leading frontier agent reached 83.5% on the cleaned-up OSWorld-Verified version, above the original human baseline.

Then researchers made the tasks much longer. OSWorld 2.0 contains 108 long-horizon workflows that take a human a median of 1.6 hours and require an average of 318 tool calls, compared with about 30 in the original benchmark. On these, the best configuration completed only 20.6% of tasks outright, with a partial-credit score of 54.8%.

The drop from 83.5% to 20.6% is the most instructive pair of numbers in recent agent research. It says that operating software is largely solved at the level of individual steps. Finishing long pieces of work is not.

The new ways to fail

The OSWorld 2.0 authors did something more useful than report a score. They studied how agents failed. The answer was not, mainly, clumsy clicking. Agents lost track of instructions such as required file formats or naming rules, missed information that appeared midway through a task, guessed when they should have asked the user, and skipped verification. Recovery and repair together accounted for under 7% of agents' effort across the systems studied.

These are the failure modes of doing, and they are different from the failure modes of answering.

Submission is not verification

A wrong answer sits on a screen until someone reads it. A wrong action changes the world. The OSWorld 2.0 paper gives a simple example from an expense-reimbursement task: an agent reads the account-code rules, cross-checks bank charges and submits the report, yet the final state can still contain wrong fields, missing details or incomplete documents. The agent believed it was done. The system was not in the state it should have been.

An agent's own report that it has finished is a claim, not evidence. Anything important needs a check against the actual end state.

The same task, different results

Models are probabilistic, and their actions inherit that. The τ-bench benchmark from Sierra measures agents handling customer-service tasks with real tool access and policy rules, and checks the database state at the end of each conversation against the correct outcome. It introduced a metric called pass^k: the probability of succeeding on the same task in all of k attempts. In the original paper, a leading function-calling agent succeeded on under 50% of tasks, and its pass^8 in the retail domain fell below 25%.

For a chatbot, inconsistency is an annoyance. For a system processing thousands of refunds a day, the gap between "can do it" and "does it every time" is the whole business case.

Data that gives orders

Once a model reads emails, web pages and documents in order to act on them, any of that content can try to instruct it. Researchers described this indirect prompt injection in 2023, observing that applications built on language models "blur the line between data and instructions." For an agent, a hidden line in a web page or a file attachment can redirect what it does next.

The risk is measurable. In January 2025, the US AI Safety Institute at NIST (since renamed the Center for AI Standards and Innovation) reported agent-hijacking experiments using AgentDojo, an open-source framework from ETH Zurich. New attacks developed by its red team raised the success rate against one frontier model from 11% for the strongest baseline attack to 81%. Allowing an attacker 25 attempts instead of one raised the average success rate from 57% to 80%. The scenarios tested included exfiltrating a user's cloud files and sending phishing emails to everyone the user had a meeting with.

Too much agency

OWASP's 2025 list of top risks for applications built on language models includes "excessive agency": damage caused when a system has more functionality, broader permissions or more autonomy than its task requires. Its examples are mundane on purpose, such as an extension that only needs to read data but can also delete it. The recommended mitigations are equally mundane: least privilege, narrowly scoped tools, and human approval for high-impact actions.

Verification is the real bottleneck

Put these failure modes together and a pattern emerges. The capability to act has grown faster than the ability to confirm that actions were right.

Even researchers struggle with this. A July 2026 audit of 150 failure-scored runs across five computer-use benchmarks, including WebArena and OSWorld, found that 15.3% of the "fail" verdicts were themselves wrong: 10.7% were cases where the agent had succeeded but the automated checker rejected it, and 4.7% were broken tasks. The same authors note they did not audit "pass" verdicts. If purpose-built benchmarks misjudge agent work at that rate, a company relying on an agent's self-report should expect worse.

This is why the systems that work in production tend to share a few design choices.

They check state, not text. Success is defined as a condition in a system of record (the ticket is closed, the payment matches, the test suite passes) and confirmed by reading that system, not by trusting the model's summary.

They separate reversible from irreversible actions. Drafting, searching and reading run freely. Sending, paying, deleting and publishing wait for approval, or at least pass through a stricter check.

They log every step. When an agent takes 300 actions to finish a task, a record of what it saw and did is the only way to understand a failure, and the main way to improve.

They treat everything the agent reads as untrusted input. Permissions are scoped to the task, not to the user's full access, so that a hijacked agent can do limited harm.

Each of these practices also assigns responsibility: someone scopes the task, someone approves the risky step, someone reads the log. We argue in the era of AI employees that this is a management question as much as a technical one.

What "doing" changes next

When work, not words, is the output, value moves to the parts of the stack that make actions safe: well-defined tools, clean data, permission models and checks that confirm an outcome automatically.

Expect three consequences. Organizations will redesign processes to make outcomes verifiable, because the tasks that can be checked are the tasks that can be delegated. Evaluation will start to resemble real work, with long tasks, repeated trials and end-state checks. And agents will be wired into the systems that already run a business rather than bolted on beside them, which is the argument in AI agents are becoming the execution layer for business and in AI workflows are becoming the glue between business tools.

The move from answering to doing is not finished, and the research is honest about that. Models can now operate software at roughly human level on short tasks and still fail most long ones. Until that gap closes, the measure of an agent deployment is less how much freedom it grants than how quickly and reliably someone can tell whether the work was done.

Sources