Syntic
All posts

Published October 4, 2026 in Software

The Future of Coding Is Agentic

By Syntic AI

For most of the history of software engineering, the scarce resource was the ability to produce a correct change.

Coding agents change which part is scarce. Built on the act-and-observe loop we described in AI is moving from answering to doing, a repository-aware agent can now read a codebase, plan an edit across files, run the tests and open a pull request in minutes. Producing a plausible change is becoming cheap. Knowing whether to accept it is not.

That is the argument of this piece: the future of coding is agentic, but the value of an agent depends less on the model than on the codebase's ability to say no. Tests, types, CI pipelines, code review and permissions are what turn a fast change generator into a faster engineering team.

What the benchmarks measure, and what they miss

Public progress on coding agents has mostly been told through SWE-bench, which asks a system to resolve real GitHub issues and grades the result by running the repository's tests. Its human-screened subset, SWE-bench Verified, was introduced by OpenAI in August 2024 with 500 tasks and became the headline number for a generation of model launches.

In February 2026, OpenAI said it would stop reporting that number. Its audit of 138 problems that its o3 model did not consistently solve found that at least 59.4% had flawed tests that rejected functionally correct solutions. It also found that the frontier models it tested could reproduce verbatim gold patches for some tasks, a sign that the benchmark had leaked into training data. OpenAI now recommends SWE-bench Pro, which Scale AI built from 1,865 problems across 41 repositories, keeping a held-out set private and drawing a commercial set from startups' proprietary code, specifically to resist contamination.

A second problem is subtler. Passing the tests is not the same as being mergeable. METR, an independent evaluation group, had four active maintainers of scikit-learn, Sphinx and pytest review 296 AI-generated pull requests that had passed SWE-bench Verified's grader. Roughly half would not have been merged. Maintainer acceptance ran about 24 percentage points below the automated pass rate, for reasons from breaking other code to ignoring project conventions. METR stresses this is not a capability ceiling, since better prompting would fix much of it. Its advice: treat benchmark scores as one piece of evidence, not a forecast of usefulness.

A leaderboard says a model can satisfy someone else's tests. It does not say what share of its output your reviewers will accept. Only your own data can.

The productivity evidence points in two directions

Across randomized trials at Microsoft, Accenture and a Fortune 100 company covering 4,867 developers, access to an AI code assistant increased completed tasks by about 26%, with larger gains for less experienced developers. A randomized trial with 96 Google engineers on an enterprise-grade task estimated a speedup of about 21%.

Then METR ran a randomized trial in the setting where agents are supposed to matter most: 16 experienced developers working on 246 real issues in mature open-source repositories averaging more than a million lines of code. With AI tools allowed, mostly Cursor with the leading Claude models of early 2025, tasks took 19% longer. Before starting, the developers expected a 24% speedup. Afterwards, they still believed AI had made them about 20% faster.

That perception gap matters more than the sign of the result. Experts carry context an agent must be given or rediscover, and time spent prompting, waiting and reviewing is easy to discount.

The picture is moving. METR's follow-up, run in late 2025 with 57 developers and more than 800 tasks, estimated that tasks took 18% less time with AI for returning developers and 4% less for new recruits, with confidence intervals that include zero. METR itself calls this weak evidence, because the experiment started to break: many developers declined to take part because they did not want to do half their work without AI, some withheld tasks for the same reason, and some ran several agents at once, which made per-task timing unreliable.

Anthropic's December 2025 study of its own engineers shows the same direction from inside an AI lab. Self-reported use of Claude rose from 28% to 59% of daily work over a year, and self-reported productivity gains from 20% to 50%. Yet more than half said they could fully delegate only 0–20% of their work. The tasks they did hand off were the ones that were easy to verify, low-stakes or self-contained.

Delegation follows verifiability. That is the thread connecting every study above.

More change, not automatically better delivery

Google's DORA research program, surveying nearly 5,000 technology professionals for its 2025 report, found AI adoption associated with higher software delivery throughput but, as in earlier years, lower delivery stability. Its summary: AI does not fix a team, it amplifies what is already there. Without automated testing, mature version control and fast feedback, more change volume becomes more instability.

The AIDev dataset, covering more than 456,000 pull requests opened by five coding agents across 61,000 repositories, found that agents often outpace humans on speed but have their pull requests accepted less often. In one case, a developer submitted as many pull requests in three days as in the previous three years. Each still had to be read.

When producing a change gets cheaper, the bottleneck moves to the people and systems that accept changes.

What a well-run deployment looks like

The clearest counterexample is not a demo but a migration. Google's engineers described using an LLM-based toolkit to convert identifiers from 32-bit to 64-bit integers across a codebase where the manual effort was estimated at hundreds of engineering years.

An expert engineer identified the scope using code search and scripts. The agentic toolkit then ran autonomously and kept only changes that passed unit tests, updating tests where needed. The same engineer checked the output and fixed what the model got wrong. The changes were then split and sent to the reviewers who owned each part of the codebase. About 80% of the code modifications in landed changes were fully AI-authored, and engineers estimated the total time fell by about half.

Every element of that loop is a way of saying no: scope set by a human, tests as a gate, an expert check, and ownership-based review. The agent did most of the typing. The engineering system decided what landed.

The engineer's job moves up the stack

If agents write a growing share of the code, the work that remains is the work agents depend on.

Specification

An agent acts on what it is told plus what it can infer from the repository. Vague issues produce confident, wrong changes. Writing a precise task, with acceptance criteria and constraints, becomes core engineering work. Conventions and architectural decisions belong in versioned files that agents read and humans review.

Tests as instructions

Tests stop being only a safety net and become the most reliable way to tell an agent what "correct" means. A codebase with weak coverage gives an agent nothing to steer by, and gives reviewers nothing to lean on.

Review

Reading code becomes a larger share of the job than writing it. That makes small, well-scoped changes more valuable than ever, and makes oversized agent pull requests a cost to be managed, not a sign of productivity.

Architecture

Clear module boundaries limit how far one bad change can travel, and let an agent work on one part without touching the rest.

There is a tension here that Anthropic's engineers described directly. Supervising an agent well requires the deep familiarity that comes from debugging and building by hand, and heavy delegation can erode exactly that. Teams that want good reviewers in five years need to protect the work that produces them, especially for junior engineers.

The pipeline is the control surface

Agents act through the same channels as people: branches, pull requests, CI and deployment. Those channels are where policy belongs. Required checks, protected branches, ownership rules and mandatory review apply to an agent's change regardless of how persuasive its summary is.

The design rule is that an agent should never be able to approve its own work or loosen its own constraints. Agent credentials should be scoped to the repository and the task, not borrowed from a senior engineer's account. Destructive operations need a human gate. We explored the broader version of this argument, separating policy from the model, in AI agents are becoming the execution layer for business. In software, the CI pipeline already is that policy layer.

The agent is a new participant in your supply chain

Coding agents read untrusted text, such as issues, dependencies, documentation and web pages, and they hold real permissions. That combination has already been exploited.

In mid-2025 researchers disclosed CVE-2025-53773, a prompt-injection flaw in which instructions hidden in project content could make GitHub Copilot in Visual Studio Code edit its own settings file to auto-approve tool use, opening the door to running commands without confirmation. Microsoft patched it in August 2025. The lesson generalizes: an agent that can write to its own configuration can remove its own guardrails.

In July 2025, AWS disclosed that an inappropriately scoped token had let an attacker commit a prompt instructing the Amazon Q Developer extension for VS Code to wipe local and cloud resources. It shipped in version 1.84.0 and failed to execute only because of a syntax error. The following month, malicious versions of the Nx build system published to npm tried to use developers' locally installed AI command-line tools, invoked with flags that skip permission prompts, to search their machines for secrets. The entry point was a flawed CI workflow.

None of these was a model writing bad code. Each was a permission, pipeline or instruction channel nobody had threat-modeled for an agent. Security reviews now need to ask what an agent can read, what it can execute, and what it can change about its own controls.

What teams should measure

Lines of code, pull requests opened and suggestion acceptance will all rise with agent adoption; none says whether delivery improved. A more honest scorecard tracks what happens after a change is proposed:

  • Merge without major rework: the share of agent-originated changes accepted with minor edits, by repository and task type.
  • Review load: time to first review, review hours per merged change, and queue length.
  • Stability: change failure rate, reverts and time to restore, split by agent-assisted and human-only changes.
  • Escaped defects and security findings traced back to their originating change.
  • Coverage on touched code, because that is where the agent was steering.

Read throughput and stability together: a rise in one with a fall in the other is not a productivity gain.

The same test applies to Syntic Code, the terminal and desktop coding agent Syntic builds on Amara. Like any agent in this category, its value in a codebase depends on the tests, review and permissions around it.

The codebase that can say no

The direction is clear. Agents are writing a growing share of production code, and the most capable engineers are already spending more of their time defining, checking and integrating work than typing it.

What remains open is who benefits. The evidence so far says it will be teams whose systems can tell a good change from a plausible one: specific tasks, tests that encode intent, fast and accountable review, pipelines agents cannot talk their way past, and engineers who still understand the system well enough to judge. That is a different discipline from building a first version quickly, which we covered in from prompt to product. It is closer to what software engineering always claimed to be.

Sources