Stop counting AI coding output in lines and pull requests, because it counts waste

When a firm in Fiona Chen and James Stratton’s dataset switched on coding agents, its engineers produced 30 per cent more lines of code, 20 per cent more commits and 23 per cent more pull requests. The rate at which it closed Jira issues and epics did not change in any statistically significant way. The Harvard pair also checked whether the tickets got bigger or harder after adoption, which would have hidden a real gain, and found no such shift.

That is 300 million work events from 725,938 workers at 718 firms, January 2021 to March 2026, pulled from Jellyfish’s engineering analytics. I expected the paper to find a modest gain blurred by noise. What surprised me was how cleanly the extra code turns up somewhere else. Time from pull request to merge rose 49 per cent. The share of PRs with changes requested nearly doubled. Comments per PR rose 35 per cent. The share of workers doing code review went up 14 per cent. Every one of those extra lines became a task for a colleague.

So I think lines of code and PR counts, the numbers that fill AI adoption slides, have flipped meaning. Before agents, a pull request was a rough proxy for a human having finished something. Now an agent can open one at almost no marginal cost, and the count tracks how much unverified material has been dropped into the review queue. Reporting that figure as productivity books the cost as a gain.

Chen and Stratton are not alone. Faros AI’s 2025 telemetry from more than 10,000 developers on 1,255 enterprise teams found heavy AI users merged 98 per cent more PRs, those PRs were 154 per cent larger, review time rose 91 per cent, and company-level delivery metrics stayed flat. Secondhand reports of other work point the same way: SoftwareSeni, summarising CodeRabbit, says AI-generated PRs carried 10.83 issues each against 6.45 for human-only ones, and a write-up citing GitClear’s 211 million lines says copy-pasted code rose from 8.3 to 12.3 per cent of changes.

AI review was supposed to absorb this. By March 2026, 80 per cent of the firms used some form of it, yet agents wrote only 23.3 per cent of review comments. Humans are still reading the output, and that reading is where the agents’ speed goes.

The best case against me comes from Faros’s 2026 numbers, which show gains finally reaching the organisation: per developer, tasks completed up 33.7 per cent and epics completed up 66.2 per cent. Chen and Stratton themselves concede that measuring adoption per firm rather than per person probably makes their estimates too low. Perhaps the bottleneck is a transition cost and the PR counts will catch up.

Look at what else moved in that Faros data, though. PRs merged rose only 16.2 per cent, against 98 per cent the year before. The teams that started finishing more epics were the ones whose PR growth slowed sharply. A manager judging them by pull requests would have concluded 2026 was a worse year than 2025. The same dataset shows median time in review up 441 per cent, bugs per developer up 54 per cent, incidents per PR up 242.7 per cent and PRs merged with no review at all up 31 per cent. PR volume missed the gain and hid the cost in the same year. I cannot think of a better argument for retiring a metric.

Vendors will keep reporting volume because volume is what agents produce. GitHub said in October 2025 that its coding agent was contributing to about 1.2 million pull requests a month, a figure that tells you how busy reviewers were and nothing about what shipped. Chen and Stratton recommend investing in “downstream review capacity” over more code output. I would go further and change what gets counted, since a team rewarded for PR volume will buy more agent seats before it hires another reviewer.

Pull your own numbers for the two quarters before your agent rollout and the two after. Divide epics resolved by PRs merged, and put median review latency and incidents per PR next to it. If epics per PR fell while review time climbed, your agent spend is paying for your senior engineers’ reading time, and the lines-of-code chart should come off the next board deck.

Prompted by AI coding agents generate more code, but not more software, Ars Technica.