Coding agents raised lines of code, commits, and pull-request volume across 718 firms, but the study found no statistically significant gain in resolved issues or epics. The finding comes from a working paper by Harvard researchers Fiona Chen and James Stratton. It matters because the most visible signs of AI-assisted productivity, more code and more proposed changes, didn’t reliably turn into the study’s measures of finished work.
Where it got stuck: review. The researchers identify code review as the point where the gains failed to pass through. After adoption, reviews became longer and involved more revisions. Ars Technica, reporting on the paper, says average pull-request review time rose 49%. That fits a simple constraint: a team can create and propose changes faster than it can review, revise, test, and integrate them. Think of a factory that doubles the parts coming off the line while the quality-inspection station stays the same size.
Why the measures matter. Commits and pull requests are intermediate artifacts, not finished functionality. A pull request can be revised, left unmerged, fail testing, or be replaced. Resolved issues and epics are imperfect measures too, but they’re closer to the paper’s definition of completed work than code volume is.
What the study doesn’t show:
That coding agents have no value, or that no deployment improves delivery. “Not statistically significant” means the effect couldn’t be reliably distinguished from zero in this data, not that it’s zero. That agent-generated code is worse. The evidence doesn’t separate the possible causes of longer reviews: change size, code quality, testing requirements, reviewer availability, or integration complexity. Which fixes work. More reviewer capacity, clearer ownership, automated testing, smaller changes, and different integration practices may all help, but the paper doesn’t test any of them.
Chen’s research page describes the paper, co-authored with Stratton, as a job-market paper, and summarizes the broader finding as coding-productivity gains that don’t fully pass through to software output or employment.
Bottom line: For engineering leaders, the lesson is to measure AI tools across the whole path from proposed change to accepted, integrated work, not by commits or code volume alone. If review capacity is the constraint, adding more code generation will make the queue longer without making shipping faster.
