454 Pull Requests in Twelve Days

The METR study gets quoted at me a lot. In July 2025 they ran a controlled trial with experienced open-source developers working on their own repositories, and with AI tools they were 19% slower, while believing they had been faster. It's a good study, and I don't disagree with it.

But it measured a person typing alongside an assistant, and that isn't how I work. September was the first month I had a clean enough record to show the difference properly, so this is that record.

The product is a B2B legal and claims platform with paying customers on it: claims handling, deal diligence, corporate governance and billing. It has a production database, real invoices going out, and customers whose feedback lands in the backlog. In September I ran its engineering on my own, and I mostly didn't write or review the code.

Anyone can get agents to produce code quickly. What I want to show is why I trust what reaches production at this pace. The answer isn't one clever trick. It's a set of pieces I've built one at a time since the spring, each to fix one of the problems people rightly worry about with agents, and September was the first month they all ran together on one product.

The numbers

These come straight from the repository's history on GitHub, in UK time.

September also had 6 production releases and 20 release candidates to UAT.

From 19 September most changes went in through pull requests, so the PR figures start there.

A slightly wider window, 19 September to 3 October, gives enough data to see the shape of the work.

By 26 September about 45 finished features were sitting behind flags waiting for the product owner's sign-off. The 30 September release went to production with those flags off and passed its smoke test. The product owner signed off the same day, and I switched 21 flags on.

October carried on at the same pace, and from 1 to 22 October another 844 pull requests merged and 18 more production releases went out.

What I actually did

My September was meetings about decisions, and documents about governance.

The decisions were product ones. Does a denied claim still count against a customer's monthly allowance? Should one invoice cover two months for a customer who started mid-cycle? Is "fire" its own claim type, or a label on an accident? Those are business calls that belong to the product owner and me, and no model, however good, turns them into engineering questions.

The documents were the release notes, a list of features ready to switch on for sign-off, and a decisions register. Beyond that I raised the GitHub Actions budget when CI hit its cap, sent the changelogs out, and now and then told it to merge something I wanted out sooner. Otherwise the builds ran without me.

That isn't handing an agent a merge button and walking off. What I handed over was a stack of pieces I've spent the year building.

The pieces behind the loop

Most people's view of agents comes from using one as an assistant, and the worries that come with that are fair. It makes things up about your code. It forgets everything between sessions. It repeats a mistake you fixed last month. It writes tests that pass because it wrote them to pass. It makes product decisions nobody agreed to. It breaks production. And it costs a fortune to run.

Each of those has a specific answer, and I've built one for every one of them. Each piece below started on a different job, and each is now a stage of the loop. So when someone tells me 454 pull requests sounds reckless, what they're picturing is an agent without these pieces.

Two more worries are answered later in the loop. Product decisions go to a person through the decisions register, and production is protected by flags and releases that have to be earned.

None of these was designed for this product. The brain started as my own memory. The review gate came out of a review on an earlier product of mine that found seven bugs, one of them a privacy leak, in code that had passed 118 unit tests. Grounding packs came from watching agents search confidently in the wrong place. What September shows is what happens when they stop being separate tools and become stages of one flow.

The loop, stage by stage

Here is how those pieces sit in the loop today, in the order a piece of work meets them, and what each stage is there to stop.

Work arrives already decided

Nothing starts from a vague ticket. The work comes from where the customer already talks to me: the product owner's notes, the claims team's feedback, and since October an email watcher that turns their messages into planning records. Anything that needs a business answer goes into the decisions register, one file per open question, with the name of whoever owes the answer. The pull request that finds the question opens the file, and the one that acts on the answer closes it.

That's there so an agent never quietly makes a product decision nobody agreed to. If a unit hits a question it can't answer from the record, it's meant to stop, and the question comes to a person. The register is the decision-record habit from April, applied to product questions instead of architecture.

The brain hands over what's already known

Before anything is planned, the brain assembles a context pack: what this part of the system is, what it talks to and what talks to it, what has gone wrong there before, which patterns to copy, and which words in the brief the system can't define. The blast radius comes from the code's own imports, calls and schema references, so a change to a shared function arrives with what depends on it already listed. The product's repo map and gotcha notes supply the local facts and the past mistakes. When I measured it on a real unit for the grounding-packs article, the pack cut the builder's searching by over a third (38%) at the same quality.

Without it, an agent will happily rebuild something that already exists, or repeat a mistake that's written down three folders away.

Explore before anyone writes code

A context pack can tell you what's already known about an area. It can't tell you what nobody has looked at yet. So the plan starts with exploration: reading the files the change will touch, tracing what calls them, and writing down what it finds with the file and line. The product's repo map tells the planner where to start, and anything new it learns goes back into the map for the next one.

In September an orchestrating session did this for each batch and wrote a grounded brief per unit. On 20 October that became automatic. The drawing room now takes each request, explores the repository, slices the work into small vertical units, and writes a brief for each one with a numbered fact ledger, acceptance criteria, blast radius, the migration version and the flag name.

I don't want anything built confidently on a wrong assumption about the code. A brief that points at line 44 of the claim-type mapper can be checked. A brief that says "update the claim types" can't.

Tests first, and the builder can't touch them

The drawing room also writes the failing tests for each slice before a builder even exists, and proves each one is red wherever it can run them. Those tests go onto the slice's branch first. The builder isn't allowed to edit them, and it has to pass a check that they're unchanged before it can mark the pull request ready.

This is the stage I care about most, because an agent that writes its own tests can shape them around its own code until they pass. Freezing them means the definition of done is fixed before the code exists, by a different context from the one writing it. It's the same principle as the review gate, moved to the start of the work.

To be clear on timing, before 20 October this ran inside each builder's loop, test-first by instruction rather than by construction. Freezing the tests is one of the things October added.

Build small, isolated and flagged

Each unit gets its own worktree, so parallel builders can't trip over each other. Every Claude Code builder's session is saved to the brain turn by turn, and if one fills its context window it's handed a short card of where it left off, built from its own transcript. New behaviour goes behind a feature flag that's off by default, and off has to mean today's behaviour byte for byte, with a test that runs with the flag off to prove it. A bug fix is never flagged, and it has to come with a regression test that fails on the old code. Since 9 October CI enforces that last part, so a fix whose test also passes on the old code gets rejected.

That keeps half-finished features away from customers, and stops fixes that only look like fixes.

Review more than once, and the gate is never the builder

Review happens at more than one point. When a session plans the work, the plan itself gets reviewed before any code. The builder runs its own checks and a simplifying pass, which are useful but aren't the gate. The gate is an independent reviewer, a separate session that never saw the builder's work in progress, reading the diff for security, privacy and correctness, with performance, accessibility and data checks added when the change touches them. Its findings go back to the builder and the loop goes round again until there's nothing of medium severity or above left, or a finding is accepted with a written reason. Every finding and what happened to it goes in the pull request.

Housekeeping, such as clearing up finished worktrees, runs on a small model or a plain script, so the expensive models only do the work that needs them.

Some units clear in one round. The billing work that added bank debit payments was still drawing high-severity findings in its fifth, and needed a sixth to close. The builder never marks its own homework at the gate, and a reviewer that never saw the builder's reasoning can't share its blind spots.

The local gate and CI

Before every push, a local preflight runs the type checks, the lint ratchet, the build, the unit tests, the edge function tests, the house-style checks and the migration checks, including that every migration that changes a database function or policy has database tests. An integration gate then runs the database tests and the relevant end-to-end specs against a real local stack. CI runs the full suite again on every ready pull request that touches application code, with the end-to-end tests split across shards. A secrets scan joined in October.

The test suite only grows

For me this is the part that makes the speed safe over time, not just on the day. Every unit leaves its tests behind and they join the permanent suite, so every later change runs against all of them. A feature built in October has to pass everything September proved.

That growth is steady week by week, it's not an import. It's what 454 pull requests look like when each one has to leave proof behind.

Releases have to be earned

Merging only deploys to the development environment, and the builder walks the change there. A release tags a candidate and deploys it to UAT, but only if CI is green on that exact commit, and then a browser smoke test runs against it. Production gets exactly the commit UAT ran, and only when I explicitly say so. Every new feature is still behind a flag at that point, so going live for customers is its own deliberate decision.

The changelog that tells customers what changed gets checked line by line against the merged code and the live flag state, and then checked again by an independent agent before it goes out. "It merged" isn't the same as "it works", and I don't want a customer told about something that isn't actually live for them.

How the loop gets better

The pieces don't stand still. When something goes wrong, the fix goes into the place the next agent will read it: a gotcha note the context pack will hand over, a line in the repo map, or a check in the gate. When four branches broke each other by renumbering migrations by hand, a script took over the renumbering, a bot reruns it when another merge moves main, and the briefs started assigning the version. When the planner refused two units silently and the work sat for hours, refusals started going to a watcher that flags them to me.

That's why the gotcha count matters to me more than any model upgrade. Models improve whether I do anything or not. The 155 notes for this product are mistakes no builder on it should make twice.

The team version keeps the human gate

I've also written a softer version of this flow for an engineering team. The backbone is the same: explore and plan before building, a context pack per ticket, tests first, one builder per unit in its own worktree, and independent review. The difference is that a person approves every pull request, no agent merges, and the team stops the trial if escaped bugs or reverts go above their six-month range.

I think that's the right shape for a team, where trust is shared between people and a surprise lands on someone else. On my own product I've swapped the human approval for stricter machine gates instead: frozen tests, mandatory flags, fixes that have to prove the bug, the integration gate, and release candidates with smoke tests. Human review matters. But a person glancing at a diff is the weakest of those controls, and on a product I own I make the stronger ones mandatory.

The pushback I get

Nearly 500 changes into production and not one approved by a person? That's gambling with a nice dashboard.

Every code change was planned against the real code, built test-first, reviewed by a context that never saw its reasoning, and run against every test that came before it, and almost every new feature sat behind a flag until a person switched it on. I don't believe a person approving a 321-line diff at 23:00 is a stronger control than that. The human judgement didn't go anywhere, it moved to what gets built and when it goes live. And the outcome is there to check: no reverts in September or October.

38 PRs a day just means you've split the work into crumbs.

The median PR is 321 lines. SmartBear's study of code review at Cisco found people catch the most defects when a review covers 200 to 400 lines, so these are normal-sized pieces of work. Each one is a vertical slice small enough to review whole and prove end to end. I'd rather people look at the 45 or so features ready for sign-off by 26 September anyway. They included bank debit payments for invoices, a combined first invoice for customers who start mid-cycle, a corporate governance module that builds minutes from the record, and document intake that reads up to five corporate documents and fills in an entity for checking.

Fine for a prototype. Not for a live product.

It ran on a live product. The bank debit work follows a payment from submission to settlement and returns, so nobody gets charged twice or put on hold while money is moving. It went through six review rounds and merged with its flags off.

The method held, the laptop didn't

At this volume, the limits I hit were hardware and cost, not method.

With several builders, local integration stacks and test runs all on one laptop, the load sat at 100%, and every builder's preflight started failing on timeouts that had nothing to do with its own change. And CI cost real money. GitHub Actions hit its budget cap on 28 September and CI stopped until I raised it. By mid-October the spend had reached the new $800 cap. Both are what the next article is about.

What it actually is

None of this needed a new model. I used the same models in September that everyone else has.

What made the month possible is everything around them: a brain that remembers what the team knows, maps that show how the system fits together, packs that hand both to every agent before it starts, tests fixed before the code exists, a reviewer that can't share the builder's blind spots, a test suite that only grows, and a release path that has to be earned. Each of those was a fix for one problem when I built it. Together they are a way of building software.

So I don't think the METR result and my September contradict each other. If you use AI as a faster keyboard, you're measuring typing speed, and the study suggests you might not even get that. If you build the scaffolding and move yourself to where the decisions are, you're measuring something else. For me that was 885 commits, 454 pull requests, six production releases and no reverts on a live product, while my own time went on the decisions and the documents.

References

• METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity", July 2025.
• SmartBear, "Best Kept Secrets of Peer Code Review" (the Cisco code review study), 2006.
• Repository history for the product, GitHub, August to October 2026 (private).

Related Dendro Logic writing

• The Repo Matrix: the mechanical map agents read before they suggest code.
• The Same Bug, Every Six Months: why gotchas are first-class memory.
• Resumption-First Memory: design rules for sessions that survive interruption.
• Shipping While We Sleep: the build loop and the independent review gate.
• Closing the Loop: why the merge can safely be ceremony.
• The Drawing Office and the Yard: how work is planned, routed and built.
• The Cheapest Model That Can Safely Do the Job: routing work by complexity.
• What the Agent Does Not Know to Look For: the grounding packs each builder starts from.

Builders: mainly Codex and Claude Code, headless, with Antigravity added in October, one git worktree per unit. Review: a separate agent session per pull request. CI: GitHub Actions in September, moving to self-hosted in October. Repository figures from the GitHub API, other figures from the project's own records, 22 October 2026. Gotcha count from the knowledge base, 23 October 2026.