Taking an engineering org AI-first, in stages
Cycle time halved and substantive pull requests per contributor tripled. Over more than a year I moved the United States Gold Bureau's engineering org onto an AI-first model in stages: early Codex, then a Cursor and Claude Code mandate as the models got good, a shared knowledge base in Notion, and a move off Jira to Linear.
- 50% Reduction in cycle time
- 3× Substantive PRs per contributor
- Up Median PR size
Context
In early 2025 I could see we needed to modernize how we built software. The tools were about to change the job, and I didn’t want us catching up to that late.
The easy version of “going AI-first” is to hand the team a code generator and watch output climb. The harder and more honest version is to change how the team actually works, and to be able to show the work got better, not just faster. We did it in stages, over more than a year, as the tooling earned more trust.
The constraint
This wasn’t something you could switch on. When I first gave the team access to Codex in early 2025, we used it lightly. The models weren’t good enough yet, and most of what they produced needed heavy review, so the early return was thin. You have to be willing to sit through that phase.
Counting output wouldn’t have told us much either. Hand a team a code generator and commits, pull requests, and closed tickets all go up, and they go up fastest on trivial work. The number rises, the reporting is technically true, and the conclusion drawn from it is false. So the real problem was two-sided: adopt the tools as they got good enough to trust, and measure the change by quality, not just volume.
The call
It started slow. In early 2025 I made sure every developer had access to Codex, and we used it sparingly. It took real review to catch what it got wrong, and honestly the payoff was modest at first.
That changed at the end of 2025, when the models got good. I decided to go all in on modernizing how we worked. We mandated Cursor for the team and moved our coding onto Claude Code and Cursor. Pull request volume went up, but so did quality, and we could see it in our code-quality tooling, SonarQube and SonarCloud, rather than taking it on faith.
Then we made the environment fit the way we now worked. We retooled parts of our pipelines and added agent instruction files to the repositories, so the assistants had the context and guardrails they needed. That did two things at once: it brought our tooling costs down and it made the agents more accurate.
We also gave the team a shared memory. I built a knowledge base that ingests everything from meeting transcripts to how-tos to decision records, all reachable from Notion where people already work. That matters more than it sounds: an assistant with no access to why a decision was made will confidently rebuild something you deliberately tore out.
The last piece was project tracking. By spring 2026, Jira was fighting us. Working side by side with agents, we were shipping faster and carrying less overhead, and we didn’t have the staff or the appetite to maintain heavy story management just to keep the tool fed. I moved the team to Linear. It kept us moving, and it gave leadership the visibility they wanted, project status, timelines, and milestones, without the management overhead that quietly kills throughput.
The way we take in work changed too. Someone can suggest a project and it gets pulled into Notion, and from there the scoping starts on its own. Agents kick off, pull context from the knowledge base, review the relevant codebase, and lay out the knowns and the known unknowns as a first-pass scope. Then we review that, and the developers do another pass on their side. On the autonomous path, work can move from idea to project scope to implementation scope to code to review without a human touching each step by hand.
Where the human gate sits depends on the risk. A big project never goes straight into a coding session; a person is always in the loop before that happens. Smaller issues and bug reports can go directly to coding, and we review what comes back. Keeping people on that gate is what makes it safe to let the earlier steps run on their own.
I’ll be straight about the soft spot here. “Substantive” versus “trivial” isn’t a clean line, and I’m wary of any productivity story that rests on one tidy definition, because that’s exactly the number that’s easy to game. So I don’t hang the case on it. What I trust is that several signals moved together: median pull request size went up rather than down, the share of trivial work fell, and SonarQube gave us a real read on quality instead of just a count. Real work moving through the system pushes all of those the same way. Gaming a count with trivia would push at least one of them the other way, and it didn’t.
The tradeoff
The honest costs showed up in two places. Early on, before the models were good, we paid in review time for output we couldn’t fully trust, and for a while the return didn’t justify the effort. That phase is real and you can’t skip it.
The other cost was literal. Running these tools across a team adds up, and left alone the spend grows. Bringing it down was part of why we retooled the pipelines and added agent files, and it’s ongoing work, not a one-time fix.
The outcome
Cycle time roughly halved. Substantive pull requests per contributor tripled, with median PR size up and trivial work down, so the added throughput landed on real work rather than filler. Retooling for cost brought spend down while accuracy went up, and leadership got clearer visibility into what was actually shipping.
The team also got leaner over this period, and it’s worth being straight about why. Headcount was cut by about 30 percent, and not because of the tooling. Budget was moved into marketing and sales. What the tooling bought us was the ability to take that cut and hold output flat, then start pushing it higher. Once the board could see we were doing the same work and more with fewer people, they decided the smarter move was to invest back into the team and grow headcount, so we could take on more and help other parts of the business grow. The tooling didn’t shrink the team. A budget call did, and the productivity is what turned that call around.
The quality signal I trust most is what happens after a release. Before the agentic pipeline, cross-platform code reviews often turned into rubber-stamps, people approving changes in areas they didn’t really know, and we paid for it in regression bugs after production releases. That has flipped. Review quality is up, we catch and fix more before anything gets committed, and post-release regression bugs are down sharply. Bug resolution got faster too, because we run coding sessions directly against the bug. Fixes that used to take a couple of weeks now land in a day or less.
What I’d do differently
If I were doing it again, I’d put more of our engineers’ time into the pipelines themselves, and earlier. We were focused on delivering features and value to the business, and in that push we didn’t rebuild the pipelines properly for the new way of developing. That’s the piece I’d change.
I’d also go deeper on QA sooner, specifically automating front-end testing as part of it. That’s the kind of thing you want to handle first, close to test-driven development, and work back from there rather than bolt it on after the fact.