I built an AI Software Factory, then paused using it.

Who am I

I am a bootstrapped solopreneur building my second SaaS app. That matters as context, because I do not have VC money to burn on expensive pipelines that only offer promises. I need systems that deliver cost-effective outcomes.

Summary

Three issues caused the factory to cost far more than I expected, while not delivering enough over running agents via an ADE (Agent Development Environment) or multiple Claude Code terminals.

• High token cost in review/fix lanes to ensure autonomous delivery. (% of limit or $ raw tokens)
• You cannot easily hot-swap foundation models in lanes.
• A reviewed plan does not get you a well-built change.

Paused, not scrapped. The factory still wins when I will genuinely be away from the computer for an extended period (6hrs+) and have to ship a pre-reviewed PRD. I go in accepting that I will have to read the shipped code when I get back, and sometimes unpick a choice that is already merged.

What I stopped doing is reaching for the software factory by default. For work I am around to supervise, an ADE or a few Claude Code terminals gets me there for less, and I catch problems while they are still cheap to fix.

What my Software Factory looked like

I did what all the blog posts say and built a team of AI agents to review PRDs (Product Requirements Documents), write the code, test in my development environment, fix issues, and ship it.

Lesson 1: high token cost in review/fix lanes to ensure autonomous delivery

Though I am on the Claude Max plan, I am looking at this from a dollar cost perspective for two reasons. First, there is no documented token limit per period, e.g. the 5 hour rolling window. Second, it is the most straightforward extrapolation and comparison measure.

These are medians drawn from 455 software factory runs and ADE sessions on DeliverTrust and estimated at current API pricing.*

Takeaway: The review and fix lanes in a software factory are necessary but account for 42% of the cost of an issue, and 35% of its tokens.

Explanation: In a software factory, the work/ship agents are supposed to autonomously code the PRD. To do it right, the agent needs a full, error free, question free plan. ADE assisted deployments can mostly skip review/fix rounds because I am there and can see issues during work, answer questions the agent has, or catch bugs in the development environment before verification.

Notes:

• Stages do not share memory. Each one starts cold and re-reads whatever it needs to execute.
• If you tell your review agent to find issues with your PRD, it will find issues and you may approach an infinite loop of fix rounds. Instead, tell the review lane agent to confirm that the plan will produce the documented outcomes.

Lesson 2: you cannot easily hot-swap models in lanes

The obvious way to control cost is to match the model to the lane. Run the expensive models where judgement matters, run something cheaper for the mechanical parts, treat the choice as a dial. That assumes the models are interchangeable. They are not.

Foundation models have different, often undocumented, constraints. Those constraints decide what a lane can physically do.

My verification lane signs in to a dev copy of the product using test credentials. On Sonnet 5, the agent refused to enter credentials. I had no choice but to use Opus 5. That forced me into a model that is 2.5x more expensive.

So, the per-lane model choice is not just a cost dial, it is a capability constraint, and you find out which one you are holding after you have built the lane.

Lesson 3: a reviewed plan does not get you a well-built change

Now the problem shifts from technical to operational.

The premise is that if you get the plan right, the build is mechanical. Scope it, review it against the real code, repair what the review finds, and what comes out should be roughly what a careful engineer would have written. That is not what happens.

Even if the code is functionally correct, it very plausibly does not meet my architectural hopes or coding practices. It is not trivial to just prompt engineer your way out of that.

So, the human gate was in the wrong place. I built the pipeline to put me at the end, signing off on it after it was built and tested. What I actually needed was to be in the middle: reading the diff while it is being written.

What I did instead

I still use the plan, review, and fix lanes. Those just happen locally, using the same prompts that I developed for the software factory.

But, I moved to an ADE, Orca, which is built around the assumption that I am there: several agents working at once, each in its own isolated copy of the repo, and me watching what they are doing rather than reading a report afterwards.

The practical difference is that I can catch the agent taking a suboptimal approach before it gets to the development environment. You likely cannot have a junior engineer do this as effectively because they don't have the "spidey sense" to feel when an agent is going down a bad path.

Verification became a choice rather than a lane. Some changes get automated end-to-end checks because they are worth the tokens. Most get checked by me, in the browser, in less time than it takes an agent to describe what it is about to click. That single decision removed the most expensive automated lane I had.

If you are about to build one

First, create prompts for each lane in isolation and tune them to see how well they work and how many tokens they use (pick whatever sets of models you think are best for you). The cost implications may surprise you and stop you from continuing any further.

Then, manually run through them end to end, in fresh agents per lane, as if it were being done through the software factory. This will show how long the entire process will take, where you might have review + fix loops, and how much a single PRD will cost you.

If the result is a software factory, good for you. You did it in a thoughtful, intentional, and deliberate way.

If you stopped short of a software factory, the prompts you created can become skills to help you in your software engineering efforts.

And if you build one and then mothball it, that is not a failure either. Mine is still there for the nights I am not.

* Token counts and costs are medians across 408 real lane runs and 23 planning sessions between 9 August and 3 September 2026, computed per run from its own usage record and priced at Opus 5 API rates as of 6 September 2026: $5 per million fresh input tokens, $25 per million output, $0.50 per million cache reads and $10 per million cache writes at the 1-hour TTL these runs used. Cache reads are 93-97% of billed input, which is why the totals are far below what the headline token counts suggest. Usage is counted once per message: the transcripts record one row per content block and repeat the usage figures on each, so a naive sum roughly doubles every number. The ADE figures are the same method applied to 24 shipped workspaces over the following week, one per issue - 455 measured runs and sessions in total. The work lane runs in a container I do not collect from; its row is estimated from the ratio it held to the verify lane in a run I costed by hand.