Towards the Dark Factory: Level 2 to Level 3 in a Week

Part 1 of 5 in the “Towards the Dark Factory” series

On 24 September at a quarter to six in the evening, Petr, who runs SaaSForge, committed a design handoff for an iPhone app into an empty repository. Our products are Flutter and the JVM. He had never shipped a line of Swift. The wager he made with himself that evening was not “can an AI write Swift” (it can) but something narrower and, we think, more interesting: can you adopt a language you do not know by running the process, rather than by writing the code? No human would write code from the first commit. The human would write the rules, the specs and the rulings, and read what came back.

This series is the account of the first weeks. Part 1 is what the process looks like and what the first week produced. Part 2 is the validation layer. Part 3 is how the agents set their own pace on one Mac. Part 4 is how they feed the backlog. Part 5 is where the human still is, and it settles the wager, honestly.

The app is Sestava, an interval and strength timer for the iPhone with an open plan format you can write in a spreadsheet.

The vocabulary: levels and a wall

The name is older than software: a lights-out factory runs with the lights off because nobody is inside, and FANUC’s plant in Japan, where robots build robots, is the usual example. In January 2026 Dan Shapiro, the CEO of Glowforge, put it at the top of The Five Levels, a ladder of AI-assisted programming modelled on the levels of driving automation, from level 0, where not a character reaches the disk without your approval, to the dark software factory at level 5; two weeks later StrongDM’s AI team published how they work with no code written or reviewed by people, checked by scenarios often kept outside the codebase, like a holdout set and by digital twins of the services the code talks to. Saumya Tyagi’s The Dark Factory Pattern (HackerNoon, February 2026), prompted by StrongDM’s writeup and OpenAI’s Harness Engineering, turned this into a phased adoption plan, and its vocabulary gave us the words we had been missing. His autonomy levels, borrowed from self-driving cars: at level 1 the AI finishes your sentences; at level 2 it writes whole functions or files and you review every change; at level 3 it generates code from specs, holdout scenarios gate the quality and you approve the merge (a level 3.5 lets some services auto-merge); at level 4, the full dark factory, specs go in and tested code comes out merged. Between the layer that generates code and the layer that validates it stands the wall: the generator never sees the acceptance scenarios, the validator never learns how the code was produced, the same separation as train and test data in machine learning. And the backlog is wired to spec generation, so tickets become specs without a person retyping them.

We will use those terms throughout, and we will say plainly where our process is not the article’s. Two differences up front. Tyagi’s wall holds StrongDM’s holdout scenarios: plain-English acceptance tests judged by a separate LLM evaluator against an ephemeral deployment. Our wall is made of deterministic gates (100% coverage, a mutation runner, byte-for-byte plan replays) plus an independent reviewer, and we have exactly one holdout rule, which Part 2 explains. And our backlog is not wired to spec generation by a tag on a ticket; it is fed by retrospectives and a skill that files issues in a fixed shape, which is Part 4.

Where we were: level 2

In June we wrote about a hackathon week with 140 pull requests. Reread with Tyagi’s vocabulary, that was level 2 done at scale: agents wrote whole files, an agent swept the pull requests and humans decided on what it surfaced, and the throughput came from how many independent problems we could keep in flight. It worked and it was exhausting, because review was the serial part.

Sestava started at level 3 from its first commit. Nobody reads the diffs line by line. An agent implements, a different agent reviews adversarially, a fix round follows, the reviewer re-checks, the gates run, and the owner approves a merge from a pull request whose description was rewritten at the end from the reviews and the final gate results. The owner’s own review is of claims, not of code: does the description say what ran, on which commit, with what result.

The contract file

Everything starts from CLAUDE.md, the file Claude Code reads at the start of every session. In its first version, committed with the package scaffold on the first evening, it was 201 words. By 29 September it was 1,435 words and had been changed in 38 commits. It has three kinds of content.

Rules about the code. A few, all of them checkable:

• Evidence over hypothesis: say what you verified (ran, read, decoded) versus what you suspect.
• Any bug: write a failing test that reproduces it, watch it fail, then fix. Keep the test.
• Coverage is 100% with no exclusions. A surviving mutant is a failing build. No skipped tests.
• Typed errors carry location as data; the UI never matches on error text.
• No Date() in Sources/ outside one ticker type; engines take now as a parameter.
• All user-facing sentences come from one Copy catalogue; a test fails on drift between the catalogue and the code.

Rules about working with agents. This section did not exist on day one. Every line in it was paid for by an incident (Part 4 has the ledger): one git worktree per run, stage explicit paths and never -A in a shared checkout, briefs and reports passed as files, at most two Swift runs at a time and one while a mutation gate runs, one booted simulator at a time, finish a worktree the day its branch merges.

Pointers. Where the specs live, where the plans live, which script is which gate, how to file an issue.

The contract file is prose, and we have argued before that prose does not fail a build. That is still true; the difference here is that nearly every sentence in the rules names a script, a test or a label that does fail. The prose is the index, the gates are the law.

The unit of work: a spec and its adversary

Tyagi’s level 3 says “generates code from specs”. The interesting question is where the specs come from and who checks them, because a spec written in one pass by a capable model reads as finished and is not.

Our unit of work is a spec plus its adversarial review. One of the first specs, the session formats spec, looked coherent and had 38 gaps; three review rounds by an independent agent found them before a line of code existed. That became a rule on day three: every spec gets an independent adversarial review before a plan is written from it. On day four the same rule was extended to plans, after a plan that went to its pull request unreviewed turned out to contain three critical findings, two of them rulings the owner had given that the plan text had silently not absorbed.

A plan is then executed as a subagent-driven run: a controller agent dispatches one implementer per task, an independent reviewer per task, a fix round when the review requires changes, and a re-review of the fix. The implementer’s report has to show red first, with a count: “29 of the new tests were red on the old code” is evidence a reviewer can check, and the reviewers do check; the standard is that the reviewer re-runs the implementer’s reproductions and never takes its word.

One more rule came from the Apple Watch plan, written on day six. Its draft of the run authority compiled and had no tests. Tested for the first time, it failed 15 of 131 tests, and 14 of those were defects: a run finished twice, a recovery blocked for ever, a backup never cleared. Since then, code in a plan counts as verified only when its tests ran; “compiles” is reported as “not verified”.

The mechanics: worktrees, briefs, ledgers

Every run gets its own git worktree, named after the run. Every agent gets one scratch directory, named after the agent, deleted when it finishes. Briefs go to agents as files, reports come back as files, and diffs are read from git, never pasted into a conversation. The reason is mundane: a controller that carries its whole run in its context is cut off by the harness after roughly 120 to 150 tool uses, and the runs are longer than that. So each controller keeps a ledger, progress.md, with a line per step and a RESUME POINT line before every wait. When a controller is cut off, a successor reads the ledger and continues; it never re-dispatches a task the ledger says is complete. Part 3 is about what that makes possible.

Day one, hour by hour

The commit log of 24 September is the best summary of what level 3 feels like from the outside.

Thirty commits between a design document and a Foundation-only Swift package with a workout engine, two decoders, a 100% coverage gate and a mutation gate, in six hours, in a language the person at the keyboard could not have written. Not a line of it was typed by him. The next morning’s commits tightened the format’s strictness and recorded the follow-ups from the final review.

The first week in numbers

All numbers are as of the evening of 30 September (the repository’s main at 17:39 local), six days after the first commit, read from the repository and its pull requests.

• Commits on main per day: 30, 88, 105, 158, 52, 195, 98 (24 to 30 September); 726 in all. 575 of the 582 non-merge commits carry an agent’s co-author trailer; five models appear in them, the bulk from one.
• Pull requests merged: 82, between 9 and 20 a day.
• Issues filed: 120, of which 88 open and 32 closed; 25 of them carry the label a reviewer uses when it parks a finding instead of blocking a merge (Part 4).
• Swift: 409 files, of which 203 are test files; about 18,400 lines of source and about 22,200 lines of tests across the two packages and the app.
• Tests, at the typed-errors wave’s pull request of 30 September: 621 in the core package, 557 in the app-logic package, 115 unit tests in the app target, 28 UI tests on a simulator. Coverage 100% of lines, functions and regions in both packages; the gate prints the fraction.
• Mutants: 1,730 generated on the core and 1,706 killed; 1,622 on the app-logic package and 1,615 killed; every survivor is on an allowlist with a one-line justification, and a stale entry fails the gate.
• Shipped: the first TestFlight build went out on the morning of 29 September, archived by Xcode Cloud from main; the second built itself from the next merge. The owner’s device pass on that second build, the same morning, produced ten findings within the hour, filed as one issue with numbered lines.

What was built in those six days: the core package (day one), the app-logic package with stores, import, run coordination, history and devices (day two), a web editor for the plan format in TypeScript (days two and three), a wave of session formats across all three (day four), the iPhone app target with its screens (days four and five), widgets, a Live Activity and reminders (day six), the website in English and Czech, and the beginnings of a blog. Among the other specs written and reviewed that week were the session detail model, the Apple Watch, dictation and a typed error model.

We are not going to publish the token or money figures; they are not the point of this series and they would date faster than anything else in it. What we will publish is the cost in incidents, because that is where the rules came from, and it is the honest half of the story.

What level 3 is not

Level 3 in Tyagi’s ladder still has a person approving the merge. Ours does too, and more than that: a person answering rulings on product facts, reading the Czech, and holding the phone. The interesting question for the rest of the series is not whether the agents can write the code (the table above settles that) but whether the wall between generation and validation holds without a human reading diffs, and what the human’s remaining work actually consists of. Part 2 is the wall.

This is Part 1 of the Towards the Dark Factory series. Part 2: The Wall (coming on 9 October). Part 3: The Agent Sets the Pace (coming on 16 October). Part 4: The Agent Feeds the Backlog (coming on 23 October). Part 5: Where the Human Still Is (coming on 30 October).

Related: 140 Pull Requests in a Week · Linting the Prompt Layer

Cover photo by Kamekichi Photos on Unsplash.