# Adding a software factory workflow to codex: same pass count, more time and tokens. In the last post I talked about how a skill I'd had in my agents context had been adding a large number of time and tokens with (so far) no proven benefit in terms of increased accuracy or performance. I tried a much more complex orchestration workflow in AI-DLC: [AWS Labs describing AI-DLC](https://github.com/awslabs/aidlc-workflows/blob/v2.11.0/README.md) paired with more complex tasks. Both workflows passed four of five repository tasks. With AI-DLC, the attempts used **{{timeRatio}}× the agent time** and **{{tokenRatio}}× the tokens** of the existing workflow. ## What I compared After testing an adversarial TDD skill, I wanted to know whether a broader delivery workflow would earn its extra effort on my specific repository work. This is basically a software factory which was trending on Linkedin and I wanted to compare whether or not the increased overhead of the requirements gathering, decisions, implementation and review through the persistent workflow would actually improve results noticeably. Each task ran once with this repositories existing instructions, skills and tools, and once with that same setup plus the AI-DLC Classic workflow profile. An unattended adapter answered approval prompts using a fixed policy and recorded the approvals as automated. Both variants used the same Codex model, harness, project instructions, skills and tools; one also used AI-DLC Classic. The tasks were deliberately selected for bounded features with interacting components, error cases and independent acceptance checks. The pool covers API, UI, search indexing (Open Library's code for adding, updating and removing works, editions and authors in its Solr search system), database and parsing work. ## Variants baseline = existing workflow; v1 = with AI-DLC. Totals include all five attempts per variant, including failures. The existing workflow used about 50 minutes of agent time and 22.0 million tokens; AI-DLC used 319 minutes and 301.3 million tokens. Totals cover all five attempts per variant and the main and child Codex sessions, including cached input and tool output returned to the model. Agent time excludes setup and grading. ## Cases 1 = artifact passed; 0 = patch replay failed. Open a prompt for time, tokens and task sources. Saved patches were replayed on fresh task repositories using the task-specific [SWE-bench Pro V2](https://github.com/scaleapi/SWE-bench_Pro-os) acceptance tests. Open Library used `openlibrary/tests/solr/test_update_work.py`. Both NodeBB patches failed to apply before testing; the other four repositories passed in both variants. ## What I take from this The added process used more time and tokens without an observed advantage on these checks. As I'm realizing more and more, the tasks these tools are being asked to run are potentially more complex than the ones I'm asking it to do but my practical lesson: before standardizing a workflow (even one that's experiencing an explosion of popularity), compare it with the setup you already use on a few real tasks. Decide what improvement would justify the extra effort. You can try this on Rulemetric. I will also be adding followup tests to check different lower powered models and other harnesses, for a more thorough study, you'll be able to see those results below: ### All the caveats, plus some explanation about the plan This is an exploratory pilot: five repository pairs, one run per variant, and a formal verdict of **{{formalVerdict}}**. The equal pass counts do not establish equivalent quality. Human review, handoffs and long-term maintenance could offer benefits this unattended test did not measure. ## Benchmark sources This study used **[SWE-bench Pro V2 · Scale AI](https://github.com/scaleapi/SWE-bench_Pro-os)**. Other candidates are [Terminal-Bench 2.1](https://www.tbench.ai/news/terminal-bench-2-1), [DeepSWE · Datacurve](https://github.com/datacurve-ai/deep-swe), GitHub's internal [CheckpointBench](https://github.blog/ai-and-ml/github-copilot/project-hydrafusion-frontier-quality-via-multi-model-orchestration/), and RuleMetric's own fixtures.