October 12, 2026 · Protocol revision 2 · Analysis revision 5 · Publication revision 2
All 18 repository attempts are now accounted for.
Full adversarial TDD skill, shorter candidate and skill absent, on three engineered repositories covering pagination, lease fencing and transactional file edits.
Evidence cutoff: October 12 at 23:07 UTC (4:07 p.m. Arizona). This descriptive view spans two registered execution windows: 11 original attempts plus seven authorized continuation attempts. The original cohort remains incomplete at its earlier cutoff; publication revision 1 is preserved.
Correct completion within the five-minute limit: 5/6 with the skill absent, 2/6 with the full skill, and 4/6 with the short candidate. Seventeen of eighteen submitted artifacts passed independent checks. Eleven attempts completed correctly within budget; seven exceeded the actor deadline. A passing artifact after the deadline is different from completing within budget.
| Arm | Observed / planned | Artifact passes | Correct within 300 s | Sum elapsed seconds | Unknown token usage |
|---|---|---|---|---|---|
| absent | 6/6 | 6/6 | 5/6 | 1,171.30 | 1/6 |
| full | 6/6 | 5/6 | 2/6 | 1,747.36 | 4/6 |
| short | 6/6 | 6/6 | 4/6 | 1,617.58 | 2/6 |
Six attempts per arm, from three engineered families repeated twice. These are descriptive counts across two execution windows, not 18 independent natural incidents or a confirmed population ranking. The same requested model and actor harness were used throughout.
Results by task family
| Engineered family | Absent | Full | Short |
|---|---|---|---|
| paginated-report | 2/2 | 1/2 | 2/2 |
| lease-queue | 1/2 | 0/2 | 1/2 |
| transactional-edits | 2/2 | 1/2 | 1/2 |
Each fraction counts correct completions within the fixed budget, out of two repeats. Family results are descriptive; no efficacy or equivalence claim follows from this small pilot.
What ran
Same requested model and native harnessgpt-6.1-sol through Codex CLI 0.161.0. Harbor 0.22 supplied task transport and Linux outcome verification; native actors ran on macOS. This does not compare smaller models or alternative actor harnesses.
Equal attempt limitsThree arms × three families × two repetitions = 18 roots. Five minutes per root, up to three depth-one descendants, sequential counterbalanced order and zero Harbor retries. Both windows were registered with a three-hour outer limit; recovery retained the continuation deadline.
Independent checksAll 504 reference checks passed on the host and in Harbor Linux containers before execution; buggy starts and six distinct incorrect controls failed. These checks do not increase the sample beyond three independent engineered families.
Resource totals remain unknown7,560,390 known whole-tree tokens across eleven fully measured attempts. Seven attempts have unknown usage. Every arm’s total tokens and dollar cost remain unknown; this subtotal is not the cost of all eighteen attempts.
Two execution windows, preserved evidence
The original cohort stopped after eleven attempts at 17:32 UTC. The earlier publication records that cutoff, including seven unattempted slots and unknown completion for cell 11. It has not been rewritten.
A separately registered continuation authorized only the seven untouched slots, starting at 22:18 UTC under protocol revision 2. After two more attempts, a native fork-header certification issue paused execution. Analysis revision 5 recovered that evidence without another actor call and reserved only the five remaining slots under the same continuation deadline. All eighteen roots were accounted for by 23:07 UTC. Completed attempts were not replayed.
Separate bounded diagnostics certified retained cell 11 at 974,228 tokens and cell 13 at 918,239 tokens. Canonical receipts and the original 2 MiB parser guard remain unchanged. Inherited fork histories do not establish enforcement of workflow information barriers.
The study detector certifies zero full-body reads; the distinct native detector certifies two full-arm and four short-arm reads. These witnesses are reported separately. Availability is the assigned treatment; results are not restricted to read-certified attempts.
Observed descendants: absent 0, full 17, short 18. All arms retained permission to delegate under the same three-child limit. The full and short arms each recorded one spawn denial; that limit may constrain a multi-cycle workflow.
Every planned cell
| Cell / task / repeat | Window | Arm | Artifact check | Correct within budget | Whole-tree tokens |
|---|---|---|---|---|---|
| 01-paginated-report-absent-r1 | 1 | absent | Pass | Yes | 122,062 |
| 02-paginated-report-full-r1 | 1 | full | Pass | No | Unknown |
| 03-paginated-report-short-r1 | 1 | short | Pass | Yes | 836,405 |
| 04-lease-queue-full-r1 | 1 | full | Pass | No | Unknown |
| 05-lease-queue-short-r1 | 1 | short | Pass | No | Unknown |
| 06-lease-queue-absent-r1 | 1 | absent | Pass | No | Unknown |
| 07-transactional-edits-short-r1 | 1 | short | Pass | No | Unknown |
| 08-transactional-edits-absent-r1 | 1 | absent | Pass | Yes | 278,391 |
| 09-transactional-edits-full-r1 | 1 | full | Fail | No | Unknown |
| 10-paginated-report-full-r2 | 1 | full | Pass | Yes | 914,894 |
| 11-paginated-report-short-r2 | 1 | short | Pass | Yes | 974,228 |
| 12-paginated-report-absent-r2 | 2 | absent | Pass | Yes | 106,286 |
| 13-lease-queue-short-r2 | 2 | short | Pass | Yes | 918,239 |
| 14-lease-queue-absent-r2 | 2 | absent | Pass | Yes | 127,572 |
| 15-lease-queue-full-r2 | 2 | full | Pass | No | Unknown |
| 16-transactional-edits-absent-r2 | 2 | absent | Pass | Yes | 161,503 |
| 17-transactional-edits-full-r2 | 2 | full | Pass | Yes | 2,056,457 |
| 18-transactional-edits-short-r2 | 2 | short | Pass | Yes | 1,064,353 |
Window 1 is the original eleven-attempt cohort; window 2 is the seven-attempt continuation. Unknown usage stays unknown. Separate diagnostic certification is recorded in the JSON alongside canonical tree status.
What this supports
In this small pilot, the full workflow produced fewer correct completions within the fixed five-minute budget than the absent arm. The shorter candidate completed four of six. This supports further investigation of workflow overhead under these limits; it does not establish general TDD effectiveness, equivalence, or a reason to delete the skill for every task.
A broader confirmation needs untouched representative repository families, a predeclared practical gain and reliability tolerance, sufficient independent families and a fixed stopping rule. Smaller-model and actor-harness comparisons need separate registrations. The installed skill is unchanged by this pilot; there is no automatic application.
Provenance and publication history
Original execution source: 189cb0bf43c3d0e3b2d009f9171c1516fc36915e. Continuation recovery source: c26361d83b28b25a4881aee3705cb2c02ef29b0e. Pilot implementation.
Latest execution receipt SHA-256: 81587321380e71d82545cdf1554e3557a4b5e6ff7bf29893e369cab0998ad86c
Latest report SHA-256: 8596d5c7c2cc2d9e7cf5bc70e3863923c4819dbf1352e407a97f8400e82c3d53
- Publication 1 / data: protocol 1, analysis 3; original 11-slot cutoff at 17:32 UTC.
- Publication 2 / data: protocol revision 2, analysis 5; completed descriptive schedule across two registered windows at 23:07 UTC.
All published studies and version policy. Publication changes preserve historical snapshots; the combined view does not silently extend the original protocol or pool it into a new efficacy estimate.