October 12, 2026 · Cohort 1 · Protocol 1 · Analysis revision 3 · Publication revision 1
A repository follow-up, stopped at 11 of 18 slots.
Full skill, shorter candidate and skill absent, on three engineered repositories covering pagination, lease fencing and transactional file edits.
Evidence cutoff: October 12 at 17:32 UTC (10:32 Arizona). The controller stopped when the last native tree could not be certified. Seven slots were never attempted. This page is a dated snapshot, not a live progress feed.
Ten of eleven retained artifacts passed their independent verifier. Four attempts have verified correct completion within the five-minute budget; six have known unsuccessful bounded completion and one remains unknown. Passing an artifact check after a deadline is different from finishing the task within budget.
| Arm | Planned | Observed | Verifier passes | Bounded successes / known | Unknown completion | Unattempted |
|---|---|---|---|---|---|---|
| absent | 6 | 3 | 3/3 | 2/3 | 0 | 3 |
| full | 6 | 4 | 3/4 | 1/4 | 0 | 2 |
| short | 6 | 4 | 4/4 | 1/3 | 1 | 2 |
Unequal observed arm counts reflect a stopped counterbalanced schedule. These fractions are descriptive and do not rank general model or skill effectiveness. Checks and repetitions do not increase the independent sample beyond three families.
What ran
Same requested model and native harnessgpt-6.1-sol through Codex CLI 0.161.0. Harbor 0.22 transports task files and executes Linux outcome checks; native actors ran on the host. This is one actor harness, not a cross-harness comparison.
Equal fixed limitsThree arms × three families × two repetitions = 18 root slots. Five minutes per root, up to three depth-one descendants, sequential counterbalanced order, zero Harbor retries and a three-hour outer deadline.
504 independently authored checksReferences passed on the host and in Harbor Linux containers. Initial buggy code and two distinct wrong controls per family failed calibration before model calls. These checks are not 504 independent study tasks.
Retained resource uncertaintyKnown whole-tree token subtotal: 2,151,752 across four fully measured attempts. Cohort and arm totals, resource ratios and dollar cost remain unknown. Partial totals must not be presented as a complete saving.
Evidence repairs and the stop
Analysis revision 2 reconstructed exact inherited immediate-parent headers for an earlier forked history without changing canonical receipts or making another actor call. It did not establish enforcement of information barriers.
Analysis revision 3 recovered cell 7 after a proven preflight timeout with zero actor calls, no parent reservation and no copied authentication. Its failure receipt was retained and the same reserved slot was reused. The original 18-slot ceiling and deadline stayed fixed; completed attempts were not rerun.
The later stop in cell 11 is different: native evidence contains lines exceeding the 2 MiB parser bound and conflicting session metadata. The diagnostic remains invalid. Its artifact passed, but bounded completion and complete usage are unknown. No zero-call recovery or automatic paid replay applies.
The study read detector reports zero certified full-body reads. The separate native detector certifies one full-arm and two short-arm reads. These are different detectors; zero here cannot support a claim that no skill was read.
Next steps
- Review cell 11 from retained evidence only. Preserve size, identity, truncation, usage and cleanup guards; any defensible reconstruction gets a new analysis revision and publication snapshot.
- Close this cohort as incomplete and account for all 18 planned slots. Its original execution window has elapsed. Any additional actor work needs a separately frozen cohort and budget; retain every completed attempt.
- Separate model-free CI status from scientific results. The pilot’s regression runs previously terminated with exit 143; local calibration is not evidence of a green remote run.
- For further exploration, register a smaller-model comparison and then a separate actor-harness comparison. Freeze exact model/harness versions, arm bytes, tools, delegation limits, task identities and resource accounting. Do not confound a model change with a harness change.
- Before broad efficacy claims, use untouched representative repository families, a declared practical effect and reliability tolerance, sufficient independent families and a fixed stopping rule. These exposed pilot fixtures can support regression and exploration.
Every planned cell
| Task / repeat | Arm | Artifact check | Correct within budget | Whole-tree tokens |
|---|---|---|---|---|
| paginated-report / 1 | absent | Pass | Yes | 122,062 |
| paginated-report / 1 | full | Pass | No | Unknown |
| paginated-report / 1 | short | Pass | Yes | 836,405 |
| lease-queue / 1 | full | Pass | No | Unknown |
| lease-queue / 1 | short | Pass | No | Unknown |
| lease-queue / 1 | absent | Pass | No | Unknown |
| transactional-edits / 1 | short | Pass | No | Unknown |
| transactional-edits / 1 | absent | Pass | Yes | 278,391 |
| transactional-edits / 1 | full | Fail | No | Unknown |
| paginated-report / 2 | full | Pass | Yes | 914,894 |
| paginated-report / 2 | short | Pass | Unknown | Unknown |
| paginated-report / 2 | absent | Unattempted | Unattempted | Unattempted |
| lease-queue / 2 | absent | Unattempted | Unattempted | Unattempted |
| lease-queue / 2 | full | Unattempted | Unattempted | Unattempted |
| lease-queue / 2 | short | Unattempted | Unattempted | Unattempted |
| transactional-edits / 2 | absent | Unattempted | Unattempted | Unattempted |
| transactional-edits / 2 | full | Unattempted | Unattempted | Unattempted |
| transactional-edits / 2 | short | Unattempted | Unattempted | Unattempted |
Provenance and versions
Execution source commit: 189cb0bf43c3d0e3b2d009f9171c1516fc36915e. Recovery source: 728e74b2ec49352a2d9c0157ebc3ebf756692293. Pilot implementation and source review.
Execution receipt SHA-256: 821dc71dea045ff8e9061b646401ef426b36396dfb8a149f6ec9b2da31602692
Report SHA-256: 5af652c335ee7f88f9456e43e72286155a75ed6cc8856cfcebf86e0802bfa48f
Publication revision 1 records analysis revision 3 of cohort 1. New analysis preserves original receipts. Changes to tasks, candidates, model, actor harness, limits or stopping rule create a new protocol/cohort. The installed skill is unchanged by this pilot; no automatic application is authorized.