# The architecture ablation — brain + planner against bare

Run **6 August 2026**, 15 runs, 3 reps × 5 tasks, one at a time. `deepseek-v4-flash`, effort `high`. Identical to the baseline in every respect except `ORANGE_BRAIN=1 ORANGE_PLAN=1`. Prices from `pricing.py`, fetched 2026-07-05, re-verified 2026-07-19.

Baseline: tag `flash-bare`, $0.0906, $0.0060/run. Ablation: tag `flash-brainplan`, **$0.1243, $0.0083/run**.

**The predictions below were written into `LOOP-STATE-04.md` before the run started**, so the result could not be rationalised afterwards.

---

## The result

| task | kind | bare | brain + plan | delta |
|---|---|---|---|---|
| T1 | fix | 1.000 | 1.000 | 0.000 |
| T2 | fix | 1.000 | 1.000 | 0.000 |
| T3 | fix | **1.000** | **0.833** [0.500–1.000] | **−0.167** |
| T4 | compound | 0.972 [0.917–1.000] | 0.944 [0.917–1.000] | −0.028 |
| T5 | vague | 0.833 [0.750–1.000] | 0.861 [0.833–0.917] | +0.028 |

**Cost rose 38%** — $0.0083 per run against $0.0060 — and call counts rose on every single task: T1 19.0 vs 16.7, T2 11.7 vs 8.7, T3 17.7 vs 13.0, T4 18.7 vs 15.3, T5 22.7 vs 18.0.

## Against the pre-registered predictions

**Prediction 1 — "if the brain helps, T5 change should rise and the overdue defect should stop being missed."**

Half satisfied, and the half that matters failed. T5 did rise, by 0.028, which is well inside the noise of three runs. But **the overdue-boundary defect was still missed in 2 of 3 runs** — `overdue set` and `due-today excluded` each failed twice, exactly as in the bare configuration. The repository map did not help the agent find the defect nothing pointed it toward, which was the entire hypothesis.

**Prediction 2 — "if the planner helps, T4 should stabilise at 1.000 rather than 0.972."**

Falsified. T4 got **worse**: 0.944 against 0.972, and `fmt(0.005)` failed twice under the ablation against once bare.

**Prediction 3 — "if Phase 0 repeats itself, both stay flat and cost rises, which is a publishable negative result."**

This is what happened, plus a regression nobody predicted.

## The unpredicted finding: T3 broke

T3 scored **1.000 in all three bare runs**. Under brain and plan it scored **0.833 [0.500–1.000]** — one run collapsing to 0.500 on a task that had never previously failed.

The failed assertions name the mechanism:

- `SO-1005 discount == 6.6446`
- `single-discount order unchanged`

T3 asks the agent to make stacked discounts compound rather than sum. In the failing run it **fixed the stacked case and broke the single-discount case** — a regression inside the task's own correctness axis, invisible to the separate regression suite because that suite deliberately asserts only things unaffected by any of the three fixes.

Adding a planning pass to a task that was already solved reliably made it unreliable. That is the strongest single result in the ablation and it runs against the architecture.

## The shape change nobody was looking for

T5's range moved in an interesting way:

| | min | max |
|---|---|---|
| bare | 0.750 | **1.000** |
| brain + plan | **0.833** | 0.917 |

The ablation **raised the floor and lowered the ceiling**. Bare reached a perfect score once in three; the ablation never did, but never dropped as low either.

A plausible reading is that the repository map and the plan constrain the agent's search — it follows the supplied structure instead of exploring, which prevents the worst run and also prevents the best one. At n=3 this is a shape in six numbers and nothing more, but it is a testable hypothesis and worth stating as one.

## Debris nearly doubled

**10 of 15 runs (67%) left a scratch file**, against 5 of 15 (33%) bare. One run (T5 rep1) left four: `_cleanup.py`, `_cleanup2.py`, `_final_cleanup.py`, `_sanity_check.py`.

**Correction, 6 Aug:** an earlier count said 9 of 15. It used a keyword list (`_verify`, `_check`, `_tmp`, ...) that failed to match `_repro.py`, left by T4 rep2. The audit recomputed debris as *any changed file beginning with an underscore*, which is the rule that should have been used, and found the extra run. The bare figure of 5 of 15 is unaffected.

The planner appears to encourage the agent to write and run its own verification scripts. That is defensible behaviour on its own terms — the agent is checking its work — but it doubles the amount of litter left in the repository, and nothing in the suite penalises it.

## Regression

**0 of 15 runs caused any collateral damage**, identical to the baseline, on the axis the `vandal` control demonstrates can detect five distinct classes of damage. The ablation neither helped nor hurt here.

## What this establishes

On this suite, with this model, the repository-map preload and the planning pass:

1. **Changed nothing on tasks that already passed** (T1, T2).
2. **Made a reliably-solved task unreliable** (T3, −0.167, one run at 0.500).
3. **Slightly worsened the compound task** (T4, −0.028).
4. **Did not fix the diagnosis failure they were supposed to fix** — T5's un-hinted defect was missed at exactly the same rate.
5. **Cost 38% more** and used more calls on every task.
6. **Nearly doubled the debris** left behind.

This replicates Phase 0's finding, and it replicates it on much better evidence. Phase 0 observed no gain on a suite so saturated it could not have shown one. This suite is not saturated — T4 and T5 both move — so the absence of a gain here is informative rather than merely uninformative.

## What this does not establish

- **n=3.** The deltas on T4 and T5 are single-task swings. Only the T3 regression is large enough to be interesting at this sample size, and even that rests on one run scoring 0.500.
- **The two features are confounded.** `ORANGE_BRAIN` and `ORANGE_PLAN` were switched together, exactly as in Phase 0, so nothing here attributes any effect to the map or the planner individually. A 2×2 design was never run and should be.
- **One model, one repository, ~200 lines.** The author's standing argument — that the brain's value is orientation speed on real projects with history, which no seeded benchmark reproduces — is untouched by this result and remains unmeasured.
- **Latency was not the measured axis**, though it rose: the ablation took 32.1 minutes against the baseline's roughly 20.