docs(tdd-criteria): rule 4 — test-immutability defence (was X; is Y marker)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

.wiki/concepts/tdd-criteria-design.md

@@ -4,6 +4,7 @@ source: .meeting-room/.archive/2026-05-07-tdd-criteria.md
 status: promoted
 type: design
 title: tdd-criteria-design
+amended: "2026-05-07: added test-immutability defence (Anti-loophole rule 4) after user noted symmetric vandalism risk on tests"
 ingested_at: '2026-05-07T04:01:23.616Z'
 ingested_by: OpeItcLoc03@DESKTOP-NSEF0UK
 source_project: .meeting-room
@@ -33,6 +34,17 @@ In a workflow that includes LLM coders (Claude, ChatGPT, future unknowns) and ro
 
 This reframes everything below:
 
+### The contract is only as strong as the contract itself
+
+But there's a **second-order vandalism mode** that the bare contract argument doesn't cover: the agent doesn't delete the code, it rewrites the **test**. Test fails → agent changes the expected value, adds `.skip`, or deletes the test → test now passes → success reported.
+
+If the contract artefact (the test) is rewritable by the same agent that's failing to satisfy it, the invariant collapses back into an artefact. The defence requires **two layers**:
+
+1. **Code is defended by tests.** Ironclad rules 1-4 below.
+2. **Tests are defended by process discipline.** Anti-loophole rule 4 below — append-only by default, modifications require literal-`was/is` marker in commit subject, test changes are separate commits from impl changes.
+
+Both layers are needed. Either alone leaves a path-of-least-resistance route to «success».
+
 - **Ironclad** rules (TDD obligatory) — zones where recovery cost > defending cost. Signal mandatory.
 - **Permissive** carve-outs — **zones of accepted risk for agentic vandalism**, not «zones where TDD doesn't apply». You explicitly accept that an agent can delete-and-claim-success here, because recovery is cheap (eyeball the next render; throwaway by contract; one-shot already ran; wrapper trivial to reconstruct).
 
@@ -85,9 +97,35 @@ Not «TDD doesn't apply». **«You accept that an agent can vandalise this witho
 
 ## Anti-loophole
 
-- **Skip без категории не существует.** One of four explicit categories — not «other reasons». No marker = violation, regardless of perceived justification.
+1. **Skip без категории не существует.** One of four explicit categories — not «other reasons». No marker = violation, regardless of perceived justification.
-- **Spike survivor rule.** If spike code is merged to master → same merge-commit creates `[backfill-tests-<slug>]` task in `.tasks/STATUS.md`. Otherwise «spike» becomes «skipped tests forever».
+2. **Spike survivor rule.** If spike code is merged to master → same merge-commit creates `[backfill-tests-<slug>]` task in `.tasks/STATUS.md`. Otherwise «spike» becomes «skipped tests forever».
-- **Friction is the point.** `[skip-tdd: visual]` 50 раз подряд при `web-design-system` итерации раздражает — это и есть замысел. Friction = fence, not bug. If it becomes unbearable, re-evaluate after ≥2 weeks of usage, not before.
+3. **Friction is the point.** `[skip-tdd: visual]` 50 раз подряд при `web-design-system` итерации раздражает — это и есть замысел. Friction = fence, not bug. If it becomes unbearable, re-evaluate after ≥2 weeks of usage, not before.
+4. **Tests are append-only by default** (added 2026-05-07). New tests: free. **Modifying** an existing assertion, **deleting** a test, or **disabling** it (`it.skip`, `xit`, `@pytest.mark.skip`, `@Disabled`, etc.) requires both:
+
+   **a)** A marker in commit subject:
+   ```
+   [test-modify: <test-name>: was <X>; is <Y>; reason: <Z>]
+   ```
+   Where `<X>` and `<Y>` are the **literal assertion expressions** before and after, not paraphrased. Example:
+   ```
+   [test-modify: validates email format: was expect(isValid("a@b")).toBe(true); is expect(isValid("a@b.com")).toBe(true); reason: tightened spec to require TLD]
+   ```
+
+   **b)** Test changes go in a **separate commit** from any impl changes. A single commit must not modify both `*.test.*` and `src/*` files (or their project-equivalents). This forces an audit-able split — `git log --grep '\[test-modify'` shows every test rewrite cleanly.
+
+   **Why literal `was/is`, not free-form reason:** an agent forced to write the literal assertion publishes exactly what they're rewriting. If `42` was the correct expectation and they changed it to `43` to make a buggy fix pass, the literal `was 42; is 43` line in `git log` identifies the culprit. A reason like «updated to match new behaviour» hides everything — agents will use it whenever it is allowed.
+
+   **Bright-line check** (for an optional pre-commit hook, see follow-up task `tdd-criteria-precommit-hook`):
+   - `git diff --cached` includes a removed `expect(...)` / `assert(...)` / `assertThat(...)` line, OR
+   - changes the arguments of an existing assertion call, OR
+   - adds `.skip`, `xit`, `@skip`, `@Disabled`, etc. annotation, OR
+   - deletes a test file or `it(...)` / `test(...)` / `def test_*` definition
+
+   AND the commit subject does not contain `[test-modify: ...]` matching the format above → block.
+
+   AND `git diff --cached --name-only` includes both test-pattern and impl-pattern files → block (require split).
+
+5. **Don't apply rule 4 retroactively** to tests written before the rule was adopted. The rule applies to test changes made after the project's CLAUDE.md picks up `follow tdd-criteria`. Existing test bodies aren't grandfathered into requiring `was/is` for a one-time rewrite.
 
 ## What's excluded as not bright-line
 
@@ -97,6 +135,7 @@ These tempting formulations were rejected:
 - ~~«When time is short»~~ — time is never abundant.
 - ~~«When the logic is obvious»~~ — it seems obvious until the first bug.
 - ~~«When the code is temporary»~~ — there's no such thing; temporary code becomes permanent.
+- ~~«Tests should not be modified casually»~~ — paraphrasable, agents will modify casually and call it «refactor». Replaced by Anti-loophole rule 4 with literal-evidence requirement.
 
 All require judgment at decision time → automatically become loopholes. The four Permissive categories are bound to **observable properties** (file extension, marker in commit subject, directory, line count) — not mood.
 
@@ -122,6 +161,7 @@ Project `CLAUDE.md` files may **extend** Permissive (e.g. `karu` is a pure-CSS p
 
 - **Supportive evidence is n=1.** The observable contrast is `books` (TDD applied selectively, features ship) vs `modules-db`/`pilonuxt` (TDD not applied, projects buksuyut). `books` could be more productive for reasons unrelated to TDD (codebase maturity, author experience, task type). The direction-of-effect agrees with first principles, so inversion is unlikely, but the magnitude is uncertain.
 - **Friction in UI iterations.** `[skip-tdd: visual]` repeated 50× during a design-system rework is annoying. Intentional. Don't relax before ≥2 weeks of usage.
+- **The literal-`was/is` requirement is verbose** for a renamed test or trivial typo fix. The verbosity is the point — an agent that genuinely fixed a typo writes the same assertion twice with one character changed; an agent that rewrote a failing test writes obviously different assertions. Reading `git log --grep '\[test-modify'` shows the difference at a glance.
 - **«Don't codify at all» was rejected.** Precisely «not codified» is what produced the situation in `books` where, after the author left, his TDD criterion can't be reconstructed — it lived only in his head and his commits. A departed contributor is an argument *for* codification, not *against*.
 - **The anti-vandalism argument is uncomfortable.** It says: «I don't trust agents not to delete my code». That's the position. If it changes, this rationale changes. Until then, this is what's load-bearing.