Skip to content

Commit the Evidence Before the Review Head

Problem

Some specs require a measurement that the review grades, for example SPEC-223's SC-3: the full scripts/run-tests.sh on the merge base and on the branch, with no branch-only red. The measurement runs at the fix head X. Its logs and its summary go into the spec directory, and committing them makes a new head X+1.

In SPEC-223 the order was: fix round at e9250d950, SC-3 run at that head, review pass 6 graded e9250d950, and only then the SC-3 logs were committed (4ae067b7b). Finalize's record-only exception (workflow-5-finalize/SKILL.md, Step 0 item 4b) admits exactly seven paths after the gate head: the spec directory's review-triage.md, state.yaml, .review-gate.md, .claude-checkpoint.md and metrics.json, plus docs/review-history.md and the root .test-gate.md. sc3-runs.md and sc3-logs/ are not among them, so finalize blocked. Its documented way out is to review again, and pass 7 was a review of nothing but the records.

Solution

Put the evidence inside the tree the review grades:

  1. Fix round, committed at X.
  2. Run the measurement at X.
  3. Commit the evidence as X+1. Change nothing else in this commit.
  4. Review X+1. The reviewers see the logs and can check each figure against its log line.
  5. The review commits its own record (X+2), which touches only record-only paths.
  6. Finalize at X+2. The record-only check passes.

The measurement still describes X, while the gate head is X+1. A carry-over rule closes that gap: the measurement holds for a later head when only its own evidence files changed. SPEC-223 states such a rule at the top of sc3-runs.md, but it exempts every file under the spec directory, and pass 7 disputed it for that reason (SPEC-95). Name the evidence files in the rule, and let the review check that X..X+1 holds nothing else.

Why not widen the record-only list

Widening the list is an owner decision, and evidence files are not inert. SPEC-223 pass 7 (SPEC-95) found that records in the spec directory are inputs to guards 54 and 126 and to two plugin suites, so a change to them after the review can change what the suite measures.

Prevention

  • Before dispatching /vt-d-4-review, git status --porcelain is empty for the spec directory, and the evidence commit is HEAD or an ancestor of it.
  • Between the review and finalize, commit only paths on the record-only list. Check with the filter from Step 0 item 4b before committing: git diff --no-renames --no-relative --name-only <gate-head>..HEAD.
  • If evidence has to change after the review, plan for another review pass. The delta is small, but the pass is still needed.

Related: docs/solutions/patterns/v025-multi-worktree-workflow.md.