Dogfooding Meta-Acceptance¶
Pattern¶
When a spec changes the printed output of a workflow command, the command's own next run is a self-verifying acceptance test. Treat the in-flight run that ships the fix as a live acceptance test: assert that the printed output contains the fixed wording, and record the line numbers in the gate notes.
Source-level grep is necessary but not sufficient. The user-facing surface is the rendered output, not the template.
When to Apply¶
- Spec edits a TL;DR, status block, decision branch, or any user-visible message in a workflow skill
- Spec changes conditional output where a branch might be unreachable (GO vs NO-GO TL;DRs, error paths)
- Spec edits canonical sequence strings, slash-command names, or anything humans pattern-match on
- Any change where the unit of value is "what the user sees when they run the command"
Why Source-Grep Alone Fails¶
| Failure mode | Source grep result | Runtime result |
|---|---|---|
| Template variable not interpolated | passes (literal in source) | broken ({{ var }} printed) |
| Conditional branch unreached | passes (string exists) | broken (other branch runs) |
| Stale cached output | passes | broken (old text still rendered) |
| Wrong heredoc / indent breaks fence | passes | broken (rendering eats text) |
| Override in higher-priority config | passes in skill | broken (overridden at runtime) |
How To Apply¶
- Write source-level assertions for the textual fix (FR-1..FR-N, grep-based regression test). This is the floor.
- Add a meta-acceptance criterion in the spec: "The TL;DR printed by the very run that finalizes this spec must contain X before Y."
- The reviewer or finalize gate verifies it live. Quote the rendered line numbers from the in-flight run in the gate notes. This is the ceiling — the source could be right and the runtime still wrong.
- Record both signals in
state.yaml. Source-level verification underbuild_gate.verification; runtime verification underfinalize_gate.notesorreview_gate.notes.
Example — SPEC-136¶
The spec required /vt-c-5-finalize's TL;DR to lead with /vt-c-complete. The acceptance bar was deliberately two-layered:
- FR-1..FR-5 — grep assertions on the SKILL.md files (
test-finalize-points-to-complete.sh). Pass = correct source. - Dogfooding meta-acceptance — the finalize run that shipped the fix had to print a TL;DR whose
Next:line itself named/vt-c-complete. The finalize_gate notes recorded:"Dogfooding meta-acceptance VERIFIED at runtime: the TL;DR printed by this finalize run leads with '/vt-c-complete' (L691/L708/L531 all confirmed)."
If source had been right but the GO TL;DR template had a typo'd variable name, FR-1..FR-5 would have passed and the user would still have seen the old wording. The runtime check closes that gap.
Relationship To Other Patterns¶
- Complements
three-pass-spec-validation.md(merit / strategic / ground-truth). This pattern adds a fourth pass: runtime self-verification — only applicable when the spec touches printed output. - Sister of regression tests: regression tests prevent future drift; meta-acceptance verifies the current ship.
Anti-Patterns¶
- ❌ "FR-6 regression test passes, ship it" — when the workflow has multiple unreached output branches the regression test may not exercise.
- ❌ Asserting via screenshot or transcript paste-back from a prior, pre-fix run — the verification must be live, post-fix.
- ❌ Skipping the runtime check because "the diff is obviously correct" — that's exactly when conditional/templating bugs slip through.
See Also¶
three-pass-spec-validation.md— broader validation framework- SPEC-136
state.yamlfinalize_gate.notes— example of how to record the runtime verification