Skip to content

Dogfooding Meta-Acceptance

Pattern

When a spec changes the printed output of a workflow command, the command's own next run is a self-verifying acceptance test. Treat the in-flight run that ships the fix as a live acceptance test: assert that the printed output contains the fixed wording, and record the line numbers in the gate notes.

Source-level grep is necessary but not sufficient. The user-facing surface is the rendered output, not the template.

When to Apply

  • Spec edits a TL;DR, status block, decision branch, or any user-visible message in a workflow skill
  • Spec changes conditional output where a branch might be unreachable (GO vs NO-GO TL;DRs, error paths)
  • Spec edits canonical sequence strings, slash-command names, or anything humans pattern-match on
  • Any change where the unit of value is "what the user sees when they run the command"

Why Source-Grep Alone Fails

Failure mode Source grep result Runtime result
Template variable not interpolated passes (literal in source) broken ({{ var }} printed)
Conditional branch unreached passes (string exists) broken (other branch runs)
Stale cached output passes broken (old text still rendered)
Wrong heredoc / indent breaks fence passes broken (rendering eats text)
Override in higher-priority config passes in skill broken (overridden at runtime)

How To Apply

  1. Write source-level assertions for the textual fix (FR-1..FR-N, grep-based regression test). This is the floor.
  2. Add a meta-acceptance criterion in the spec: "The TL;DR printed by the very run that finalizes this spec must contain X before Y."
  3. The reviewer or finalize gate verifies it live. Quote the rendered line numbers from the in-flight run in the gate notes. This is the ceiling — the source could be right and the runtime still wrong.
  4. Record both signals in state.yaml. Source-level verification under build_gate.verification; runtime verification under finalize_gate.notes or review_gate.notes.

Example — SPEC-136

The spec required /vt-c-5-finalize's TL;DR to lead with /vt-c-complete. The acceptance bar was deliberately two-layered:

  • FR-1..FR-5 — grep assertions on the SKILL.md files (test-finalize-points-to-complete.sh). Pass = correct source.
  • Dogfooding meta-acceptance — the finalize run that shipped the fix had to print a TL;DR whose Next: line itself named /vt-c-complete. The finalize_gate notes recorded:

    "Dogfooding meta-acceptance VERIFIED at runtime: the TL;DR printed by this finalize run leads with '/vt-c-complete' (L691/L708/L531 all confirmed)."

If source had been right but the GO TL;DR template had a typo'd variable name, FR-1..FR-5 would have passed and the user would still have seen the old wording. The runtime check closes that gap.

Relationship To Other Patterns

  • Complements three-pass-spec-validation.md (merit / strategic / ground-truth). This pattern adds a fourth pass: runtime self-verification — only applicable when the spec touches printed output.
  • Sister of regression tests: regression tests prevent future drift; meta-acceptance verifies the current ship.

Anti-Patterns

  • ❌ "FR-6 regression test passes, ship it" — when the workflow has multiple unreached output branches the regression test may not exercise.
  • ❌ Asserting via screenshot or transcript paste-back from a prior, pre-fix run — the verification must be live, post-fix.
  • ❌ Skipping the runtime check because "the diff is obviously correct" — that's exactly when conditional/templating bugs slip through.

See Also

  • three-pass-spec-validation.md — broader validation framework
  • SPEC-136 state.yaml finalize_gate.notes — example of how to record the runtime verification