External CLI: Pinned, Auto-Installed, Drift-Checked, Warn-Gated

Problem

A toolkit that depends on an external CLI tool for a workflow-critical step (e.g. specify for /vt-c-2-plan, gh for GitHub issue intake, pandoc for doc pipelines) faces a four-way trap:

1. Version drift: different team members install at different times, end up on different upstream releases, observe inconsistent behaviour.
2. Missing-tool silent fallback: workflow degrades to a half-broken path without telling the user — silent quality loss.
3. Trust-the-skill-text contract: a skill says "MUST not proceed without X", but there is no mechanism behind the words.
4. Inconsistent install paths: some users run setup, some install manually, some inherit from a colleague's machine state. No single source of truth.

Each problem alone is annoying. Together they create a class of bugs where "it works on my machine" hides a fragile dependency.

Observed regression (2026-05-22, SPEC-128 origin): DaWiVisi reported: "Users who have not installed Spec-Kit go through the standard procedure in the planning process without Spec-Kit. The framework/workflow has Spec-Kit anchored at its core ... [it] is essential, especially in the planning phase, in order to obtain reasonable and uniform specifications."

The enforcement level is a proportionality decision

The naive reading of problem #3 is "make the gate a real, blocking exit code". That is correct at scale — many contributors, unreliable installs, a real cost to a diverged spec. It is wrong for a near-solo toolkit: a hard block on the planning phase locks the owner out exactly when tooling is already broken (failed auto-install, missing uv), which is a strictly worse failure than the near-improbable drift it prevents — especially once auto-install already ships (pillar 2).

So this pattern has two enforcement tiers for the consumer side. The first three pillars are the same in both; pillar 4 is chosen by scale:

• Warn tier (default, current SPEC-128 shipping) — the consumer warns loudly and keeps the fallback. Pin + auto-install + drift-warn already close the uniformity gap for a disciplined solo/small team.
• Hard-gate tier (deferred upgrade) — a blocking hook + opt-out, added only when an explicit trigger fires (team grows to multiple contributors AND auto-install proves unreliable in practice). See "Deferred: Hard-Gate Tier" below.

The mistake to avoid is building the hard-gate tier by default because it feels more robust. Maintenance surface (fork-safety, hook registration — cf. SPEC-150) must be paid for by a real, present threat.

Pattern: Four Pillars

1. Manifest Pin (single source of truth)

The version reference for the external tool lives in one file, not embedded in install scripts or docs. The file is the contract; everything else reads from it.

# configs/security/python-tool-versions.yaml — SPEC-128 instance
tools:
  spec-kit:
    package: "specify-cli"
    git_url: "https://github.com/github/spec-kit.git"
    git_ref: "v0.11.5"
    install_command: 'uv tool install specify-cli --from "git+...@v0.11.5"'
    verify_command: 'uv tool list | grep "^specify-cli " | awk "{print \$2}"'
    pinned_at: "2026-07-06"
    pinned_by: "SPEC-128"

The pin selection is gated by explicit acceptance criteria (e.g. in releases, age ≥14 days, no open P0 issues, installs cleanly, and the command surface the consumers depend on is preserved — verify this across a multi-minor-version jump). The criteria are recorded in the spec's decisions.md so future maintainers can reproduce or revise.

Why a separate manifest rather than baking into setup.sh: - The pin survives setup.sh refactors. - --verify and downstream tools all read from one place. - Update protocol is single-step: edit YAML → re-run setup --update. - Companion to existing manifests (e.g. mcp-package-hashes.yaml for npm packages) — same pattern, different distribution semantics (git/uv tag vs. npm hash), so a separate file keeps the contracts clean rather than mixing them.

2. Install-Time Auto-Bootstrap with Warn Fallback

The toolkit's --safe / --full install mode auto-installs the CLI from the manifest pin. If the bootstrap tool (uv here) is missing, warn clearly with a manual install hint rather than silently failing.

# scripts/setup.sh — pattern
if command -v specify &>/dev/null; then
    ok "SpecKit CLI available"
elif command -v uv &>/dev/null; then
    info "SpecKit CLI not found — installing via uv..."
    _sk_ref=$(_speckit_pinned_ref) || true   # reads manifest
    if uv tool install specify-cli --from "${_sk_ref}" 2>/dev/null; then
        ok "SpecKit CLI installed"
    else
        warn "SpecKit CLI installation failed — install manually: uv tool install ... ${_sk_ref}"
    fi
else
    warn "uv not found — cannot auto-install. Install uv first: https://docs.astral.sh/uv/"
fi

A helper (_speckit_pinned_ref()) extracts the URL once; install sites in different setup modes reuse it. Avoids duplicated literal URLs. Guard the helper against a missing/incomplete manifest — fall back to the unpinned URL with a warning rather than emitting an empty --from.

3. Verify-Time Drift Detection (warn, non-fatal)

setup.sh --verify (the read-only integrity check) compares the installed CLI version against the manifest pin and warns on drift. This catches the slow-drift case where a user installs at toolkit vX (pin = tagA), then weeks later upstream releases tagB, then someone re-installs without bumping the manifest.

Critically, drift is a warning, not an error. --verify conventionally exits non-zero only on hard errors (missing files, broken symlinks), and consumers (CI, /vt-c-0-start) may treat that exit code as fatal. Making drift fatal would flip every user's --verify to non-zero until they re-install — a self-inflicted block of the same shape pillar 4 rejects. So drift increments the warn counter and prints a run setup.sh --update hint, but does not change the exit code.

Verify the exit contract before shipping any new --verify signal. Grep for how the dispatcher maps --verify to an exit code (ERRORS vs WARNINGS) and which consumers treat non-zero as fatal. A new "helpful" non-zero exit can silently break an unrelated gate.

The drift branch is structurally guarded by a test that extracts the do_verify external-dependency slice and asserts the version-comparison is present (per structural-test-assertions). A full heuristic-calibration-gate (a no-fire/fire corpus for the version-extraction heuristic) is deferred with the hard-gate tier — proportional to the current warn scope.

4. Consumer-Side Warn (default tier)

The consumer skill (/vt-c-2-plan) detects the missing prerequisite and emits a loud, unambiguous warning naming the concrete remediation, while retaining the standard-planning fallback. No exit code, no hook.

3. If NO `.specify/` directory (loud warn, not a hard block):
   - Display: "⚠ SpecKit missing — specs may drift from the uniform team
     structure; run `scripts/setup.sh --full` ... or `specify init . --ai claude`"
   - Ask: "Continue with standard planning, or set up SpecKit first?"
   - If continuing → standard planning (deliberate soft fallback).

The warn's job is to make the quality cost visible and attributable, not to prevent the user from proceeding. The retained "Continue with standard planning" offer is the anti-block guarantee — a structural test asserts both the warn wording and that the fallback offer survives, and that no hard-gate mechanism token (PreToolUse, check-speckit, vt_skip_speckit_check) leaked into the skill.

Test prose intent, not prose vocabulary. A grep that forbids the string "hard block" false-fires on documentation that describes the warn-only choice ("this is not a hard block"). Assert on unambiguous mechanism tokens and on the retained fallback offer instead.

Deferred: Hard-Gate Tier (documented upgrade, NOT shipped)

When the trigger fires (multiple contributors AND unreliable installs), promote pillar 4 to a real gate. The mechanism, for whoever builds it:

• PreToolUse hook + thin wrapper: hook fires on Skill invocation; wrapper filters by skill name; a check script exits 0/non-zero; continueOnError: false makes it a hard gate.
• Mandatory opt-out: a one-line escape so a power user is never trapped. Toolkit convention: opt-out keys use vt_<name> at the top level of ~/.claude/settings.json (e.g. vt_skip_speckit_check: true), with an env-var alternative (VT_SKIP_SPECKIT_CHECK=1) for one-shot bypass.
• Failure-mode UX: stderr lists ALL escape paths verbatim (auto-install, manual install, opt-out) so the user self-rescues without reading docs.
• Calibration HARD GATE: an e2e test exercising happy / block / opt-out / pass-through-other-skills, per heuristic-calibration-gate.

Do not build this until the trigger is real — it re-introduces fork-safety and hook-registration surface (cf. SPEC-150 gotchas) for a check the warn already covers at small scale.

When to Use This Pattern

• An external CLI is required for a toolkit-critical workflow phase.
• The CLI is upstream-managed (we don't fork; we depend).
• A silent fallback when missing is worse than failing visibly.
• Different team members on different machines should converge on the same CLI version for consistent behaviour.

Pillars 1–3 apply whenever these hold. Pillar 4's tier is chosen by team scale (warn by default; hard gate only past the trigger).

When NOT to Use This Pattern

• Truly optional dependencies: don't check a tool the workflow can complete without. Optional tooling gets a warn-only check at most.
• Internal/forked tools: if you vendor the source, the version pin lives in your repo (lockfile, git submodule) — no manifest needed.
• Hard gate on a solo/small toolkit: the block cost (locking the owner out on broken tooling) exceeds the drift it prevents. Warn instead.

Anti-Patterns

Anti-pattern	Why it fails
Pin baked into shell script	Bumping the pin requires hunting through scripts; drift goes silent
Mixing git-tag and npm-hash pins in one manifest	Muddies each manifest's contract; use a companion file
Drift check that exits non-zero	Flips --verify fatal on every un-updated machine; may break CI/0-start consumers — warn instead
Hard gate by default on a near-solo toolkit	Blocks the owner when tooling is already broken; maintenance surface ≫ payoff
Helper that emits empty --from on missing manifest	Silent broken install; fall back to unpinned URL + warn
Negative test that greps English prose ("hard block")	False-fires on docs describing the warn choice; assert mechanism tokens
Pin bump without command-surface check across minors	A renamed/removed subcommand silently produces malformed artifacts

Related Patterns

• defensive-toolkit-install — the three-mode dispatch (--safe/--verify/--update) is where pillars 2 and 3 attach. This pattern extends defensive-install with a CLI-pin concern.
• heuristic-calibration-gate — the drift-detection heuristic (pillar 3) and the deferred e2e gate (pillar 4 hard tier) want calibration gates.
• structural-test-assertions — tests against setup.sh install slices and the skill-text warn region extract the region first, then assert membership.
• fork-safe-single-source-stub — the maintenance surface the hard-gate tier would re-introduce; a reason to stay at the warn tier until the trigger fires.

Instances

• SPEC-128 (this pattern's origin, re-shaped to warn 2026-07-06) — specify CLI for /vt-c-2-plan. Shipped: pin manifest configs/security/python-tool-versions.yaml (v0.11.5); _speckit_pinned_ref helper + pinned install in do_full/do_verify; drift warn in do_verify; loud warn (not block) in workflow-2-plan/SKILL.md Step 0; prereq docs across README/installation/CLAUDE. Tests: scripts/tests/setup/12–16-*.sh, scripts/tests/docs/01-*.sh. Deferred (not shipped): check-speckit.sh, the PreToolUse hook, the vt_skip_speckit_check opt-out, and the e2e/hook calibration gates — the hard-gate tier above, gated on the team-growth trigger.