External CLI: Pinned, Auto-Installed, Drift-Checked, Warn-Gated

Problem

A toolkit that depends on an external CLI tool for a workflow-critical step (e.g. gh for GitHub issue intake, pandoc for doc pipelines, or any git-installed Python/Node CLI) faces a four-way trap:

1. Version drift: different team members install at different times, end up on different upstream releases, observe inconsistent behaviour.
2. Missing-tool silent fallback: workflow degrades to a half-broken path without telling the user — silent quality loss.
3. Trust-the-skill-text contract: a skill says "MUST not proceed without X", but there is no mechanism behind the words.
4. Inconsistent install paths: some users run setup, some install manually, some inherit from a colleague's machine state. No single source of truth.

Each problem alone is annoying. Together they create a class of bugs where "it works on my machine" hides a fragile dependency.

Origin (2026-05-22, SPEC-128): the pattern was extracted from the specify (Spec-Kit) CLI dependency of /vt-d-2-plan. A stakeholder reported that contributors without Spec-Kit installed produced non-uniform specs. That first instance was removed by SPEC-158 (the dependency was never actually scaffolded and the toolkit built stronger, in-repo uniformity machinery). The pattern outlives its first instance — see "Instances" below.

The enforcement level is a proportionality decision

The naive reading of problem #3 is "make the gate a real, blocking exit code". That is correct at scale — many contributors, unreliable installs, a real cost to a diverged artifact. It is wrong for a near-solo toolkit: a hard block on a workflow phase locks the owner out exactly when tooling is already broken (failed auto-install, missing bootstrap tool), which is a strictly worse failure than the near-improbable drift it prevents — especially once auto-install already ships (pillar 2).

So this pattern has two enforcement tiers for the consumer side. The first three pillars are the same in both; pillar 4 is chosen by scale:

• Warn tier (default) — the consumer warns loudly and keeps the fallback. Pin + auto-install + drift-warn already close the uniformity gap for a disciplined solo/small team.
• Hard-gate tier (deferred upgrade) — a blocking hook + opt-out, added only when an explicit trigger fires (team grows to multiple contributors AND auto-install proves unreliable in practice). See "Deferred: Hard-Gate Tier" below.

The mistake to avoid is building the hard-gate tier by default because it feels more robust. Maintenance surface (fork-safety, hook registration — cf. SPEC-150) must be paid for by a real, present threat.

Live companion

The one manifest still exercising this machinery in-repo is configs/security/mcp-package-hashes.yaml (npm-distributed MCP servers, pinned by hash and verified at install). Its git-CLI sibling configs/security/python-tool-versions.yaml is retained as an empty scaffold — its only entry (spec-kit) was removed by SPEC-158 — ready for the next git-installed Python CLI. Both follow the four pillars below; only the distribution semantics differ (git tag vs npm hash), which is why they stay separate files with clean contracts.

Pattern: Four Pillars

1. Manifest Pin (single source of truth)

The version reference for the external tool lives in one file, not embedded in install scripts or docs. The file is the contract; everything else reads from it. Template for a git-installed CLI:

# configs/security/<tool-family>-versions.yaml
tools:
  <tool-name>:
    package: "<pip/npm package name>"
    git_url: "https://github.com/<org>/<repo>.git"
    git_ref: "vX.Y.Z"            # the pinned tag — source of truth
    install_command: 'uv tool install <package> --from "git+<git_url>@<git_ref>"'
    verify_command: 'uv tool list | grep "^<package> " | awk "{print \$2}"'
    pinned_at: "YYYY-MM-DD"
    pinned_by: "SPEC-NNN"

The pin selection is gated by explicit acceptance criteria (e.g. in releases, age ≥14 days, no open P0 issues, installs cleanly, and the command surface the consumers depend on is preserved — verify this across a multi-minor-version jump). The criteria are recorded in the spec's decisions.md so future maintainers can reproduce or revise.

Why a separate manifest rather than baking into setup.sh: - The pin survives setup.sh refactors. - --verify and downstream tools all read from one place. - Update protocol is single-step: edit YAML → re-run setup --update. - Companion to sibling manifests (e.g. mcp-package-hashes.yaml for npm packages) — same pattern, different distribution semantics (git/uv tag vs. npm hash), so a separate file keeps the contracts clean rather than mixing them.

2. Install-Time Auto-Bootstrap with Warn Fallback

The toolkit's --safe / --full install mode auto-installs the CLI from the manifest pin. If the bootstrap tool (e.g. uv) is missing, warn clearly with a manual install hint rather than silently failing.

# scripts/setup.sh — pattern (per-tool)
if command -v <tool> &>/dev/null; then
    ok "<tool> available"
elif command -v uv &>/dev/null; then
    info "<tool> not found — installing via uv..."
    _ref=$(_pinned_ref "<tool-name>") || true   # reads the manifest
    if uv tool install <package> --from "${_ref}" 2>/dev/null; then
        ok "<tool> installed"
    else
        warn "<tool> installation failed — install manually: uv tool install ... ${_ref}"
    fi
else
    warn "uv not found — cannot auto-install. Install uv first: https://docs.astral.sh/uv/"
fi

A helper (a single _pinned_ref() that takes the tool name) extracts the URL once; install sites in different setup modes reuse it. Avoids duplicated literal URLs. Guard the helper against a missing/incomplete manifest — fall back to the unpinned URL with a warning rather than emitting an empty --from.

3. Verify-Time Drift Detection (warn, non-fatal)

setup.sh --verify (the read-only integrity check) compares the installed CLI version against the manifest pin and warns on drift. This catches the slow-drift case where a user installs at toolkit vX (pin = tagA), then weeks later upstream releases tagB, then someone re-installs without bumping the manifest.

Critically, drift is a warning, not an error. --verify conventionally exits non-zero only on hard errors (missing files, broken symlinks), and consumers (CI, /vt-d-0-start) may treat that exit code as fatal. Making drift fatal would flip every user's --verify to non-zero until they re-install — a self-inflicted block of the same shape pillar 4 rejects. So drift increments the warn counter and prints a run setup.sh --update hint, but does not change the exit code.

Verify the exit contract before shipping any new --verify signal. Grep for how the dispatcher maps --verify to an exit code (ERRORS vs WARNINGS) and which consumers treat non-zero as fatal. A new "helpful" non-zero exit can silently break an unrelated gate.

The drift branch is structurally guarded by a test that extracts the do_verify external-dependency slice and asserts the version-comparison is present (per structural-test-assertions). A full heuristic-calibration-gate (a no-fire/fire corpus for the version-extraction heuristic) is deferred with the hard-gate tier — proportional to the current warn scope.

4. Consumer-Side Warn (default tier)

The consumer skill detects the missing prerequisite and emits a loud, unambiguous warning naming the concrete remediation, while retaining the standard fallback. No exit code, no hook.

3. If the required CLI is missing (loud warn, not a hard block):
   - Display: "⚠ <tool> missing — <named quality cost>;
     run `scripts/setup.sh --safe` to auto-install the pinned version"
   - Ask: "Continue with the fallback, or install <tool> first?"
   - If continuing → the fallback path (deliberate soft fallback).

The warn's job is to make the quality cost visible and attributable, not to prevent the user from proceeding. The retained "Continue" offer is the anti-block guarantee — a structural test asserts both the warn wording and that the fallback offer survives, and that no hard-gate mechanism token (PreToolUse, a check-<tool> script, a vt_skip_<tool>_check opt-out) leaked into the skill.

Test prose intent, not prose vocabulary. A grep that forbids the string "hard block" false-fires on documentation that describes the warn-only choice ("this is not a hard block"). Assert on unambiguous mechanism tokens and on the retained fallback offer instead.

Deferred: Hard-Gate Tier (documented upgrade, NOT shipped)

When the trigger fires (multiple contributors AND unreliable installs), promote pillar 4 to a real gate. The mechanism, for whoever builds it:

• PreToolUse hook + thin wrapper: hook fires on Skill invocation; wrapper filters by skill name; a check script exits 0/non-zero; continueOnError: false makes it a hard gate.
• Mandatory opt-out: a one-line escape so a power user is never trapped. Toolkit convention: opt-out keys use vt_<name> at the top level of ~/.claude/settings.json (e.g. vt_skip_<tool>_check: true), with an env-var alternative (VT_SKIP_<TOOL>_CHECK=1) for one-shot bypass.
• Failure-mode UX: stderr lists ALL escape paths verbatim (auto-install, manual install, opt-out) so the user self-rescues without reading docs.
• Calibration HARD GATE: an e2e test exercising happy / block / opt-out / pass-through-other-skills, per heuristic-calibration-gate.

Do not build this until the trigger is real — it re-introduces fork-safety and hook-registration surface (cf. SPEC-150 gotchas) for a check the warn already covers at small scale.

When to Use This Pattern

• An external CLI is required for a toolkit-critical workflow phase.
• The CLI is upstream-managed (we don't fork; we depend).
• A silent fallback when missing is worse than failing visibly.
• Different team members on different machines should converge on the same CLI version for consistent behaviour.

Pillars 1–3 apply whenever these hold. Pillar 4's tier is chosen by team scale (warn by default; hard gate only past the trigger).

When NOT to Use This Pattern

• Truly optional dependencies: don't check a tool the workflow can complete without. Optional tooling gets a warn-only check at most.
• Internal/forked tools: if you vendor the source, the version pin lives in your repo (lockfile, git submodule) — no manifest needed.
• Hard gate on a solo/small toolkit: the block cost (locking the owner out on broken tooling) exceeds the drift it prevents. Warn instead.
• A dependency you can remove: if in-repo machinery can cover the need, dropping the external CLI beats gating it (SPEC-158's resolution for the origin instance).

Anti-Patterns

Anti-pattern	Why it fails
Pin baked into shell script	Bumping the pin requires hunting through scripts; drift goes silent
Mixing git-tag and npm-hash pins in one manifest	Muddies each manifest's contract; use a companion file
Drift check that exits non-zero	Flips --verify fatal on every un-updated machine; may break CI/0-start consumers — warn instead
Hard gate by default on a near-solo toolkit	Blocks the owner when tooling is already broken; maintenance surface ≫ payoff
Helper that emits empty --from on missing manifest	Silent broken install; fall back to unpinned URL + warn
Negative test that greps English prose ("hard block")	False-fires on docs describing the warn choice; assert mechanism tokens
Pin bump without command-surface check across minors	A renamed/removed subcommand silently produces malformed artifacts
Gating a dependency that was never actually needed	The gate warns about a tool nothing consumes; remove the dependency instead (SPEC-158)

Related Patterns

• defensive-toolkit-install — the three-mode dispatch (--safe/--verify/--update) is where pillars 2 and 3 attach. This pattern extends defensive-install with a CLI-pin concern.
• heuristic-calibration-gate — the drift-detection heuristic (pillar 3) and the deferred e2e gate (pillar 4 hard tier) want calibration gates.
• structural-test-assertions — tests against setup.sh install slices and the skill-text warn region extract the region first, then assert membership.
• fork-safe-single-source-stub — the maintenance surface the hard-gate tier would re-introduce; a reason to stay at the warn tier until the trigger fires.
• count-reconciliation-single-source — the single-source-of-truth discipline the manifest pin embodies.

Instances

• SPEC-128 → removed by SPEC-158 (the pattern's origin) — specify (Spec-Kit) CLI for /vt-d-2-plan. Shipped the four pillars at the warn tier (pin manifest, uv auto-install, drift warn, loud consumer warn), then was removed in full by SPEC-158 when the audit found the dependency was never scaffolded and in-repo machinery (plan-checker loop, SPEC-143 diet rules, .plan-gate.md) already delivered stronger, actually-running uniformity. The pattern is retained as the reusable template above; the manifest configs/security/python-tool-versions.yaml survives as an empty scaffold for the next git-installed CLI.
• configs/security/mcp-package-hashes.yaml — the live sibling that still exercises pillars 1–3 for npm-distributed MCP servers (hash pin + install-time verify). Confirms the pattern outlives its first instance.