Skip to content

Testing Procedures

Canonical reference for test frameworks, TDD workflow, quality gates, and the test-runner agent.

Testing in the toolkit follows a TDD-first, gate-enforced model. Tests are written before code (Phase 3), verified automatically before review (Phase 4), and checked again before deployment (Phase 5). The .test-gate.md file connects these phases.

Supported Frameworks

The test-runner agent auto-detects the project's test framework from configuration files.

Framework Detection File Targeted Command Full Suite Command
Jest jest.config.* or package.json devDeps npx jest path/to/test.test.ts npm test
Vitest vitest.config.* or package.json devDeps npx vitest run path/to/test.test.ts npx vitest run
Angular + Vitest angular.json + vitest.config.* npx vitest run path/to/test.spec.ts npx vitest run
pytest pytest.ini, pyproject.toml, setup.cfg python -m pytest path/to/test_file.py -v python -m pytest -v
RSpec Gemfile with rspec bundle exec rspec spec/path/to/spec.rb bundle exec rspec
Go test go.mod go test ./path/to/package/ -run TestName -v go test ./... -v
Cargo test Cargo.toml cargo test test_name cargo test
Bun test bunfig.toml or bun.lockb bun test path/to/test.test.ts bun test
Deno test deno.json or deno.jsonc deno test path/to/test.ts deno test
Playwright playwright.config.* npx playwright test path/to/test.spec.ts npx playwright test

If no framework is detected, the test-runner reports "No test framework detected" and lists the files it checked.

Test Runner Modes

The test-runner agent operates in three modes. It runs in an isolated context (context: fork) and returns a structured report without modifying any files.

Targeted Mode (TDD Inner Loop)

Runs a single test file or pattern. Optimized for fast feedback (< 30s timeout). Use during Phase 3 to verify RED and GREEN steps.

Invoke: Dispatch test-runner with specific test file path
When: After writing a new test (RED) or implementing code (GREEN)

Full Suite Mode (Pre-Review Gate)

Runs the entire test suite including browser tests for web projects. Generates .test-gate.md — the quality gate file read by Phase 4 and Phase 5.

Invoke: Dispatch test-runner in full-suite mode
When: Before running /vt-d-4-review

E2E Mode (Playwright Browser Tests)

Runs browser-based end-to-end tests using Playwright. Automatically included in Full Suite mode for web projects. Can also run standalone.

Requires a dev server. The agent checks for scripts/with_server.py to auto-start it, or reports that manual startup is needed.

Web Project Detection

Playwright e2e tests are mandatory in Full Suite mode when any of these indicators match:

Indicator Meaning
playwright.config.{ts,js} Explicit Playwright setup
react, vue, svelte, angular, next, nuxt, remix in package.json Frontend framework
app/views/ directory Server-rendered templates
vite.config.*, webpack.config.* Frontend build tooling
src/**/*.{jsx,tsx,vue,svelte} files Frontend component files

Non-web projects (backend APIs, CLI tools, libraries) skip browser tests.

TDD Cycle

Phase 3 enforces this cycle for every feature:

RED ──→ VERIFY ──→ GREEN ──→ VERIFY ──→ REFACTOR ──→ COMMIT
Step Action Test Runner Mode
RED Write a failing test for the new behavior —
VERIFY Run the test — confirm it fails for the right reason (not syntax/import errors) Targeted
GREEN Write the minimal code to make the test pass —
VERIFY Run the test — confirm it passes and no regressions Targeted
REFACTOR Clean up code while keeping tests green —
COMMIT Commit the passing test and implementation together —

Tests before code

Write the test first. If the test passes without new code, it is not testing new behavior. The test-runner flags this: "Test passes without production code — this test may not be testing new behavior."

The Test Gate

.test-gate.md is the bridge between testing (Phase 3) and review/finalize (Phases 4–5).

How it works

  1. Run the test-runner in Full Suite mode — it writes .test-gate.md to the project root
  2. Phase 4 (/vt-d-4-review) reads the gate file to verify tests passed before dispatching reviewers
  3. Phase 5 (/vt-d-5-finalize) reads the gate file as a finalization prerequisite

Gate file format

---
status: PASS          # or FAIL
date: 2026-02-15T10:30:00Z
branch: feature/add-auth
framework: vitest
mode: full-suite
total: 47
passed: 47
failed: 0
skipped: 2
duration: 12.3s
browser_tests: true   # or false or N/A
browser_passed: 8     # or N/A
browser_console_errors: 0  # or N/A
coverage: 84%         # or "not measured"
---

Gate states

State Effect on Phase 4 Effect on Phase 5
status: PASS Reviewers dispatched normally Deployment proceeds
status: FAIL Review blocked — fix tests first Deployment blocked
File missing Review blocked — run full suite first Deployment blocked
Wrong branch Stale gate — re-run full suite on current branch Deployment blocked

Note

.test-gate.md is a local workflow artifact. Add it to .gitignore — it should not be committed to version control.

Test Organization

Recommended conventions by framework:

Framework Test Location Naming Pattern
Jest / Vitest __tests__/ or co-located *.test.ts, *.spec.ts
Angular + Vitest co-located with component *.spec.ts
pytest tests/ test_*.py
RSpec spec/ *_spec.rb
Go Same package *_test.go
Cargo Same file or tests/ #[cfg(test)] module
Playwright tests/e2e/ or e2e/ *.spec.ts

Manual and Acceptance Testing

Not all verification can be automated. The test plan template (plugins/vt-product-dev/skills/workflow-4-review/assets/test-plan-template.md) tracks both automated and manual tests.

  • Automated tests — status populated from .test-gate.md during review
  • Manual tests — added during planning (/vt-d-2-plan) or development (/vt-d-3-build), verified by humans during review

Manual and acceptance cases follow one case standard: the ISTQB core of ISO/IEC/IEEE 29119-3 plus a Given/When/Then expectation, with every case within the standard's length budget and explicit enough that an AI agent can run it without asking back. Write cases from a requirement with /vt-d-test-case-design author <source> and review existing cases, in any format, with /vt-d-test-case-design review <paths>. The review is read-only: it returns findings, a verdict per case and rewritten drafts, and the case owner decides what to apply.