verification-matrix

name: verification-matrix description: > Choose the smallest correct validation set for a change in this repository. Use before or after edits to map touched files to the repo validation matrix and separate pre-existing failures from newly introduced ones.

Verification Matrix

This skill turns the repo's written validation rules into an explicit workflow.

Use it to choose the smallest correct set of checks instead of running full CI by reflex.

When to use

• before implementing a change, to plan validation
• after implementing a change, to decide which checks are required
• when a task touches multiple subsystems and the correct check set is unclear
• when you need to explain whether a failure is pre-existing or introduced

When NOT to use

• when the user explicitly asks for full CI regardless of touched surface
• as a substitute for targeted browser verification when runtime UI behavior is uncertain
• as a substitute for deeper regression judgment in high-risk engine work

Source of truth

The validation matrix in AGENTS.md is the canonical policy:

• docs, setup, and low-risk config -> npm run typecheck
• UI, hooks, auth, and general TS changes -> npm run lint and npm run typecheck
• engine, parser, or dictionary changes -> npm run test:unit, npm run regression, and npm run regression:gold
• deploy or env-boundary changes -> npm run build

If a change spans multiple categories, run the union of the required checks.

Current package-script relationships:

• npm run ci:check runs check:migrations, check:repo-boundaries, lint, typecheck, test, and build
• npm run test runs test:unit and breadth regression
• npm run regression:gold is separate from npm run test
• npm run test:overlay is separate from npm run test and ci:check

Workflow

1. Start with repo reality

Run:

git status --short

Separate:

• pre-existing edits
• the files touched by the current task
• any pre-existing failures already known in the repo

Do not assume the baseline is clean.

2. Map touched files to validation categories

Use the smallest matching category or union of categories.

Typical mappings in this repo:

• docs and skill files, README.md, CLAUDE.md, AGENTS.md -> docs/setup
• app/, components/, hooks/, most TypeScript modules -> UI/hooks/general TS
• lib/auth/, lib/supabase/, proxy.ts -> auth/general TS
• lib/engine/, lib/parser.ts, lib/dictionaries.ts -> engine/parser/dictionary
• build or deploy contract files, env-validation scripts, deployment boundary files -> deploy/env boundary

When in doubt, choose the narrower valid category first and expand only if the change actually crosses boundaries.

3. Produce the exact command set

Recommend the minimal commands required by the mapped categories.

Examples:

• skill or docs-only change -> npm run typecheck
• dashboard hook refactor -> npm run lint and npm run typecheck
• lib/parser.ts change -> npm run test:unit, npm run regression, and npm run regression:gold
• env-validation or build-boundary change -> npm run build

Mixed-surface changes should use the union, not full CI by default.

4. Interpret failures correctly

When checks fail:

• call out whether the failure is pre-existing or introduced by this task
• avoid claiming unrelated failures as regressions from the current change
• note when the repo baseline is known-dirty and the new work did not add to it

5. Triage failures narrowly

When a check fails, do a focused triage before widening the run:

1. Record the exact command, exit code, and first useful error.
2. Rerun the narrowest reproducer when one exists, such as a single node --test file or the failing regression case.
3. Compare the failure to baseline evidence:
   • pre-existing failure: same failure was already known, or reproduces without the current change surface
   • introduced failure: points at touched files or disappears when the current change is removed
   • unknown: evidence is insufficient; say what would prove it
4. Identify the likely seam: lint/type error, unit contract, regression output, build/env boundary, browser/runtime behavior, or external service.
5. Recommend the smallest next check or fix. Do not turn a narrow failure into full CI unless the failure shows broader blast radius.

6. Escalate only when the change surface justifies it

Do not widen validation because it feels safer.

Run more than the matrix minimum only when:

• the user explicitly asks for broader confidence
• the touched surface has unclear blast radius
• a required check reveals an issue that justifies a broader pass

Output contract

Keep the result compact:

1. Touched surface
2. Validation category mapping
3. Exact commands to run
4. Failure triage or pass/fail interpretation

Non-goals

• replacing business judgment on intentional regression changes
• forcing full CI on doc-only or narrow changes
• pretending the baseline is clean when it has known unrelated failures