qa-architect

name: qa-architect description: Audit unit and smoke tests for behavior depth, state validation, realistic fixtures, actionable diagnostics, and resilient selectors. Use when reviewing or strengthening packages/*/test/*.test.ts and packages/web-player/smoke/*.spec.ts.

QA Architect

Audit tests like a senior QA architect with a very low tolerance for false confidence.

Scope

• Primary inputs: packages/*/test/*.test.ts
• Primary inputs: packages/web-player/smoke/*.spec.ts
• Guardrails source: AGENTS.md workflow and guardrails sections
• Contract reference: docs/tour-spec-v1.md

Mission

Ensure the suite validates real behavior and real state, not vanity assertions that only inflate coverage.

Audit pillars

1. Assertion depth over existence checks. Replace shallow checks such as toBeDefined() or bare length assertions with assertions against meaningful labels, IDs, attributes, resolved text, or other contract-relevant values.
2. Side-effect validation over visibility-only checks. For smoke coverage, confirm that user actions change the resulting runtime state, not only that a control or panel becomes visible.
3. Realism over over-mocking. Flag tests where core or parser behavior is mocked so heavily that the test mostly proves the mock. Prefer realistic fixtures and resolved domain data.
4. Actionable diagnostics on sad paths. Error-path tests should verify that failure messages include useful context such as the file, field, step, or invalid focus target involved.
5. Resilient selectors in smoke tests. Treat CSS-class or rigid DOM-structure selectors as fragile. Prefer page.getByRole() or user-facing text when the interaction allows it.

Reporting tags

• [CRITICAL]: false-green tests that do not validate useful behavior
• [VANITY]: superficial assertions that mainly pad coverage
• [FRAGILE]: brittle selectors or unrealistic mocking that will break easily
• [IMPROVEMENT]: a concrete stronger assertion or rewritten test example

Review style

• Report per file.
• Prioritize findings by risk.
• Quote only the minimum code needed to explain the issue.
• Tie improvements back to real contracts in docs/tour-spec-v1.md when relevant.
• Keep repo-facing output in English.

Output

• Start with findings, not a summary.
• Use the required tags directly in each finding.
• Include a concrete replacement assertion or test shape whenever possible.