qa-test

name: qa-test description: "CodexBar live QA/e2e testing: run provider usage matrix checks, validate real app config, use Peekaboo for menu proof, use Browser Use/official docs for API spec or logged-in dashboard checks, and handle 1Password credentials safely."

CodexBar Live QA

Use for live provider testing, release smoke tests, menu verification, or debugging “provider works/fails” reports.

Rules

• Work from the CodexBar repo checkout.
• Use the packaged CLI first: CodexBar.app/Contents/Helpers/CodexBarCLI.
• Do not use CodexBar.app/Contents/MacOS/codexbar; that is the app binary and may appear to hang as a CLI.
• Never run broad env, set, or secret regex dumps.
• Use $one-password for secrets: all op commands inside one persistent tmux session, service account first, no raw secret output.
• Treat browser-cookie/keychain flows as prompt-risky. Prefer CLI/API-token checks and KeychainNoUIQuery-safe tests unless the user explicitly requested live UI.
• For current API behavior, browse official provider docs only.

CLI Matrix

Run the bundled script:

.agents/skills/qa-test/scripts/live_provider_matrix.sh --enabled

Useful modes:

.agents/skills/qa-test/scripts/live_provider_matrix.sh --provider all
.agents/skills/qa-test/scripts/live_provider_matrix.sh --providers openai,zai,deepseek
.agents/skills/qa-test/scripts/live_provider_matrix.sh --default

Interpretation:

• --enabled asks CodexBarCLI config providers for enabled providers, honoring CODEXBAR_CONFIG and default toggles.
• --default runs the app-facing default command with no provider override.
• --provider all forces every registered provider and is expected to fail for providers without sessions/keys.
• A green app config needs --enabled and --default clean; --provider all is a discovery/triage tool.

Config QA

Validate config:

CodexBar.app/Contents/Helpers/CodexBarCLI config validate
stat -f '%Lp %N' "$HOME/.codexbar/config.json"

Redact config shape:

jq '(.providers // []) |= map(.apiKey = (if .apiKey then "<redacted>" else .apiKey end) |
  .secretKey = (if .secretKey then "<redacted>" else .secretKey end) |
  .cookieHeader = (if .cookieHeader then "<redacted>" else .cookieHeader end) |
  (if .id == "stepfun" and has("region") then .region = "<redacted>" else . end) |
  .tokenAccounts = (if .tokenAccounts then (.tokenAccounts | .accounts = (.accounts | map(.token = "<redacted>"))) else .tokenAccounts end))' \
  "$HOME/.codexbar/config.json"

Before editing config, make a backup:

cp "$HOME/.codexbar/config.json" "$HOME/.codexbar/config.pre-qa-$(date +%Y%m%d%H%M%S).json"
chmod 600 "$HOME/.codexbar"/config.pre-qa-*.json

Live Menu QA

Use Peekaboo after CLI checks:

pkill -x CodexBar || pkill -f 'CodexBar.app/Contents/MacOS/CodexBar' || true
open -n "$PWD/CodexBar.app"
peekaboo menu list-all --json | rg -i 'codexbar'
peekaboo menu click-extra --title codexbar-merged --json
screencapture -x /tmp/codexbar-live-menu.png

Crop top-right menu if needed:

sips --cropToHeightWidth 900 340 --cropOffset 20 2650 /tmp/codexbar-live-menu.png \
  --out /tmp/codexbar-live-menu-crop.png >/dev/null

Verify visually with view_image. Confirm provider tabs/rows match enabled config and no failing provider dominates the first screen.

Browser Use

Use $browser-use only when a logged-in dashboard, API key page, or provider docs need browser/profile state.

Existing Chrome path:

mcporter call chrome-devtools.list_pages --args '{}' --output text
mcporter call chrome-devtools.navigate_page --args '{"url":"https://provider.example"}' --output text
mcporter call chrome-devtools.take_snapshot --args '{}' --output text

If Browser Use is unavailable, say so and use web search for public official docs; do not substitute isolated Playwright for login/profile-dependent pages.

Fix Triage

• Missing auth/session: configure key/session if available; otherwise leave provider disabled or report blocked auth.
• Wrong provider API/spec: inspect official docs, then patch fetcher/settings/tests.
• Provider key exists but live API rejects it: keep key stored if useful, disable provider if the menu would show a persistent error.
• User-facing behavior changes need CHANGELOG.md.
• Code fixes need focused tests, make check, $autoreview, and live CLI proof before landing.

Known CodexBar QA Notes

• OpenAI Admin API key is the useful usage provider key. Project OPENAI_API_KEY values can fail legacy credit-balance fallback with 403.
• Deepgram usage requires a key/project with Management API permissions; transcription-only keys can return 403.
• Groq usage uses the Prometheus metrics API, not ordinary inference endpoints.
• MiniMax pay-as-you-go API keys and Token Plan/Coding Plan keys are different; wrong key kind can leave usage unavailable.