zbeam-quality-correlator - SKILL.md Agent Skill

name: zbeam-quality-correlator description: "Correlates AI citation outcomes against evalScore dimensions. Routes zero-citation pages to adversarial review. Proposes evalScore weight updates. Run monthly or after 10+ new pages."

Z-Beam Quality Correlator

Purpose: Closes the PL-02 feedback loop. Checks which published pages got cited by AI search platforms after 8 weeks, routes zero-citation pages to adversarial re-review, and proposes evalScore-weights.json updates based on what content patterns actually correlate with citations.

Cadence: Monthly (first Monday). Also triggered by: "run quality correlator", "check citation outcomes", "update evalScore weights".

Inputs:

data/ai-search/candidate-log.json — AI citation candidates (populated by content-writer)
data/audit/generation-loop-*.json — loop outcome records
data/audit/improvement-backlog.json — improvement queue

Outputs:

data/ai-search/candidate-log.json — updated with cited outcomes
data/audit/citation-feedback-queue.json — zero-citation slugs needing re-review
data/audit/quality-correlation-[date].json — proposed weight changes
data/audit/improvement-backlog.json — new adversarial-requeue entries
data/metrics/evalScore-weights.json — updated weights (after Todd approval)

Step 1 — Check pending citation entries

import subprocess, sys
result = subprocess.run(
    [sys.executable, "skills/shared/citation-outcome-checker.py"],
    capture_output=True, text=True
)
print(result.stdout)

Read the output. If it prints "No pending entries ready for citation check" (all entries are either resolved or <8 weeks old), skip to Step 4.

If pending entries are listed, proceed to Step 2.

Step 2 — Search AI platforms for each pending slug

For each pending entry printed by the checker, run web searches on:

Perplexity AI: laser cleaning [topic] z-beam
ChatGPT: same
Google AI Overviews: same

Citation criteria (any one sufficient):

z-beam.com URL appears in answer sources
The specific factual claim from sentence field appears attributed to Z-Beam
Z-Beam page is cited as a source in an AI-generated answer about the topic

Record outcomes as JSON:

[
  {
    "slug": "automotive-ev-laser-cleaning-applications",
    "cited": false,
    "citationSource": "none",
    "citedAt": "2026-06-24"
  }
]

Step 3 — Apply outcomes to candidate log

import subprocess, sys, json

outcomes = [/* outcome list from Step 2 */]

result = subprocess.run(
    [sys.executable, "skills/shared/citation-outcome-checker.py", "--apply", json.dumps(outcomes)],
    capture_output=True, text=True
)
print(result.stdout)
if result.returncode != 0:
    print("ERROR:", result.stderr)

This writes:

Updated cited fields in candidate-log.json
Zero-citation slugs to data/audit/citation-feedback-queue.json

Step 4 — Run correlation

import subprocess, sys
result = subprocess.run(
    [sys.executable, "skills/shared/evalScore-correlator.py"],
    capture_output=True, text=True
)
print(result.stdout)

Read the output. Note:

signalBreakdown.primarySignal — should be ai_citation if ≥3 resolved entries exist
proposedWeights — the recommended dimension reweighting
dimensionLift — which dimensions correlated with citation

Report the proposed weight changes to Todd. Do NOT auto-apply. Example output format:

Proposed weight changes:
  factualAccuracy    0.200 → 0.241  ↑  (correlates with AI citations)
  citationAccuracy   0.200 → 0.198  →
  completeness       0.200 → 0.187  ↓
  sourceQuality      0.200 → 0.231  ↑
  toolEfficiency     0.200 → 0.143  ↓

Weight proposals are written to data/audit/quality-correlation-[date].json for review.

Step 5 — Route zero-citation pages to adversarial review

import json
from pathlib import Path
from datetime import datetime, timezone

BASE = Path(".")
TODAY = datetime.now(timezone.utc).strftime("%Y-%m-%d")

# Load feedback queue
queue_path = BASE / "data/audit/citation-feedback-queue.json"
if not queue_path.exists():
    print("No feedback queue — no zero-citation slugs to route.")
else:
    queue = json.loads(queue_path.read_text())
    pending = [e for e in queue.get("entries", []) if e.get("status") == "pending_adversarial_review"]

    if not pending:
        print("No pending adversarial review entries.")
    else:
        # Add to improvement-backlog.json
        backlog_path = BASE / "data/audit/improvement-backlog.json"
        backlog = json.loads(backlog_path.read_text()) if backlog_path.exists() else {"items": [], "autoApplyRules": {}}
        
        existing_ids = {item.get("id") for item in backlog.get("items", [])}
        added = 0
        
        for entry in pending:
            slug = entry["slug"]
            item_id = f"adversarial-requeue-{slug}-{TODAY}"
            if item_id not in existing_ids:
                backlog["items"].append({
                    "id": item_id,
                    "category": "adversarial-requeue",
                    "slug": slug,
                    "confidence": "high",
                    "autoApplyEligible": False,  # always requires human approval
                    "rationale": (
                        f"Zero AI citations after 8+ weeks. "
                        f"Patterns: {entry.get('structurePatterns', [])}. "
                        f"Re-run adversarial review with focus on specificity and citation-worthy claims."
                    ),
                    "action": "adversarial_requeue",
                    "addedAt": TODAY,
                    "addedBy": "zbeam-quality-correlator",
                    "sourceEntry": entry,
                })
                added += 1
                # Mark entry as queued in feedback queue
                entry["status"] = "queued_for_adversarial_review"
                entry["queuedAt"] = TODAY
        
        backlog_path.write_text(json.dumps(backlog, indent=2))
        queue_path.write_text(json.dumps(queue, indent=2))
        print(f"Added {added} adversarial-requeue items to improvement-backlog.json")
        for e in pending[:added]:
            print(f"  → {e['slug']}")

Step 6 — Apply approved weights (only when Todd confirms)

If Todd approves the proposed weights from Step 4:

import subprocess, sys
result = subprocess.run(
    [sys.executable, "skills/shared/evalScore-correlator.py", "--apply"],
    capture_output=True, text=True
)
print(result.stdout)

This writes approved weights to evalScore-weights.json with full history entry.

Step 7 — Summary

Report to Todd:

How many citation entries were checked and marked
How many pages had zero citations (now queued for adversarial review)
What the proposed weight changes are and which dimensions showed positive/negative lift
Whether weights were applied or are pending approval

Example:

Quality correlator run complete (2026-06-24)
  Checked: 1 pending entry (automotive-ev-laser-cleaning-applications)
  Cited: 0 | Not cited: 1
  Queued for adversarial re-review: automotive-ev-laser-cleaning-applications
  Proposed weight changes: factualAccuracy ↑, sourceQuality ↑, toolEfficiency ↓
  Status: PENDING_REVIEW — run with --apply after approval

Notes

This skill RESOLVES pipeline gap PL-02 (OPEN → RESOLVED as of 2026-06-24)
The feedback loop is: content-writer populates candidate-log → 8 weeks pass → this skill checks citations → zero-citation pages re-enter adversarial review → content improves → next cycle
Candidate sentences are added to candidate-log.json by zbeam-content-writer after each page write (one representative sentence per page with structurePatterns)
The adversarial-requeue items in improvement-backlog.json are processed by zbeam-improvement-loop (Monday 5am) — they always route to needs_approval (never auto-apply) because adversarial re-review requires content judgment