zbeam-gap-researcher

name: zbeam-gap-researcher description: "Identifies missing content pages by cross-referencing existing pages against keyword opportunities. Trigger on 'what pages are we missing'."

Z-Beam Gap Researcher

You identify content pages that z-beam.com is missing. Output: a ranked gap report that feeds directly into zbeam-content-researcher and zbeam-differentiation-researcher.

Intelligence freshness check

Use load_strategy_weights() from skills/shared/intelligence-utils.md. If age > 45, flag and continue. Apply weights to prioritize gap type scoring and search depth decisions.

Step 0: Read gap performance log (adaptive scoring — run before any searches)

Before scoring any candidate, read the historical performance of prior gap recommendations to calibrate the scoring model:

import json, os, glob

perf_path = 'data/seo/gap-performance.json'

if os.path.exists(perf_path):
    perf = json.load(open(perf_path))
    entries = perf.get('recommendations', [])

    # Compute performance by gap type and content type
    type_performance = {}
    for e in entries:
        key = f"{e.get('aiGapType','unknown')}/{e.get('contentType','unknown')}"
        p = type_performance.setdefault(key, {'count': 0, 'written': 0, 'impressions_30d': [], 'impressions_90d': []})
        p['count'] += 1
        if e.get('written'): p['written'] += 1
        if e.get('impressions30d') is not None: p['impressions_30d'].append(e['impressions30d'])
        if e.get('impressions90d') is not None: p['impressions_90d'].append(e['impressions90d'])

    # Derive score modifiers (+1 / 0 / -1) per type
    score_modifiers = {}
    for key, data in type_performance.items():
        avg_90 = sum(data['impressions_90d']) / len(data['impressions_90d']) if data['impressions_90d'] else None
        if avg_90 is not None:
            if avg_90 >= 20:
                score_modifiers[key] = +1
                print(f'BOOST +1: {key} (avg 90d impressions: {avg_90:.0f})')
            elif avg_90 <= 3 and len(data['impressions_90d']) >= 3:
                score_modifiers[key] = -1
                print(f'DISCOUNT -1: {key} (avg 90d impressions: {avg_90:.0f})')

    # Flag written rate — gap types that are consistently recommended but never written
    for key, data in type_performance.items():
        if data['count'] >= 3 and data['written'] / data['count'] < 0.25:
            print(f'LOW WRITE RATE: {key} recommended {data["count"]}x but written {data["written"]}x — consider whether Z-Beam fit is lower than scored')
else:
    score_modifiers = {}
    print('No gap performance log found — using unweighted scoring')

Apply score_modifiers when scoring candidates in Step 4: add the modifier to each candidate's computed score based on its aiGapType/contentType key.

After producing the gap report, append all new recommendations to the performance log:

import json, os
from datetime import date

perf_path = 'data/seo/gap-performance.json'
perf = json.load(open(perf_path)) if os.path.exists(perf_path) else {'recommendations': []}

# recommended_pages: list of dicts from Step 5 "Recommended next 3 pages"
for page in recommended_pages:
    perf['recommendations'].append({
        'slug': page['slug'],
        'keyword': page['keyword'],
        'contentType': page['contentType'],
        'aiGapType': page.get('aiGapType', 'unknown'),
        'recommendedDate': date.today().isoformat(),
        'written': False,          # visibility tracker updates this to True when page appears in inventory
        'impressions30d': None,    # visibility tracker fills in at 30 days
        'impressions60d': None,
        'impressions90d': None
    })

json.dump(perf, open(perf_path, 'w'), indent=2)

Page-level visibility (impressions/clicks/position) now comes from the GSC export in data/search-console/ (run npm run seo:gsc:export; 90-day window via npm run seo:gsc:export:correlator), not the retired zbeam-visibility-tracker. The gap researcher reads that data on the next run.

Step 1: Check for input files

Before doing any web searches, load available structured inputs in this priority order:

Pipeline auditor gap signals (highest priority) — look for: data/audit/gap-signals-[YYYY-MM-DD].json (most recent by date)

If found, this is the pre-synthesised output of all data streams combined. Extract:

• highPriorityGaps — these are confirmed by multiple signal sources, treat as Tier 1
• contentStrengtheningTargets — existing pages needing enrichment, not new pages

SEO tracker output — look for the most recent file matching: data/seo/weekly-[YYYY-MM-DD].json

If found, extract contentOpportunities and keywordOpportunities arrays. These are pre-scored signals — treat them as confirmed candidates, not hypotheses.

AI search tracker output — look for the most recent file matching: data/ai-search/weekly-[YYYY-MM-DD].json

If found, extract summary.gapsByType:

• typeB items → highest priority, tag as ai-search-first-mover
• typeA items → medium priority, tag as ai-search-authority-gap
• typeC items → content depth queue, tag as ai-search-page-depth

If neither file exists, proceed using web searches only and note the missing inputs.

Step 2: Read the existing page inventory

ls frontmatter/applications/ | sed 's/\.yaml//'
ls frontmatter/locations/ | sed 's/\.yaml//'
ls frontmatter/materials/ | sed 's/\.yaml//'
ls static-pages/ | sed 's/\.yaml//' 2>/dev/null

Build a plain-language coverage list by stripping -laser-cleaning, -applications etc. from each slug. A topic with partial coverage (mentioned on a page but not the primary subject) still counts as a gap.

Step 3: Supplement with web searches

Search depth is tiered by AI gap type — spend more searches on higher-priority gaps:

Type B gaps (AI leaves question unanswered — highest ROI): run 4–5 searches, including competitor content and People Also Ask results. These are first-mover opportunities where a new Z-Beam page could become the AI's primary citation source.

Type A gaps (AI answers but doesn't cite Z-Beam): run 2–3 searches focused on what differentiates Z-Beam's angle from what AI is already saying.

Type C / SEO-only gaps: 1–2 searches to confirm demand and check competitor coverage depth. No deep research needed at this stage — the content researcher will do that.

GSC high-impression / 0-click seeds — confirmed demand, no conversions. Always include in the scored candidate list:

Run searches on each seed before scoring — demand shifts over time.

For any candidate not surfaced by input files or seeds:

• "laser cleaning [industry/application] Bay Area"
• "laser cleaning [city] California" — San Jose, Napa, Marin, Oakland, Palo Alto
• Emerging modifiers: "lead paint", "historic preservation", "weld prep", "marine"

Step 4: Score each candidate gap

Only include gaps where user intent signal and Z-Beam fit are both Medium or above.

Step 5: Output the gap report

## Content Gap Report — [Date]

### Confirmed gaps (ranked by opportunity)

| Priority | Topic | Target keyword | Demand signal | AI gap type | Suggested content type |
|---|---|---|---|---|---|
| 1 | Lead paint removal | laser lead paint removal Bay Area | SEO tracker + no local page | Type B | applications |
| 2 | … | … | … | … | … |

### Already covered (not gaps)
[List topics considered but already having a page]

### Recommended next 3 pages
1. Slug: [slug] | Keyword: [keyword] | Type: application/location | Rationale: [one sentence]
2. …
3. …

Save the report to: data/seo/gap-report-[YYYY-MM-DD].md

The recommended next 3 pages feed directly into zbeam-content-researcher and zbeam-differentiation-researcher — pass the topic, target keyword, and content type when invoking those skills.