mitre-coverage-report

name: mitre-coverage-report description: 'MITRE ATT&CK Coverage Report — YAML-driven PowerShell pipeline gathers analytic rule MITRE tags, custom detection techniques, SOC Optimization recommendations, and alert/incident operational data via az rest/az monitor/Graph API, writes a deterministic scratchpad, LLM renders the report. Covers tactic-level coverage matrix, technique-level drill-down with rule mapping, coverage gap identification, SOC Optimization threat scenario alignment, untagged rule remediation, ICS/OT technique tracking, and MITRE Coverage Score (5 weighted dimensions). Inline chat and markdown file output.' threat_pulse_domains: [incidents] drill_down_prompt: 'Run MITRE ATT&CK coverage report — tactic/technique coverage, gaps, SOC optimization'

MITRE ATT&CK Coverage Report — Instructions

Purpose

This skill generates a comprehensive MITRE ATT&CK Coverage Report analyzing detection coverage across the ATT&CK Enterprise framework. It inventories all analytic rules and custom detections, maps them to MITRE tactics and techniques, identifies coverage gaps, and provides prioritized recommendations for improving detection posture.

Entity Type: Sentinel workspace (from config.json)

What this report covers: Tactic-level coverage matrix with per-tactic technique counts and percentages, technique-level drill-down with rule-to-technique mapping, coverage gap identification against the full ATT&CK Enterprise framework, SOC Optimization threat scenario alignment (AiTM, ransomware, BEC, etc.), untagged rule remediation with AI-suggested MITRE tags, ICS/OT technique tracking, operational MITRE correlation (which rules actually fire), and a composite MITRE Coverage Score.

Complementary to: This skill pairs with the sentinel-ingestion-report skill — ingestion report covers data volume, tier optimization, and cost; MITRE coverage report covers detection posture against the ATT&CK framework. Run both for a complete workspace assessment.

Architecture

 ┌──────────────────────────────────────────────────────────────────┐
 │  YAML query files        PowerShell script         LLM render   │
 │  queries/phase1-3/  ──→  Invoke-MitreScan.ps1  ──→  Phase 4    │
 │  (6 .yaml files)         (~1030 lines)             (SKILL-      │
 │                          • az rest (Sentinel API)   report.md)  │
 │                          • Invoke-MgGraphRequest                │
 │                          • az monitor (KQL)                     │
 │                          • mitre-attck-enterprise.json          │
 │                          • m365-platform-coverage.json (CTID)   │
 │                          ↓                                      │
 │                     temp/mitre_scratch_<ts>.md                  │
 │                     (~35 KB, 18+ sections)                     │
 └──────────────────────────────────────────────────────────────────┘

Execution model:

• Phases 1-3 (data gathering): Fully automated by Invoke-MitreScan.ps1. Phase 1 uses az rest (Sentinel REST API) and optionally Invoke-MgGraphRequest (Graph API). Phase 2 uses az rest (SOC Optimization API). Phase 3 uses az monitor log-analytics query (KQL).
• Phase 4 (rendering): LLM reads the scratchpad + SKILL-report.md and renders the report. This is the only phase requiring LLM involvement.

Static reference: mitre-attck-enterprise.json contains ATT&CK Enterprise v16.1 with 14 tactics, 216 techniques, and 475 sub-techniques. The PS1 loads this at startup to compute coverage gaps against the full framework. This file is version-controlled and should be updated when MITRE publishes new ATT&CK releases.

Platform coverage reference: m365-platform-coverage.json is a compact CTID (Center for Threat-Informed Defense) mapping of M365 Defender product capabilities to ATT&CK techniques. Contains detect/protect/respond coverage for 81 detect techniques across 38 capabilities (7 SecurityAlert product groups). Used for the 3-tier platform coverage classification:

• Tier 1 (Alert-Proven): SecurityAlert from M6 query has MITRE technique attribution — highest confidence
• Tier 2 (Deployed Capability): Product is active (has alerts) and CTID claims detect coverage for the technique — medium confidence
• Tier 3 (Catalog Capability): CTID maps coverage but no alert evidence for the product in this workspace — lowest confidence

To rebuild from upstream: download the CTID M365 mapping JSON, transform with PowerShell (group by parent technique, map capabilities to SecurityAlert ProductName). See temp/ctid_raw.json for the raw source.

Companion Files — When to Load

📑 TABLE OF CONTENTS

1. Quick Start - 3-step execution pattern
2. Critical Workflow Rules - Prerequisites and prohibitions
3. Execution Workflow - Phases 0-4
4. Query File Reference - All 5 YAML files
5. Output Modes - Inline chat vs. Markdown file
6. Deterministic Rendering Rules - Rules A-D (mandatory for Phase 4)
7. MITRE Coverage Score - 5-dimension scoring methodology
8. Domain Reference - ATT&CK interpretation, tactic priorities, Sentinel-specific mappings
9. SVG Dashboard Generation - Visual dashboard from completed report

Quick Start (TL;DR)

3-step execution pattern:

Step 1:  Run Invoke-MitreScan.ps1 (Phases 1-3 — data gathering)
Step 2:  Read scratchpad + SKILL-report.md (Phase 4 prep)
Step 3:  Render report incrementally (§1 via create_file, then §2–§6 appended via replace_string_in_file)

Step 1: Run Data Gathering

# From workspace root — run all phases (default: 30 days alert/incident lookback):
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1"

# Specify a custom alert/incident lookback:
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1" -Days 7

# Run a specific phase (for re-runs / debugging):
& ".github/skills/mitre-coverage-report/Invoke-MitreScan.ps1" -Phase 1

Output: Scratchpad file at temp/mitre_scratch_<timestamp>.md (~28 KB, 12 sections).

Timing: Full run takes ~60-90 seconds (varying with REST API response times and KQL auth state).

Step 2: Load Rendering Context

1. Read the scratchpad file (path printed by PS1 at completion)
2. Read SKILL-report.md for rendering templates

Step 3: Render Report (Incremental Writes)

Render the report across multiple tool calls — one section per call — to avoid single-call output token limits that truncate large reports:

1. create_file → header + disclaimer + §1 (Executive Summary, Score, Inventory, Top 3 Recs)
2. replace_string_in_file → append §2 (Tactic Coverage Matrix)
3. replace_string_in_file → append §3 (Technique Deep Dive — largest section)
4. replace_string_in_file → append §4 (Coverage Gap Analysis)
5. replace_string_in_file → append §5 (Operational MITRE Correlation)
6. replace_string_in_file → append §6 + Appendix

Apply SKILL-report.md templates to scratchpad data, following Rules A–D. See SKILL-report.md for full section templates and the anchor pattern for each append.

⛔ Do NOT render §1–§6 in a single create_file call. The output will truncate silently. The scratchpad is ~60 KB; the rendered report exceeds the single-call output budget.

🔴 ALL 6 APPENDS ARE MANDATORY. Do NOT stop after §5 — §6 (Recommendations) and the Appendix (Score Methodology, Limitations) are critical and must be appended. After the 6th append, run grep_search for ## 6. Recommendations and ## Appendix on the report file to verify both exist. If either is missing, append the missing content immediately.

⚠️ CRITICAL WORKFLOW RULES - READ FIRST ⚠️

Before starting ANY MITRE coverage report:

1. Run Invoke-MitreScan.ps1 — this single script handles ALL data gathering (Phases 1-3). The LLM does NOT run queries, transcribe output, or write scratchpad sections
2. Read config.json for workspace ID, tenant, subscription, and Azure MCP parameters
3. ALWAYS ask the user for output mode if not specified: inline chat summary, markdown file report, or both (default: both)
4. ALWAYS ask the user for timeframe if not specified: the -Days parameter controls the alert/incident KQL lookback (Phase 3). Default: 30 days. Phases 1-2 (REST API) are not time-bounded
5. ALWAYS use create_file for markdown reports (never use terminal commands)
6. ALWAYS sanitize PII from saved reports — use generic placeholders for real rule names, workspace names, and tenant GUIDs in committed files
7. Read scratchpad + SKILL-report.md before rendering — the scratchpad is the sole data source
8. Custom Detections may be SKIPPED — the Graph API requires CustomDetection.Read.All which needs admin consent. If skipped, the report notes this and shows AR-only analysis. Do NOT treat SKIPPED as an error — it's a graceful degradation

Prerequisites

🔴 PROHIBITED

• ❌ Running REST/KQL queries via MCP tools during data gathering — PS1 handles all queries
• ❌ Writing or modifying scratchpad sections manually — PS1 is the sole writer
• ❌ Fabricating technique counts, rule names, or coverage percentages
• ❌ Inventing ATT&CK technique IDs or names not in the reference JSON
• ❌ Overriding MITRE Coverage Score dimensions — the PS1 computes these deterministically
• ❌ Rendering the report without first reading the scratchpad file
• ❌ Reporting "100% coverage" for any tactic unless the data actually shows every technique covered

Execution Workflow

Phase 0: Initialization

1. Read config.json for sentinel_workspace_id, subscription_id, Azure MCP parameters
2. Confirm output mode and timeframe with user (pass -Days to PS1; default 30)
3. Verify prerequisites: az login session active, correct subscription set

Phases 1-3: Data Gathering (automated by PS1)

Run Invoke-MitreScan.ps1 — it handles all 3 phases automatically:

Post-processing (automated by PS1):

Scratchpad output: PS1 writes all results to temp/mitre_scratch_<timestamp>.md (~28 KB, ~12 named sections). See SKILL-report.md for the Section-to-Scratchpad Mapping.

Phase 4: Render Output (LLM)

🔴 MANDATORY — Load scratchpad + report template before rendering:

1. Read the scratchpad file (path printed by PS1). This single file contains ALL data from Phases 1-3.
2. Read SKILL-report.md for the complete rendering templates and formatting rules.

Pre-render validation:

1. Verify scratchpad has all 3 phase sections (PHASE_1 through PHASE_3)
2. Check SCORE section has all 5 dimensions
3. If Phase 3 shows FAILED for M4/M5 (token expiry), note this in the report — the Operational dimension defaults to 0

Render — Section-by-Section:

Query File Reference

All queries are defined as YAML files in queries/phase1-3/.

YAML Format

id: mitre-m1                                   # Unique identifier
name: Analytic Rule MITRE Extraction            # Human-readable name
description: Fetch rules with tactics/techniques # What it does
phase: 1                                        # Which phase (1-3)
type: rest                                      # rest | graph | kql
url: https://management.azure.com/...           # REST API URL with placeholders
jmespath: value[].{...}                         # JMESPath projection (REST)

Complete Query Inventory

Output Modes

Mode 1: Inline Chat Summary (default for quick requests)

Compact executive summary rendered directly in chat with MITRE Coverage Score and top coverage gaps.

Mode 2: Markdown File Report

Full detailed report saved to reports/sentinel/mitre_coverage_report_<YYYYMMDD_HHMMSS>.md.

Mode 3: Both (default when user says "report" or "generate report")

Inline chat executive summary + full markdown file.

Ask user if not specified:

"How would you like the MITRE coverage report? I can provide:

1. Inline chat summary — MITRE Score + top gaps in chat
2. Markdown file — detailed report saved to reports/sentinel/
3. Both (recommended) — summary in chat + full report file"

Deterministic Rendering Rules

These rules eliminate LLM interpretation variance. Apply them EXACTLY during Phase 4 rendering.

Rule A: Coverage Level Classification

Assign emoji badges to each tactic row in the coverage matrix based on the percentage of techniques covered:

⛔ PROHIBITED: Assigning badges based on "importance" or "this tactic is more relevant." The badge MUST match the percentage threshold table above.

Rule B: Threat Scenario Priority

When rendering SOC Optimization threat scenarios, order by coverage gap (recommended minus active) descending, but assign badges based on completion rate (proportional to scenario size):

Why rate-based? Recommendation counts reflect the full Content Hub template catalogue including templates for vendor products not deployed in the environment (e.g., all firewall vendors). A 609-rule scenario will be permanently 🔴 under absolute-gap thresholds even at 80% coverage. Rate-based badges give proportional, meaningful progress signals.

CompletedBySystem note: CompletedBySystem is a SOC Optimization state, not a rate indicator. Some CompletedBySystem entries have low rates (recommended >> active). Always use the completion rate for badge assignment. The State column is displayed for context but does NOT override the rate-based badge.

Rule C: "Paper Tiger" Detection

When Phase 3 data is available, identify paper tiger rules — rules with MITRE tags that have NEVER produced an alert in the lookback period. These rules are tagged but non-operational, and their coverage is theoretical, not proven.

⛔ PROHIBITED: Reporting coverage percentages as "validated" when Phase 3 data is missing. If M4/M5 failed, state: "Coverage percentages reflect rule tagging only — operational validation unavailable (Phase 3 KQL queries failed)."

Rule D: Recommendation Ranking

Rank recommendations by impact using this priority order:

Rule E: CompletedByUser Completion-Rate Gate

When a SOC Optimization threat scenario has State == CompletedByUser, the user has manually marked it as reviewed. However, marking a scenario "complete" after enabling 2/500 recommendations is fundamentally different from enabling 28/46. Use the completion rate (ActiveDetections / RecommendedDetections × 100) to determine rendering treatment:

Threshold: 50% is the default. This balances trust in the user's judgment against protection from rubber-stamped completions.

Scratchpad column: CompletionRate is pre-computed by the PS1 and included in the ThreatScenarios row. The LLM reads this value directly — do not recompute it.

Interaction with Rule B (rate-based badges): Rule B still applies for badge assignment on all scenarios. Rule E only controls where CompletedByUser scenarios are rendered (active table vs reviewed summary) and whether they appear in §6 recommendations.

CompletedBySystem scenarios are not affected — they continue to use rate-based badges (Rule B) without the completion-rate gate, since the system assessment is independent of user action.

MITRE Coverage Score

The MITRE Coverage Score is a composite metric (0-100) computed by the PS1 from 5 weighted dimensions. Each dimension scores 0-100 independently, then the weighted sum produces the final score.

Dimensions

Score Interpretation

Score Context Notes

• Operational = 0 when Phase 3 KQL queries fail (token expiry). Report this: "Operational score 0 reflects data unavailability, not necessarily poor operational coverage."
• SOC Alignment = 50 (default) when no SOC Optimization recommendations exist. This is a neutral baseline, not a penalty.
• Breadth score is naturally low because the ATT&CK framework contains 216+ techniques, many of which are endpoint-specific or pre-compromise with limited Sentinel visibility. Do NOT present this as a crisis — contextualize it: "Prioritize coverage by threat scenario relevance rather than pursuing raw percentage."
• Custom Detections SKIPPED affects Breadth and Tagging dimensions (rules not counted). Note the impact in the report.
• Platform Coverage is reported as a supplementary metric alongside the MITRE Score (not folded into the 5 dimensions). The scratchpad includes Platform_Tier1/2/3, Platform_ActiveProducts, and RuleBasedPlusPlatform_Coverage. Render this in §1 and §5 per SKILL-report.md templates. The CTID tier classification requires m365-platform-coverage.json — if the file is missing, platform tiers default to empty and the report notes the limitation.

Domain Reference

ATT&CK Enterprise Tactic Kill Chain Order

The 14 ATT&CK Enterprise tactics in kill chain order (PS1 uses this ordering for all output):

Detectability classification:

• ✅ Detectable: Techniques in this tactic generate observable events in Sentinel data sources (sign-in logs, audit logs, endpoint telemetry, email events, etc.). KQL detection rules can be written and deployed.
• ⬜ Inherent blind spot: Techniques in this tactic describe attacker activity that occurs outside the monitored environment (e.g., attacker creating fake accounts on external services, acquiring infrastructure). CTID mappings for these tactics are typically protect/respond capabilities (Conditional Access blocking, PAM restrictions), not detect. No KQL detection rules exist or can realistically be created. Do not recommend deploying rules for inherent blind spot tactics — acknowledge the limitation and recommend compensating controls (e.g., brand monitoring services, threat intelligence feeds) if relevant.

Sentinel-Specific MITRE Mapping Notes

• Sentinel uses PascalCase for tactic names in the REST API: InitialAccess, CommandAndControl, CredentialAccess. The ATT&CK STIX data uses kebab-case (initial-access). The reference JSON maps between these.
• Sub-techniques (T1xxx.xxx) are tracked by Sentinel but the REST API properties.techniques field may contain both parent techniques (T1078) and sub-techniques (T1078.004). The PS1 counts at the parent technique level for coverage matrix purposes.
• ICS/OT techniques (T0xxx) use a separate numbering scheme from ATT&CK for ICS. These are extracted and reported separately since they don't map to the Enterprise framework.
• Custom Detection mitreTechniques uses the same technique ID format but may specify sub-techniques that analytic rules don't. The PS1 aggregates both sources.

Tactic-Specific Detection Guidance

When rendering recommendations (§6), use these cloud/identity-relevant technique priorities:

SOC Optimization Threat Scenario Reference

SOC Optimization recommendations map to named threat scenarios. When rendering §4, interpret these:

SOC Optimization Recommendation States

SVG Dashboard Generation

After the report is generated, the user may request an SVG dashboard visualization.

Trigger: "generate SVG dashboard", "visualize this report", "SVG from the MITRE report"

Workflow:

1. Load the svg-dashboard skill
2. Use the rendered report + scratchpad data to build visualization widgets
3. Recommended widget types for MITRE coverage:
   • Score card — MITRE Coverage Score (40.1/100) with 5 dimension breakdown
   • Bar chart — Per-tactic coverage percentages (14 bars)
   • Donut chart — Rule inventory breakdown (AR enabled/disabled, CD enabled/disabled, untagged)
   • Table — Top 5 coverage gaps (tactic + gap %)
   • KPI cards — Total techniques covered, SOC scenarios met, untagged rules

Troubleshooting