analyze-latest-policy-sweep - SKILL.md Agent Skill

name: analyze-latest-policy-sweep description: Analyze latest policy analysis sweep runs (policy_analysis) by comparing episodes/summary metrics, diagnostics, and video artifacts; generate a concise markdown report and optional frame snapshots.

Analyze Latest Policy Sweep

Use this skill when asked to compare recent *_policy_analysis_* benchmark runs (fast_pysf_planner, PPO, ORCA, etc.) and produce a summary report with diagnostics and video links.

Quick start

Confirm run directories exist and contain episodes.jsonl + summary.json.
Load metrics from episodes.jsonl and aggregates from summary.json.
Produce a report in specs/409-planner-comparison/.

Required report contents

Aggregate metrics table (success rate, collision rate, time_to_goal_norm, avg_speed, near_misses, comfort_exposure, energy, jerk/curvature raw + filtered if present).
Worst collision scenarios (top 5) per run.
Lowest success scenarios (bottom 5) per run.
Top problem episodes (score = collisions10 + comfort2 + failure) with video paths.
Diagnostics: path_efficiency saturation, shortest_path_len presence, low-speed filter behavior (ε=0.1).

Optional

Extract frames (25/50/75% of duration) for top problem episodes using ffmpeg into output/analysis/<report_id>/frames/ and list PNG paths in the report.

Template

Use specs/409-planner-comparison/analyze_latest_policy_sweep_prompt.md as a template if you need a consistent structure.

Video naming convention

output/recordings/<run_dir>/<scenario>_seed<seed>_<policy>.mp4

Diagnostics details

Path-efficiency saturation: fraction with path_efficiency >= 0.999 (clipped at 1.0).
Shortest path presence: ensure shortest_path_len is present in metrics when available.
Low-speed filter: report low_speed_frac and filtered metrics (jerk_mean_eps0p1, curvature_mean_eps0p1 when present). Reason: curvature divides by |v|^3; near-zero speed inflates noise.