By name: forecaster description: Superforecasting methodology and calibrated probability estimation best practices. Use when making forecasts to apply evidence-backed reasoning patterns.
Forecaster Methodology
Evidence-backed forecasting principles (benchmark analysis: +63.90 advantage).
Core Principles
- Start with base rates - Calculate empirical frequencies from historical data
- Quantify adjustments - Each factor as explicit "+X% because [evidence]", not vague leans
- Calculate uncertainty - Use historical volatility to set confidence intervals
- Seek disconfirming evidence - Actively look for reasons you're wrong
Probability Aggregation Format
Always show your reasoning chain:
BASE RATE: X% [source, N=sample_size]
ADJUSTMENTS:
+Y% [factor 1 with specific evidence]
-Z% [factor 2 with specific evidence]
FINAL: X% + Y% - Z% = RESULT%
Calibration Guidelines
| Range | Use When | Evidence Level |
|---|---|---|
| 35-65% | High uncertainty | Limited/conflicting data |
| 25-75% | Moderate confidence | Some empirical data |
| 15-85% | Strong confidence | Multiple data sources |
| 5-95% | Near certainty | Exceptional evidence (rare) |
Log scoring severely punishes overconfidence. When uncertain, widen intervals.
Anti-Patterns
These caused -500+ point losses in benchmarks:
- ❌ Qualitative-only for quantifiable questions → ✅ Calculate empirical base rates
- ❌ Vague adjustments ("~10% shift") → ✅ Quantified ("+5% due to X")
- ❌ Ignoring contradicting evidence → ✅ Weigh ALL evidence
- ❌ Narrow percentile spreads → ✅ 10th-90th spread = 20-40% of range
Numeric Questions
Critical: Your 10th-90th percentile range should contain the answer ~80% of time.
If unsure, make distribution WIDER. Ask: "What if my central estimate is completely wrong?"
Required percentiles: 1, 5, 10, 20, 25, 30, 40, 50, 60, 70, 75, 80, 90, 95, 99