What is extremizing in forecasting?

Extremizing is a statistical technique that adjusts aggregated predictions away from the 50% midpoint toward the extremes (0% or 100%) to correct for the natural conservatism and underconfidence of crowd-sourced forecasts.

Does this work for small teams or 'superforecasters'?

Extremizing is less effective for small samples due to statistical noise and is generally unnecessary for elite 'superforecaster' teams who are already optimally calibrated.

How is the extremizing factor calculated?

The factor usually ranges from 1.2 to 1.5. It is applied to the log-odds of a probability, with higher factors used for more diverse crowds and lower factors for more uniform groups.

When should I use the Extremizing Forecasts skill?

It is most effective when you have independent, diverse forecasts from a large group (ideally 20+ sources) that show significant convergence around a specific probability.

Extremizing Forecasts

Name: Extremizing Forecasts
Author: lev-os

bylev-os

0•

Data Science & ML

Adjusts and optimizes aggregated probability forecasts to compensate for crowd conservatism and improve predictive accuracy.

This skill implements the extremizing algorithm derived from Philip Tetlock's superforecasting research, designed to refine collective predictions by pushing aggregated probabilities away from the 50% mean toward more accurate extremes. It is particularly effective when dealing with diverse, independent forecaster groups where convergence suggests a higher certainty than the raw average implies. By applying a log-odds transformation, the skill helps teams transform 'crowd wisdom' into high-conviction insights for geopolitical, financial, or strategic decision-making while avoiding common statistical biases.

Key Features

01Validation support via Brier score calculation to measure accuracy improvements

02Built-in logic to detect and handle crowd conservatism bias

03Guidance on selecting optimal forecast pools and avoiding groupthink pitfalls

040 GitHub stars

05Automated log-odds transformation for probability calibration

06Adjustable extremizing factors based on forecaster diversity and agreement

Use Cases

01Aggregating internal team predictions for product launch success or market entry outcomes

02Refinement of geopolitical risk assessments based on diverse analyst inputs

03Enhancing financial market sentiment analysis where analyst consensus is highly convergent

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add lev-os/agents extremizing-forecasts

For use in Claude.ai and ChatGPT

Download Skill