Skew-Adjusted Extremized-Mean: A Simple Method for Identifying and Learning From Contrarian Minorities in Groups of Forecasters

Ben Powell; Ville Satopaa; Niall MacKay; Philip Tetlock

doi:10.1037/dec0000191

Skew-Adjusted Extremized-Mean: A Simple Method for Identifying and Learning From Contrarian Minorities in Groups of Forecasters

Ben Powell, Ville Satopaa, Niall MacKay, Philip Tetlock

Mathematics

Research output: Contribution to journal › Article › peer-review

Abstract

Recent work in forecast aggregation has demonstrated that paying attention to contrarian minorities among larger groups of forecasters can improve aggregated probabilistic forecasts. In those papers, the minorities are identified using `meta-questions' that ask forecasters about their forecasting abilities or those of others. In the current paper, we explain how contrarian minorities can be identified without the meta-questions by inspecting the skewness of the distribution of the forecasts. Inspired by this observation, we introduce a new forecast aggregation tool called \textit{Skew-Adjusted Extremized-Mean} and demonstrate its superior predictive power on a large set of geopolitical and general knowledge forecasting data.

Original language	English
Pages (from-to)	173–193
Journal	Decision
Volume	11
Issue number	1
DOIs	https://doi.org/10.1037/dec0000191
Publication status	Published - 2024

Keywords

Forecasting
crowd-wisdom

Access to Document

10.1037/dec0000191

skew_adjusted_extremized_mean_v5_cleaned_of_commentsAccepted author manuscript, 421 KBLicence: GNU GPL

Cite this

@article{9e6e3ba7d2b5413481c793429794c35c,

title = "Skew-Adjusted Extremized-Mean: A Simple Method for Identifying and Learning From Contrarian Minorities in Groups of Forecasters",

abstract = "Recent work in forecast aggregation has demonstrated that paying attention to contrarian minorities among larger groups of forecasters can improve aggregated probabilistic forecasts. In those papers, the minorities are identified using `meta-questions' that ask forecasters about their forecasting abilities or those of others. In the current paper, we explain how contrarian minorities can be identified without the meta-questions by inspecting the skewness of the distribution of the forecasts. Inspired by this observation, we introduce a new forecast aggregation tool called \textit{Skew-Adjusted Extremized-Mean} and demonstrate its superior predictive power on a large set of geopolitical and general knowledge forecasting data.",

keywords = "Forecasting, crowd-wisdom",

author = "Ben Powell and Ville Satopaa and Niall MacKay and Philip Tetlock",

year = "2024",

doi = "10.1037/dec0000191",

language = "English",

volume = "11",

pages = "173–193",

journal = "Decision",

issn = "2325-9965",

publisher = "American Psychological Association",

number = "1",

}

TY - JOUR

T1 - Skew-Adjusted Extremized-Mean

T2 - A Simple Method for Identifying and Learning From Contrarian Minorities in Groups of Forecasters

AU - Powell, Ben

AU - Satopaa, Ville

AU - MacKay, Niall

AU - Tetlock, Philip

PY - 2024

Y1 - 2024

N2 - Recent work in forecast aggregation has demonstrated that paying attention to contrarian minorities among larger groups of forecasters can improve aggregated probabilistic forecasts. In those papers, the minorities are identified using `meta-questions' that ask forecasters about their forecasting abilities or those of others. In the current paper, we explain how contrarian minorities can be identified without the meta-questions by inspecting the skewness of the distribution of the forecasts. Inspired by this observation, we introduce a new forecast aggregation tool called \textit{Skew-Adjusted Extremized-Mean} and demonstrate its superior predictive power on a large set of geopolitical and general knowledge forecasting data.

AB - Recent work in forecast aggregation has demonstrated that paying attention to contrarian minorities among larger groups of forecasters can improve aggregated probabilistic forecasts. In those papers, the minorities are identified using `meta-questions' that ask forecasters about their forecasting abilities or those of others. In the current paper, we explain how contrarian minorities can be identified without the meta-questions by inspecting the skewness of the distribution of the forecasts. Inspired by this observation, we introduce a new forecast aggregation tool called \textit{Skew-Adjusted Extremized-Mean} and demonstrate its superior predictive power on a large set of geopolitical and general knowledge forecasting data.

KW - Forecasting

KW - crowd-wisdom

U2 - 10.1037/dec0000191

DO - 10.1037/dec0000191

M3 - Article

SN - 2325-9965

VL - 11

SP - 173

EP - 193

JO - Decision

JF - Decision

IS - 1

ER -