Forecasting Research Institute (@Research

Forecasting Research Institute

262 posts

Forecasting Research Institute

@Research_FRI

We advance the science of forecasting to improve decision-making on high stakes issues. Co-founded by chief scientist Philip Tetlock.

forecastingresearch.org

Joined May 2023

Following

3,522

Followers

Forecasting Research Institute
@Research_FRI
Oct 8, 2025
Replying to @Research_FRI
⬆️ LLMs’ forecasting abilities are steadily improving. GPT-4 (released March 2023) achieved a difficulty-adjusted Brier score of 0.131. Nearly two years later, GPT-4.5 (released Feb 2025) scored 0.101—a substantial improvement. A linear extrapolation of state-of-the-art LLM
229K
Forecasting Research Institute
@Research_FRI
Nov 10, 2025
Today, we are launching the most rigorous ongoing source of expert forecasts on the future of AI: the Longitudinal Expert AI Panel (LEAP). We’ve assembled a panel of 339 top experts across computer science, AI industry, economics, and AI policy. Roughly every month—for the next
87K
Forecasting Research Institute
@Research_FRI
Sep 2, 2025
We now have the first accuracy results from the largest-ever existential risk forecasting tournament. In 2022, we convened 80 experts and 89 superforecasters for the Existential Risk Persuasion Tournament (XPT), which collected thousands of forecasts in 172 questions across
48K
Forecasting Research Institute
@Research_FRI
Sep 2, 2025
Replying to @Research_FRI
The International Mathematical Olympiad results were even more surprising. AI systems achieved gold-level performance at the IMO in July 2025. Superforecasters assigned this outcome just a 2.3% probability. Domain experts put it at 8.6%.
30K
Forecasting Research Institute
@Research_FRI
Oct 8, 2025
Is AI on track to match top human forecasters at predicting the future? Today, FRI is releasing an update to ForecastBench—our benchmark that tracks how accurate LLMs are at forecasting real-world events. A trend extrapolation of our results suggests LLMs will reach
44K
Forecasting Research Institute
@Research_FRI
Mar 11, 2024
Today, we published results from an adversarial collaboration focused on forecasting risks from AI. This project: - Involved intensive engagement among 22 carefully chosen superforecasters and domain experts - Aimed to identify key drivers of disagreement and "crux" questions
112K
Forecasting Research Institute
@Research_FRI
Jul 1, 2025
Our new study finds: recent AI capabilities could increase the risk of a human-caused epidemic by 2-5x, according to 46 biosecurity experts and 22 top forecasters. One critical AI threshold that most experts said wouldn't be hit until 2030 was actually crossed in early 2025. But
37K
Forecasting Research Institute
@Research_FRI
Sep 2, 2025
Replying to @Research_FRI
Respondents—especially superforecasters—underestimated AI progress. Participants predicted the state-of-the-art accuracy of ML models on the MATH, MMLU, and QuaLITY benchmarks by June 2025. Domain experts assigned probabilities of 21.4%, 25%, and 43.5% to the achieved outcomes.
25K
Forecasting Research Institute
@Research_FRI
Oct 1, 2024
Today, we're excited to announce ForecastBench: a new benchmark for evaluating AI and human forecasting capabilities. Our research indicates that AI remains worse at forecasting than expert forecasters. 🧵 Arxiv: arxiv.org/abs/2409.19839 Website: forecastbench.org
arxiv.org
ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities
Forecasts of future events are essential inputs into informed decision-making. Machine learning (ML) systems have the potential to deliver forecasts at scale, but there is no framework for...
45K
Forecasting Research Institute
@Research_FRI
Oct 8, 2025
Replying to @Research_FRI
You’ve probably noticed that we’re missing ratings for newer models such as GPT-5 or Grok 4. That’s because ForecastBench has a 50-day delay for leaderboard inclusion. We've found that very short-run forecasting accuracy isn't always correlated with actual forecasting ability.
6.2K
Forecasting Research Institute
@Research_FRI
Oct 29, 2024
Today we're releasing findings from the largest-ever forecasting survey of nuclear risk experts & superforecasters (151 total), conducted with @OpenNuclear. The median expert saw a 5% chance of catastrophe by 2045 - but identified policies that could cut this risk in half. 🧵
14K
Forecasting Research Institute
@Research_FRI
Oct 8, 2025
🔮 When will AI forecasters match top human forecasters at predicting the future? In a recent @cowenconvos podcast episode, @NateSilver538 said 10–15 years while @tylercowen predicted 1–2 years. Who was right? Our updated AI forecasting benchmark, ForecastBench, suggests that
Forecasting Research Institute
@Research_FRI
Oct 8, 2025
Is AI on track to match top human forecasters at predicting the future? Today, FRI is releasing an update to ForecastBench—our benchmark that tracks how accurate LLMs are at forecasting real-world events. A trend extrapolation of our results suggests LLMs will reach
34K
Forecasting Research Institute
@Research_FRI
Jul 10, 2023
Replying to @Research_FRI
Our main results: 🟡Superforecasters estimated a 1% chance of human extinction by 2100, while experts estimated a 6% chance 🟡Superforecasters and experts were furthest apart on AI risk (and less so on nuclear risk) 🟡For both, intersubjective accuracy predicted lower estimates
54K
Forecasting Research Institute
@Research_FRI
Aug 12, 2024
Today, we're releasing a report on our study using "conditional trees" to generate high-value forecasting questions about AI risk. Our method produced more informative questions than those on existing forecasting platforms. Here are the key findings: 🧵
15K