EMNLP 2026

LEAP: Likelihood Elicitation and Aggregation for LLM-based Probabilistic Forecasting

LEAP interprets retrieved evidence one item at a time, elicits local likelihoods, and combines them with an explicit prior through a deterministic Bayesian update.

Yufei Chen · Yiran Zhao · Xiaogang Xu† · Qipeng Xie · Jiafei Wu · Zhe Liu†

Shandong University · Nanjing University of Aeronautics and Astronautics · Zhejiang University · The Hong Kong University of Science and Technology (Guangzhou)

† Corresponding authors

Paper Code
Comparison between monolithic prediction and LEAP evidence-level probabilistic aggregation
Monolithic prediction reads all evidence at once. LEAP exposes evidence-level support and produces an auditable posterior.

Overview

Forecast from evidence without hiding the evidence.

Forecasting agents have become better at search and tool use, yet the final step often remains holistic: one LLM call reads the entire evidence bundle and writes a prediction. This makes it difficult to isolate what any single source contributed and can compress uncertainty into one confident answer.

LEAP changes only this prediction stage. It keeps the gathered evidence fixed, uses the LLM as a local evidence interpreter, and lets an explicit probabilistic model make the final judgment.

Method

Local elicitation, explicit aggregation.

The LLM estimates inputs to a probabilistic model. It never receives the full evidence set during elicitation and never writes the final forecast.

01

Interpret evidence locally

The LLM sees the task and one evidence item at a time. It returns structured likelihood parameters, not a final answer.

02

Start from an explicit prior

A data-derived or elicited prior records the base rate before any retrieved evidence is examined.

03

Aggregate deterministically

A closed-form Bayesian update combines the prior and retained likelihoods into the final posterior distribution.

04

Audit every contribution

Leave-one-out recomputation shows how each retained evidence item changes the forecast.

LEAP pipeline from evidence collection to likelihood elicitation and deterministic posterior aggregation
LEAP sits after evidence collection and can be attached to an existing forecasting agent as a prediction module.

Results

Consistent gains with the evidence held fixed.

On the paper benchmark drawn from FutureX, GAIA, and BrowseComp, LEAP improves FutureX, Spherical score, accuracy, and NCRPS for every tested base model under the fixed ReAct-style agent loop.

The same pattern largely carries over to four external agent CLI frameworks. Their macro-average improves on all five reported metrics when LEAP replaces the final monolithic prediction step.

5

base models

Fixed ReAct-style loop and a shared evidence set for both methods.

4

agent CLI frameworks

DeerFlow, Hermes, OpenClaw, and MiroFlow on unmodified traces.

3

forecast types

Continuous, single-choice, and multi-choice targets.

Forecast horizon analysis

LEAP becomes more useful as the horizon lengthens.

On the diagnostic subset, the absolute improvement over monolithic prediction rises across the tested 7-day, 30-day, and 60-day horizons. LEAP responds to indirect evidence with wider posteriors instead of sharp overconfidence.

LEAP improvement over monolithic prediction across 7, 30, and 60 day forecast horizons

Open source

Use LEAP as a pluggable forecasting skill.

The repository packages the Bayesian forecasting workflow as a skill that can operate after an agent has collected its evidence.

View code Read paper