IndiCause-Bench

IndiCause-Bench tests whether an AI model can work out what a market shock actually does, not just describe it. It contains 1,532 public shock scenarios across calm, crisis and recovery regimes, each with an exact ground-truth effect, plus 382 held back for private checks. Models are scored on whether they beat the naive answer "nothing changes", on error size, and on whether they name the right causal path. The leaderboard below is dated 30 October 2026.

benchmark.openkuber.comOpen

Use it here

What the benchmark measures, and why Indian markets need their own

Indian markets run on signals that general-purpose models are rarely trained on: exchange filings in mixed formats, policy announcements, state-owned bank dynamics and flows. To test causal reasoning without letting models recall history, the current season uses a fictional economy modelled on that structure: 12 firms in four sectors (lenders, builders, shippers, power), a composite index, a central bank and 22 variables, governed by a known structural causal model. Each regime has its own rules: some effects flip sign in a crisis, the stress regime has thresholds and fat tails, and the recovery regime has lags and momentum. About 15% of scenarios are decoys with no causal path to the target, where the right answer is zero.

Task format

Each scenario gives the model the state of all 22 variables on the day before, a shock and a news line. Example:

shock:    export_orders  −0.757  (medium, down)
target:   QUIL_Return  (5-day cumulative effect)
headline: "Export order books shrink, a clear move by recent standards"
state:    22 variables on day t−1

The model returns a direction, a magnitude in percentage points with a confidence interval, the causal path as a list of edges, and a short rationale. The ground-truth effect for this scenario is −0.47 points.

Scoring

• Beats "nothing changes": the share of scenarios where the model's error is smaller than predicting zero effect. This is the headline ranking.
• Mean L1: average absolute error of the predicted 5-day effect, in percentage points.
• CRI (Causal Reasoning Index, 0–100): 25% causal-path F1, 25% correct direction, 20% dose monotonicity across small, medium and tail shocks, 20% normalised L1, 10% interval calibration.

Leaderboard

Public scenarios, as of 30 October 2026. Some models answered fewer scenarios because of provider errors or timeouts.

What stands out: in the crisis regime, five of the seven models size shocks worse than predicting nothing at all. claude-haiku-5-5 is the only outside model that beats the baseline in every regime, and sarvam-105b misses by the widest margin. The median model overshoots small shocks (163% of the true size) and undershoots tail shocks (71%).

Submit a model

Email [email protected] with the model name and how to call it. We run it on the public scenarios and the 382 held-out scenarios, and publish the public score with the date. Teams entering LabhLabs Season 2 can run a private pre-check first.

Related: How IROHA works · LabhLabs