An LLM vs the market on breaking news
Give a language model the resolution rules and fresh headlines, hide the price, and ask for a fair probability. Where it disagreed with the market, the market was right.
The hypothesisMarkets are slow to digest news. An LLM reading dated headlines against each market's official resolution criteria — with the market price withheld so it cannot anchor — will find genuine mispricings, especially in neglected long-tail markets where no professional is paying attention.
The test
Daily, for a set of news-sensitive markets: fetch the market's resolution rules and fresh dated headlines, ask the model for an independent probability and a confidence score, then compare to the live executable ask. Signals fired only when the model's edge cleared a threshold with adequate confidence and a sane spread. Two lanes ran separately — liquid markets and a long-tail lane ($100–$1k daily volume) where the lazy-price hypothesis should be strongest. Every signal settled at resolution against the ask that was actually available.
The result
Eighteen would-enter signals settled: 3 wins, 15 losses, −70% ROI at the logged prices, with both lanes negative. The failure pattern was consistent: when the model diverged sharply from the market, the divergence was the model's misreading — stale stories dressed as breaking news, resolution criteria subtleties, or probability estimates anchored on narrative plausibility rather than base rates. The market price, which aggregates people betting actual money, had already digested everything the headlines contained.
Conclusion
The market reads the news faster and better than a language model prompted with it. This retired the entire "LLM forecaster" wing of our research and left a narrower doctrine that later experiments confirmed from both sides: LLMs earn their keep classifying and extracting — never forecasting. Anyone selling AI-predicts-the-market signals is selling the left half of this result and hiding the right half.