An LLM judging violent repricings
When a market moves 20¢ in an hour, minute zero has no news — the wires catch up while the book is still dislocated. The one LLM configuration that has ever tested positive here: judging, not forecasting.
The hypothesisThe general LLM-forecaster failed — the market beats the model on ambient news. But a dislocation is different: a 20¢-in-an-hour reprice is either real information or panic, the wires lag the move by minutes, and a model reading fresh headlines against resolution rules can classify which is which while the book is still displaced. Judge the move, don't predict the world.
The test
An event-triggered daemon scans top markets for violent hourly repricings on real volume (sports excluded — those moves are just the game), then has an LLM judge each one up to three times as headlines develop: given the resolution rules, dated fresh news, and the price withheld, is the new level justified? Verdicts with the model's own confidence score become would-enter signals at the live executable ask, settled at resolution. Same model family that failed the forecasting test — different question shape.
The result
The confidence gate is doing exactly what doctrine says it should: all 33 settled signals average +14.9% ROI-at-ask, but the cohort where the model's confidence cleared its threshold runs 10 wins, 3 losses, +88.4% — while low-confidence verdicts are negative on both sides, which is what an honest confidence signal looks like. Thirteen of the thirty high-confidence settlements the verdict gate requires are in; dislocations worth judging arrive only a few times a week, so the gate is months out at natural accrual.
Conclusion
Open, and the strongest evidence yet for the doctrine this lab keeps re-learning: language models earn nothing predicting markets and something real classifying discrete situations — especially their own certainty. If the gate confirms, the next decision is widening detection to feed the judge more cases without diluting the confidence gate that is carrying the entire result.