METHOD

How this works,
in full.

No hand-picked questions, no edited answers, no quietly deleted misses. The whole loop is one Cloudflare Worker and the code is public.

Where the questions come from

Two sources. The first is prediction markets — Polymarket and Kalshi — where people risk money on a dated, written-down outcome. Those questions arrive already falsifiable and already carry a price, which is what we benchmark the models against. The second is a search-grounded sweep of one news beat per run, which catches the things nobody has opened a market on.

Everything is then rewritten into a single declarative statement with a deadline. Anything vague, duplicated, unfalsifiable, or about a private individual is dropped at this step.

How the forecast is made

Six models from six different labs answer the same prompt independently: Gemini, Claude, GPT, Grok, DeepSeek, Qwen. The published number is the median, not the mean — one model at 2% shouldn't drag a panel that otherwise agrees. We also publish the spread, because six models agreeing at 60% and six models scattered from 10 to 95 are very different claims.

No model is ever shown the market price. If it were, the comparison on every page would only measure how well a model can read a number out of its prompt.

How outcomes get decided

This is the part that can put a falsehood on a page as a fact, so it is tiered by how much the evidence is actually worth.

First, the exchange. Most questions here come from Polymarket or Kalshi, and those markets settle themselves — with real money paid out, a formal dispute window, and bonds that reach six figures. When the exchange has paid out, that is the answer. No model gets a vote. Because a model rewrote the market's question into our headline, one cheap check confirms the two still mean the same thing before the settlement is applied; if the wording drifted, or if the resolution was itself disputed, it goes to a human instead.

Then, a jury. For questions with no market behind them, a search-grounded model gathers evidence and cites it. Three models from three different labs then rule on that evidence alone — they cannot search, so they cannot talk themselves into a story, and they never see each other's answers. The verdict publishes only if all three agree, with at least two sources. Each juror also has to say what it is standing on, and a "didn't happen" resting purely on nobody having written about it never publishes automatically — absence of coverage is the easiest way to be confidently wrong.

Then, a person. Everything else waits. The sources are printed on the page, the method is named on every verdict, and disputing one takes a minute.

How the scoreboard works

Every settled question scores every model that answered it, using the Brier score: the squared error of the probability. Lower is better; 0.250 is what you get by answering 50 to everything. The market price on the day we asked is scored the same way on the same questions, which is the only honest way to ask whether any of this beats the crowd. Nothing is graded retroactively and no forecast is edited after the fact.

Why forecasts arrive in batches

The machine wakes once an hour and does one slice of the work: harvest a few questions, put a couple to the panel, check what has come due. Nothing is queued up and released on a schedule — a forecast appears when its turn comes round. Between slices it keeps re-pricing open markets and recomputing the scoreboard, which needs no model at all.

Check it yourself

Every claim on this page is a claim about code you can read. The prompts, the median, the Brier maths and the three-tier resolver are all in one MIT-licensed repository, and the forecasts are on a public API.

Read the source

Limits, stated plainly