Every Model Turns Into a Coin Flip. AI Is a Model.
· AI & Machine Learning

Every Model Turns Into a Coin Flip. AI Is a Model.

Every model's forecast starts as a narrow spike and spreads until it's flat. Hurricane cones, price models and language models all follow that curve.

By Geordie Everitt

Curated from CIqSi (ciqsi.net)

Every model that predicts the future turns into a coin flip, and it happens faster than most people expect. AI is a model, so it does too.

I picture every prediction as a curve. At the moment you ask, the curve is a tall, narrow spike. The model knows where things stand, and for the next little while it's pretty good. How good depends on what you feed it and what you ask it. Then the spike spreads. Each step forward makes it wider and lower, and it widens by a steady percentage each time, which is what exponential means. At T=100, whatever the units are for your subject, the curve is flat. A flat curve gives every outcome about the same chance, so it's a coin toss with background noise. The units depend on the subject, and the shape is the same.

I've been using the word "model" loosely, so I should say what I mean by it. A hurricane model takes the latest readings and predicts where a storm will go. A price model takes recent prices and predicts the next one. A language model takes the words so far and predicts the next one. A person with an opinion about the future is running a model too, one built from everything they've read. The curve belongs to all of them.

The cone is the curve

Hurricanes are the best place to watch it, because the Hurricane Center publishes the numbers. The white cone on a hurricane map is the curve drawn on a map. It's a string of circles, one for each forecast time, and each circle holds the storm's center about two-thirds of the time. At twelve hours the circle has a radius of about 29 miles. At five days it's about 230. Day by day it grows by about half again.

The Hurricane Center draws tracks out to five days and stops. Past that, forecasters switch from tracks to odds.

A spaghetti plot draws the same curve with a different pen. The plot at the top of this page is Hurricane Ian as twenty models saw it at 5 PM on September 26, 2022. Ian came ashore in southwest Florida two days later. All twenty start from the storm's position, and the lines leave together and fan out. The fan is the spike spreading. Near the start the lines agree because most of those models are fed the same readings of the atmosphere, the same satellites, buoys and aircraft. Part of the agreement is the shared data talking, so a tight cluster can make the uncertainty look smaller than it is.

Some questions decay slower than others

The curve depends on what you ask. Forecasters are far better at where a storm goes than at how strong it gets. Track forecasts have improved steadily for decades, and intensity forecasts have crept along. The worst case is rapid intensification, a gain of 35 miles per hour in a day, and until about ten years ago the models had little skill at it. Otis, in 2023, is the famous miss. A day before landfall the official forecast said Category 1 for Acapulco. Otis came ashore as a Category 5 with 165 mile-per-hour winds, after gaining about 110 miles per hour in 24 hours. Same storm, same models, two different curves.

The oldest models in the room

I talked with a day trader this week who's using language models to write trading software. I've always stayed away from anything that looks like gambling, partly because of the math, so I asked a lot of questions. My understanding is that day traders love "stochastics," indicators built from recent price moves, and from where I sit they look like features in a model. The goal is a prediction machine that calls one thing, a price, right a little more than half the time, over as short a stretch as possible. It gets fed as much as it can hold about the state of the world and how that world has behaved before. A short stretch is where the spike is still tall.

What struck me was the order of things. He's using a model to build models, and the models he's building are much older than the one doing the building. The language model writes the code. The predictions come from hand-built models of a kind finance has used for a long time.

Those older models have had decades to work on the question everyone's asking right now, and they still can't say when the bubble bursts. Just about everyone expects a repricing, and just about everyone is trying to time it with expensive models, most of them hand-coded and not trained. A spaghetti plot of those timing forecasts would look like one for a storm still a week away, lines fanned across the map. The burst sits far out on the curve, where it's flat.

AI is a model too

Most people treat AI as a new kind of thing, one that gets around the limits every earlier model ran into. That's the gambler's belief in a system, the idea that there's a way to make the call before it happens.

I understand how people get there. A language model predicts tokens. It takes the words so far and guesses the next one, and it does that well enough to write, translate and code. Nothing before it did that, and a new property is easy to mistake for magic. Arthur C. Clarke's old line says a sufficiently advanced technology is indistinguishable from it.

The curve still applies. Take a model that gets each step of a job right 99 times in 100. Chain 69 steps and the odds of finishing the job correctly have fallen to a coin flip. At 999 in 1,000, the coin flip arrives at about 700 steps. A tenfold gain in reliability buys a job ten times longer, and every job still ends at the flat part of the curve.

Builders see the same curve from the inside. The error of a language model falls as a power of the compute spent. Each tenfold increase removes a smaller slice than the last, and what's left is set by the thing being predicted. I covered that floor in Intelligence Is Prediction, and Prediction Has a Ceiling. Better models help at the near end. A learned model from Google DeepMind joined the field partway through the 2025 hurricane season. The Hurricane Center's own verification report found it the one model that slightly beat the official forecast on Atlantic track, at 12 to 72 hours. The curve is still there past that.

One difference: the boom reads its forecasts

A hurricane follows the air, whatever the forecast says. The AI boom listens. A forecast of fast progress draws the capital that produces fast progress, and a forecast of a crash moves the money that causes one. A stock market model is the closer comparison, since a market model can move the market it models.

The forecasters are inside the boom too. A hurricane forecaster gets nothing from a bigger storm. A model builder profits when the model looks powerful, and a forecast that it could end the world makes it look very powerful. That puts a catastrophe forecast from a builder in the same class as a sales forecast. It calls for extra suspicion of every forecast, this essay's included.

What forecasters do about the flat part

Hurricane forecasters plan around the curve. They draw the cone so it admits it will miss one time in three. They score every model against what the storm did, so a new model gets judged next to the old ones. Past five days they publish odds, and they tie each watch and warning to an action agreed in advance, which is why coastal towns draw their evacuation zones in the spring. A company's version is a trigger on paper: when a model passes our acceptance test on this task at this price, we rebuild the process.

Forecasts of the AI boom can be scored the same way. A ledger lists who forecast what, with a date and a number, checked a year later. Someone who called the last three jumps has a record. So does someone who's called a crash every year since the first chatbot, and it's just as informative.

Book a time with us

Trouble with the scheduler? Open it in a new tab.