Home / Method
How we keep score
We back open AI and we say so. The scoreboard only means something if the rules are written down and closed can win under them.
The categories
One point per category, to whichever side leads on the stated measure. A category changes hands when the measure does, not when we feel like it.
| Category | What we measure | Leader now |
|---|---|---|
| Ability | How many months the best open model is behind the best closed one. Closed leads as long as every published estimate is above zero. | Closed |
| Price | Price for the same amount of work. Open leads while the typical open model is cheaper. | Open |
| Control | Whether you can hold and run the AI yourself. This is built into the definitions, so it doesn't change. | Open |
| Usage | Share of all AI work done on OpenRouter. Open leads above 50%. | Open |
| Money | Share of dollars spent on OpenRouter. Closed leads above 50%. | Closed |
| Fixing mistakes | Whether a flawed AI can be fixed after release. Built into the definitions, like control. | Closed |
| Washington | Where U.S. policy is heading. No point until something becomes law. | No point |
The score over time
The tally has changed once this year.
Closed led on ability, usage, money and fixing mistakes. Open led on price and control.
Usage flipped. Open models started doing more than half the work on OpenRouter in April and have stayed there.
Open models first did more than half the work on OpenRouter briefly in February, and have stayed above half since April. Source: OpenRouter.
What counts as open
An AI model is open if anyone can download it and run it themselves, even if it comes with some conditions. It is closed if you can only use it through its maker's app or website. This is the line most researchers use. Our grades show that some open AI comes with more strings than others.
How plays are tagged
- Point · Open when something makes open AI easier to get, better, or safer from being banned.
- Point · Closed when a move restricts open models, widens the closed lead, or strengthens the case against open AI.
- Split when one move does both, or when each side can fairly claim it.
Plays don't add up to the score. They explain why the categories move.
Sources and corrections
Every number links to where it came from, and the full source list is public. We prefer the original document to a news story about it. If we got something wrong, tell us and we'll fix it and say that we did.
Who pays for this
Open vs Closed is independent. No AI company funds it.
Use the data
Everything on this site comes from one file. Take it: data.json. There is also a feed of plays and an embeddable scoreboard.