Rankings · Translators' guild · Czech ⇄ English
Czech translators graded every AI.
Blind comparisons by people who do this work. Each saw a real task and two answers without names, picked the one they'd hand in, and fixed it. Updated with every judgment.
Grade them yourselfHow the score works
The Bradley–Terry model, which public AI leaderboards also use and chess ratings are close cousins of: from who beat whom, it estimates how likely each model is to beat each other one. 1,000 is the average of this lineup. A tie is half a win for both. “Neither is good enough” doesn't move the score; it shows up as a rejection.
Who counts
By default, people who said they do this work for a living or as part of their job. Judgments made in under 4 seconds are left out: nobody reads two answers that fast. Every judgment's fingerprint is on the public ledger.
The columns that matter
“Ready as is”: when practitioners picked a model's answer, how often they'd hand it in without changing a word. “Rewritten”: how much of it they changed, typically. “Rejected”: how often neither answer was good enough. A model can win often and still need a lot of fixing.