Calibration check
When the model says 70%, does it happen 70% of the time? Every graded pick is grouped by its predicted probability and compared with how often that group was actually right.
Reliability diagram
Each point is one probability band. Horizontal position: the average probability the model gave. Vertical position: the share of those picks that were correct. Points on the dashed diagonal are perfectly calibrated; above it the model was too cautious, below it too confident. Higher bands beat lower ones in 7 of 7 steps.
Observed hit rate (point size = number of picks) Perfect calibration
| Band | Picks | Predicted | Observed | Gap |
|---|---|---|---|---|
| Under 55% | 2059 | 52% | 52% | +0 |
| 55–59% | 3152 | 57% | 53% | -4 |
| 60–64% | 5987 | 62% | 58% | -4 |
| 65–69% | 5004 | 67% | 66% | -1 |
| 70–74% | 2670 | 72% | 73% | +1 |
| 75–79% | 2628 | 77% | 76% | -1 |
| 80–84% | 2020 | 82% | 77% | -5 |
| 85+% | 973 | 86% | 79% | -7 |
Calibration by market
| Market | Picks | Predicted | Observed |
|---|---|---|---|
| Both teams to score: yes | 7455 | 61% | 57% |
| Match result (1X2) | 5931 | 65% | 62% |
| Double chance | 3117 | 83% | 77% |
| Over 2.5 goals | 2263 | 65% | 68% |
| Over 1.5 goals | 2081 | 75% | 75% |
| Under 3.5 goals | 1755 | 73% | 72% |
| Under 2.5 goals | 1259 | 61% | 59% |
| Both teams to score: no | 598 | 53% | 56% |
| Over 3.5 goals | 32 | 63% | 53% |
Weekly drift
Why we publish this
Hit rate alone can flatter any tipster: back only short favourites and you will be right most of the time while losing money. Calibration asks a harder question – are the probabilities themselves honest? If the 60% band lands 60% of the time and the 80% band 80%, the numbers can be trusted when you compare them with bookmaker odds.
Single weeks wobble; the 90-day view is the one to judge. Profit at level stakes, market by market, is on the leaderboard, every graded pick on the track record, and the method in our methodology.
Calibration: do our tips win as often as we say?
Any tipster can claim to be accurate. Calibration is how you check. We take every settled tip, group them by the chance we gave them – 50–60%, 60–70%, 70–80% and so on – and compare that chance with how often each group actually won. If our 70% tips win about 70% of the time, our numbers are honest. If they win 60%, we are overconfident; if they win 80%, we are too cautious.
Reading the chart
Each point shows one group of tips. The diagonal line is perfect calibration – where the chance we gave equals the rate at which they won. Points close to the line mean the percentages beside our tips can be trusted. Points well below it mean we have been promising too much in that range.
Why this matters to you
Everything useful about a tip – its fair odds, whether a price is value, how much to stake – depends on the chance being right. A tipster who says 80% when the truth is 65% will lead you to back prices that look generous but are not. Calibration is the difference between a number you can bet with and a number that only looks impressive.
Small samples, honest limits
Groups with only a few dozen tips will wander away from the line by chance alone. The figures become meaningful as hundreds of tips are settled in each group, so read the chart with the sample sizes in mind, and expect it to tighten over time.
What we do with the results
If a range is consistently off, the ratings and the weight we give to bookmaker prices are adjusted. You can follow the effect in the prediction archive and on the market profit table. Then judge for yourself. 18+.
Worked example: reading one band
Suppose the 70–80% band holds 900 settled tips with an average stated chance of 74.5%, and 652 of them won – 72.4%. The gap is 2.1 points. With 900 tips, random variation alone could easily produce a gap of two or three points, so this band is well calibrated. If the same band showed 61% from 900 tips, the gap would be far beyond chance, and we would be overrating our strongest tips – something you would want to know before trusting a "74%" label.
Calibration checklist
- Look at sample sizes first.
- Small gaps in big bands are normal.
- Consistent gaps in one direction are the real warning.
Quick answers
What does "overconfident" mean? That tips in a band win less often than the chance we gave them.
Is perfect calibration possible? Not exactly – random variation always leaves small gaps. The goal is no consistent gap in one direction.
How many tips does a band need? A few hundred before small gaps mean much.
Does calibration measure profit? No – it measures honesty of the percentages. Profit is on the market profit table.
Where can I learn more? Our value betting lesson explains why honest percentages matter so much for finding value.
Frequently asked questions
What does calibration mean?
A model is calibrated when its probabilities match reality: of all the picks it rates at 70%, about 70% should be correct. Calibration is the fairest single test of a prediction model.
Is Winning Arena calibrated?
Across 24493 graded picks in the last 90 days, the mean predicted probability was 67% and the observed hit rate 64% – a difference of -3 points. The band-by-band table shows where the model runs hot or cold.
Why can a band look off?
Small samples. A band with a dozen picks can miss its target by 15 points through luck alone. Trust the bands with hundreds of picks most.
How are picks graded?
On the 90-minute score. Extra time and penalties are excluded; postponed or abandoned games are void and left out of the test.