The Brier Score in Football: How to Judge a Probability Without Calling It a Promise

The Brier score measures the error of a probability forecast by comparing its stated probabilities with what happened. It rewards accurate confidence across a collection of events. A result on RubiScore can supply the outcome for a football evaluation, but that outcome alone cannot tell whether the original confidence was sensible.

A match record at https://rubiscore.com and a forecast saved before kick-off serve different purposes. The record establishes the result; the saved forecast establishes what the forecaster believed beforehand. Keeping both allows an assessment more informative than counting how often the favourite won.

Why is picking the winner an incomplete test?

Imagine two forecasters who both make the home team their most likely winner. One considers the match close. The other thinks a home victory is almost certain. A simple right-or-wrong test treats their selections identically, although their beliefs are very different.

That distinction matters when the match is drawn. The cautious forecaster allowed substantial room for another outcome. The confident forecaster did not. A useful probability assessment should notice the difference rather than erase everything except the largest number.

Glenn Brier introduced his probability-verification approach in meteorology. The underlying task also appears in football: compare a forecast distribution with an observed event. The method evaluates the numerical forecast, rather than requiring the most likely outcome to occur every time. Brier's original paper.

It also prevents a misleading description of uncertainty. A forecast can assign the largest share to a home win while assigning more probability to all other outcomes combined. Calling that prediction a promise of victory changes its meaning before the evaluation has even begun.

How does the arithmetic work?

For a binary event, such as whether the home team wins, write the predicted probability as a number between zero and one. Write the observed outcome as one if the event occurs and zero otherwise. Subtract the outcome from the probability, square the difference, and average those squared differences across the matches.

Here is an invented arithmetic example, not a prediction or a recorded football result. A home-win probability of 0.70 receives an error of 0.09 if the home team wins: the difference between 0.70 and one is squared. If the team does not win, the error is 0.49.

The unequal penalties are intentional. Being confidently wrong creates a larger error than being cautiously wrong. Being confidently right creates a smaller error than being cautiously right. Across repeated forecasts, the calculation gives the confidence level a meaningful role.

Zero is the best possible binary score. However, a low score on one match should not be mistaken for evidence of a strong forecasting method. Any uncertain event can produce an outcome that flatters a poor forecast or punishes a reasonable one.

What changes when football has three outcomes?

A standard match-result forecast has home win, draw and away win as mutually exclusive categories. Their probabilities should add to one. The observed result is represented by one in the category that occurred and zero in the other categories.

One common multicategory convention squares the error for each category, sums those errors for the match, and then averages across matches. Under that convention, a perfect forecast scores zero and the worst possible forecast scores two. Other presentations rescale the result, so the published convention must be stated.

For another purely hypothetical calculation, assign probabilities of 0.50, 0.30 and 0.20 to home win, draw and away win. If the match is drawn, the category errors are 0.25, 0.49 and 0.04. Their sum is 0.78. These numbers illustrate the formula; they are not estimates about any real teams.

A binary home-win score and a three-outcome score are therefore not interchangeable. Nor should a score divided by the number of categories be compared directly with an unscaled sum. Before comparing model quality, establish which calculation produced the headline value.

What does the score reward over many matches?

Probability evaluation concerns a sequence of decisions. A model that repeatedly gives strong confidence to outcomes that do not happen will accumulate large errors. A model that distinguishes easier fixtures from more uncertain fixtures can improve on a forecast that says the same thing every time.

Calibration is one part of that story. When a model regularly assigns a similar probability to an event, the observed frequency in that group should broadly match that probability over an adequate sample. But calibration alone does not establish that the model separates different types of match usefully.

Research on football forecast verification examines Brier scores alongside reliability and discrimination analyses precisely because a single aggregate score conceals different behaviours. The practical implication is to inspect the forecast distribution as well as its overall error. Football probability-verification research.

For a reader, a useful companion question is whether the model recognises uncertainty where it should. A method that treats balanced contests as obvious victories may occasionally look decisive, while producing a poor long-term assessment of how football actually varies.

Why does the comparison baseline matter?

A score becomes easier to interpret when it is compared with a simple forecast made under the same conditions. An example baseline might use historical outcome frequencies from an appropriate training period. It must be built without looking at the results being tested.

The comparison should use the same fixtures, outcome definitions and scoring convention. A model evaluated on an easier collection of matches can look better without having a better forecasting method. Comparing unrelated headline scores hides that possibility.

RubiScore can help identify the competition and completed fixtures used in a reader's evaluation. The forecasting probabilities, their timestamps and the scoring calculation still need to be supplied separately. A football results platform should not be credited with producing a model assessment that the analyst performed.

There is also no universal score that marks a model as good in every setting. The distribution of outcomes and the difficulty of the evaluated fixtures affect what a reasonable baseline can achieve. Improvement over a declared reference is more interpretable than an isolated decimal.

Which mistakes make an evaluation look stronger than it is?

The scoring formula is short; the evaluation design is where avoidable errors accumulate. Check the following before trusting a comparison:

• Forecasts were saved before the information cutoff relevant to the exercise.
• Every eligible fixture was retained, including surprising results.
• Home win, draw and away win use the same match-duration definition throughout.
• Models were tested on the same matches and in the same probability scale.
• Missing forecasts were reported rather than silently removed.
• The simple comparison forecast was built without using test results.

The duration definition deserves particular attention in knockout football. A draw after regulation time and a team advancing after extra time are different outcomes. A probability for one cannot be scored against the other simply because the match page shows a winner.

Selective reporting is another problem. Publishing only the strongest league, month or model version after examining every result makes the displayed performance difficult to reproduce. Keep the full evaluation plan and distinguish exploratory findings from a genuinely separate test.

What cannot the Brier score decide?

The score does not explain why a forecast was wrong. A bad result could reflect weak inputs, an inappropriate model, a probability that needed recalibration or ordinary uncertainty in a small sample. The numerical error is evidence to investigate, not a complete diagnosis.

It also cannot establish the usefulness of every decision made from the forecast. Different decisions may carry different consequences, whereas the scoring rule evaluates probability quality according to its own mathematical structure. Choosing an evaluation rule and choosing an action are separate tasks.

Research comparing football scoring rules makes this distinction visible: different rules respond differently to the probabilities assigned across the available outcomes. That is a reason to disclose the rule and examine complementary diagnostics, rather than select whichever metric makes a preferred model look strongest. Research comparing football scoring rules.

The strongest use of a Brier score is modest and specific. Preserve the forecast, define the outcome, disclose the calculation, compare like with like and examine a substantial sequence of matches. The result then tells the reader something about the quality of stated uncertainty, which a tally of correct winners cannot provide.