Can You Trust a 70% Forecast? Calibration and Brier Scores Explained

A probability needs a clearly defined event

When software displays “Banker 70%,” the first question is what that statement means. Is it predicting a Banker win among all Banker, Player and Tie outcomes? Has Tie been excluded? Is the label actually saying Banker-or-Tie rather than Banker alone?

An unlabeled percentage is not an auditable forecast. The event definition, conditioning information and time of prediction must all be recorded before a later result can be evaluated.

Calibration compares stated probabilities with frequencies

Suppose a model gives roughly seventy-percent probabilities to a large group of future binary events. If the event occurs in only forty-five percent of those cases, the model is poorly calibrated. A well-calibrated predictor should produce observed frequencies near the probabilities it assigns, across sufficiently large, appropriately grouped evaluation samples.

One incorrect-looking outcome does not settle the question. If a particular event has a predicted probability of seventy percent, a non-event remains possible. Calibration is evaluated over many observations, not one hand.

The binary Brier Score is a simple starting point

For a binary event, let p be the forecast probability and y equal one if the event occurs and zero otherwise. The Brier Score for the observation is (p − y)^2, with lower scores indicating better forecasts.

If p is 0.7 and the event occurs, the score is 0.09. If the event does not occur, the score is 0.49. Averaging over an untouched evaluation sample provides a way to compare models under the same definition.

Baccarat naturally has three result categories. A full multiclass forecast can be scored by summing squared differences between the predicted probabilities and the one-hot actual outcome across all three classes. Reports must specify their normalization convention because different software may scale the resulting score differently.

Compare with a transparent baseline

A complicated model should be compared against a simple rule-based baseline: the theoretical outcome frequencies under the same baccarat rules and payout assumptions. Both models must be evaluated on identical samples with identical scoring definitions.

Calibration and discrimination are not the same. A predictor may provide probabilities close to the overall base rates yet fail to distinguish one case from another. Conversely, it may rank cases differently while systematically overstating confidence. A good report investigates both.

Profit screenshots are not probability validation

A short profit curve mixes payouts, amount wagered, selected time periods and chance variation. It does not establish that stated probabilities are well calibrated. Even a model with a respectable probabilistic score would not automatically overcome the game's mathematical costs.

A credible analytics application should disclose timestamped forecasts, unchanged out-of-sample outcomes, Brier scores, reliability plots and the baseline used for comparison. Forecasts should be archived before outcomes are known.

This is a guide to evaluating probability claims, not advice about placing wagers.