ST Model Accuracy
Out-of-sample backtests of the ST Model — projections vs. what actually happened, and the game model vs. real outcomes
Pitcher projections
Beat the Marcel baseline by 6% on full-season accuracy · p10–p90 coverage 82% (2025 backtest, 1,127 preds)
Game model vs results
Brier 0.2448 vs home-field base 0.2489 · picks the winner 56% (17,881 games)
Player Accuracy
per 700 BF vs Marcel · 1,127 predsskill 6%p10–p90 cov 82%
| Stat | N | Model | Base | Skill | Cov |
|---|---|---|---|---|---|
| Rate Stats | |||||
| ERA | 161 | 0.78 | 0.83 | +6% | 71% |
| WHIP | 161 | 0.11 | 0.13 | +11% | 91% |
| Strikeouts & Command | |||||
| Strikeouts | 161 | 17.22 | 18.07 | +5% | 84% |
| Walks | 161 | 8.3 | 8.12 | -2% | 91% |
| Run Prevention | |||||
| Earned Runs | 161 | 12.59 | 13.47 | +7% | 78% |
| Hits Allowed | 161 | 11.84 | 13.62 | +13% | 80% |
| HR Allowed | 161 | 5.1 | 5.18 | +2% | 80% |
Game Model — Out-of-Sample Backtest
backtest
| Season | Games | Brier | Base | Win% | Margin MAE | Total MAE | Vegas MAE | Beat% |
|---|---|---|---|---|---|---|---|---|
| 2025 | 2,425 | 0.2445 | 0.2482 | 56% | 3.55 | 3.7 | — | — |
| 2024 | 2,427 | 0.2455 | 0.2495 | 55% | 3.47 | 3.6 | — | — |
| 2023 | 2,429 | 0.2456 | 0.2496 | 56% | 3.48 | 3.7 | — | — |
| 2022 | 2,424 | 0.2430 | 0.2489 | 57% | 3.36 | 3.9 | — | — |
| 2021 | 2,426 | 0.2455 | 0.2486 | 56% | 3.47 | 3.8 | — | — |
| 2020 | 895 | 0.2449 | 0.2475 | 56% | 3.61 | 3.8 | — | — |
| 2019 | 2,427 | 0.2443 | 0.2491 | 56% | 3.56 | 3.9 | — | — |
| 2018 | 2,428 | 0.2449 | 0.2492 | 55% | 3.45 | 4.0 | — | — |
| Overall | 17,881 | 0.2448 | 0.2489 | 56% | 3.48 | 3.8 | — | — |
The model's pre-game win probability is scored by Brier against every completed game's outcome; a Brier below the home-field Basemeans it adds signal beyond "home teams win." Margin/Total MAE measure the projected score. Vegas MAE / Beat% read — for this sport: the free odds feed carries no historical closing lines yet, so those columns populate as line history accrues.