ST Model Accuracy
Out-of-sample backtests of the ST Model — projections vs. what actually happened, and the game model vs. real outcomes
Batter projections
Beat the Marcel baseline by 3% on full-season accuracy · p10–p90 coverage 80% (2025 backtest, 2,772 preds)
Game model vs results
Brier 0.2448 vs home-field base 0.2489 · picks the winner 56% (17,881 games)
Player Accuracy
per 650 PA vs Marcel · 2,772 predsskill 3%p10–p90 cov 80%
| Stat | N | Model | Base | Skill | Cov |
|---|---|---|---|---|---|
| Power | |||||
| Home Runs | 308 | 6.19 | 6.17 | -1% | 85% |
| Doubles | 308 | 5.24 | 5.22 | -0% | 82% |
| Triples | 308 | 1.54 | 1.64 | +6% | 83% |
| Run Production | |||||
| Runs | 308 | 10.15 | 10.6 | +4% | 82% |
| RBI | 308 | 13.08 | 13.1 | +0% | 74% |
| Contact & Speed | |||||
| Hits | 308 | 11.72 | 12.22 | +4% | 83% |
| Stolen Bases | 308 | 5.1 | 5.52 | +8% | 82% |
| Plate Discipline | |||||
| Walks | 308 | 10.04 | 10.19 | +2% | 77% |
| Strikeouts | 308 | 19.14 | 19.43 | +2% | 76% |
Game Model — Out-of-Sample Backtest
backtest
| Season | Games | Brier | Base | Win% | Margin MAE | Total MAE | Vegas MAE | Beat% |
|---|---|---|---|---|---|---|---|---|
| 2025 | 2,425 | 0.2445 | 0.2482 | 56% | 3.55 | 3.7 | — | — |
| 2024 | 2,427 | 0.2455 | 0.2495 | 55% | 3.47 | 3.6 | — | — |
| 2023 | 2,429 | 0.2456 | 0.2496 | 56% | 3.48 | 3.7 | — | — |
| 2022 | 2,424 | 0.2430 | 0.2489 | 57% | 3.36 | 3.9 | — | — |
| 2021 | 2,426 | 0.2455 | 0.2486 | 56% | 3.47 | 3.8 | — | — |
| 2020 | 895 | 0.2449 | 0.2475 | 56% | 3.61 | 3.8 | — | — |
| 2019 | 2,427 | 0.2443 | 0.2491 | 56% | 3.56 | 3.9 | — | — |
| 2018 | 2,428 | 0.2449 | 0.2492 | 55% | 3.45 | 4.0 | — | — |
| Overall | 17,881 | 0.2448 | 0.2489 | 56% | 3.48 | 3.8 | — | — |
The model's pre-game win probability is scored by Brier against every completed game's outcome; a Brier below the home-field Basemeans it adds signal beyond "home teams win." Margin/Total MAE measure the projected score. Vegas MAE / Beat% read — for this sport: the free odds feed carries no historical closing lines yet, so those columns populate as line history accrues.