ST Model Accuracy
Out-of-sample backtests of the ST Model's calibrated engine — projections vs. what actually happened, and vs. Vegas closing lines
Player projections
Beat the naive baseline by 6% on per-game accuracy · p10–p90 coverage 78% (2025 backtest, 30,322 preds)
Team margins vs Vegas
Model MAE 13.34 vs Vegas 12.2 · beats the line 50% (5,297 games)
Player Accuracy
per-game rate vs naive · 30,322 predsskill 6%p10–p90 cov 78%
| Stat | N | Model/g | Base/g | Skill | Cov |
|---|---|---|---|---|---|
| Passing | |||||
| Pass Yards | 530 | 64.2 | 69.28 | +7% | 81% |
| Pass TDs | 530 | 0.52 | 0.57 | +8% | 85% |
| Pass INTs | 530 | 0.28 | 0.3 | +6% | 76% |
| Pass Att | 530 | 8.24 | 8.87 | +7% | 81% |
| Completions | 530 | 5.21 | 5.64 | +8% | 80% |
| Rushing | |||||
| Rush Yards | 1556 | 16.58 | 17.6 | +6% | 79% |
| Rush TDs | 1556 | 0.22 | 0.23 | +7% | 76% |
| Rush Att | 1556 | 3.02 | 3.22 | +6% | 78% |
| Receiving | |||||
| Receptions | 3106 | 0.78 | 0.85 | +8% | 78% |
| Rec Yards | 3106 | 10.65 | 11.69 | +9% | 79% |
| Rec TDs | 3106 | 0.12 | 0.13 | +7% | 75% |
| Defense | |||||
| Tackles | 2189 | 1.1 | 1.14 | +3% | 79% |
| Solo Tackles | 2189 | 0.67 | 0.68 | +1% | 80% |
| Tackles for Loss | 2189 | 0.17 | 0.18 | +2% | 79% |
| Sacks | 2189 | 0.09 | 0.09 | +6% | 72% |
| Passes Defended | 2189 | 0.13 | 0.14 | +2% | 80% |
| QB Hurries | 2189 | 0.11 | 0.12 | +5% | 75% |
| Kicking | |||||
| FG Made | 184 | 0.34 | 0.36 | +6% | 83% |
| FG Att | 184 | 0.38 | 0.4 | +5% | 79% |
| XP Made | 184 | 0.81 | 0.89 | +8% | 81% |
Game Model — Out-of-Sample Backtest
backtest
| Season | Games | Brier | Base | Win% | Margin MAE | Total MAE | Vegas MAE | Beat% |
|---|---|---|---|---|---|---|---|---|
| 2025 | 809 | 0.1883 | 0.2408 | 72% | 13.03 | 13.7 | 11.82 | 52.7% |
| 2024 | 803 | 0.1957 | 0.2425 | 67% | 13.07 | 13.9 | 12.06 | 50.5% |
| 2023 | 797 | 0.1862 | 0.2437 | 72% | 12.98 | 14.1 | 12.11 | 50.8% |
| 2022 | 784 | 0.1997 | 0.2437 | 68% | 13.28 | 14.6 | 12.05 | 48.1% |
| 2021 | 781 | 0.2018 | 0.2440 | 68% | 14.01 | 13.9 | 12.58 | 46.6% |
| 2020 | 538 | 0.1921 | 0.2460 | 71% | 13.79 | 14.5 | 12.87 | 53.0% |
| 2019 | 785 | 0.1756 | 0.2410 | 73% | 13.37 | 14.7 | 12.17 | 49.3% |
| Overall | 5,297 | 0.1913 | 0.2431 | 70% | 13.34 | 14.2 | 12.20 | 50.0% |
The model's pre-game win probability is scored by Brier against every completed game's outcome; a Brier below the home-field Basemeans it adds signal beyond "home teams win." Margin/Total MAE measure the projected score. Vegas MAE / Beat% benchmark the projected margin against the closing line — the model never sees the line, it's stored only to grade honestly.