Daily verification · methodology v0.3.1

How far off was each weather model?

No group in this view yet has enough scored days to rank (n < 30). Every number below is still published, greyed out, with its sample size.

Scope: raw model output on the native 0.25° grid. This view scores the sampled daily maximum against the maximum of the four *observed* samples on the same day. Errors are computed in °C and shown in °F (methodology v0.3.1).

Window
the last 30 days
Variable
sampled daily maximum
Init
12Z
Interpolation
bilinear
Data through
2026-08-30
Updated
Next update

Leader, lead day 1

no ranked group with an interval yet

n < 30 everywhere

Persistence baseline

2.83°F

[2.48, 3.22]

yesterday's observation — the bar every model has to clear

Scored days

30

23 stations · 12 systems

in the last 30 days, lead day 1

Systems ranked

0 of 12 ranked yetn < 30

next update

below n = 30: published, greyed, unranked

Mean absolute error with 95 % confidence intervals

Lead day 1 · the last 30 days · 23 stations pooled · 12Z · bilinear · sampled daily maximum. Whiskers are moving-block bootstrap intervals; models whose whiskers overlap are not distinguishable at this sample size. Baselines are drawn in grey and never ranked.

MAE by model with 95 % confidence intervals, lead day 1, 30d, 12Z, bilinear012345mean absolute error, °FECMWF AIFS SingleECMWF AIFS Single: 2.04 °F2.04ECMWF IFS HRESECMWF IFS HRES: 2.19 °F2.19Pangu-Weather (IFS init)Pangu-Weather (IFS init): 2.46 °F2.46FourCastNet v2 (GFS init)FourCastNet v2 (GFS init): 2.55 °F2.55FourCastNet v2 (IFS init)FourCastNet v2 (IFS init): 2.6 °F2.6Persistence (baseline)Persistence (baseline): 2.83 °F2.83Pangu-Weather (GFS init)Pangu-Weather (GFS init): 2.93 °F2.93GraphCast (IFS init)GraphCast (IFS init): 3 °F3NCEP GFSNCEP GFS: 3.19 °F3.19Aurora (IFS init)Aurora (IFS init): 3.46 °F3.46GraphCast (GFS init)GraphCast (GFS init): 3.72 °F3.72Aurora (GFS init)Aurora (GFS init): 4.24 °F4.24

Source: CastCheck 0.1.0, methodology v0.3.1 · ECMWF open data, NOAA/NCEP GFS, NOAA/CIRA AIWP, NWS ASOS · data through 2026-08-30 · CC BY 4.0. Hover a bar for its value.

Lead day 1target = init + 1 d

permalink

The last 30 days, all 23 stations pooled, 12Z initialization, bilinear interpolation, sampled daily maximum against the maximum of the four *observed* samples on the same day. Lower MAE is better; bias is positive when the model is too warm. Every number links to its permanent page.

Skill is 1 − MAE ÷ MAE(persistence) computed on the days both have a value — the small print in that column is that denominator and the size of the intersection, so the number can be checked against the persistence row rather than contradicting it. The persistence row's own n is its whole record, marked all days. Skill, debiased (out-of-sample) repeats the score after removing a bias estimated on the 30 scored days before each day and applied forward, never on the day itself. An interval reads when the bootstrap could not run: fewer than 28 scored days or fewer than 4 blocks. Hit-rate intervals are Wilson score intervals, not bootstraps. in the rank column means fewer than 30 scored days, so the group is published but not ranked.

Lead day 1, the last 30 days, 12Z, bilinear, sampled daily maximum.
RankModelMAE °FBias °F±3 °F SkillSkill, debiased (out-of-sample) nvs leader
ECMWF AIFS Singleaifs_single2.04[1.98, 2.11]−0.38[−0.49, −0.27]78% +0.27[+0.18, +0.34]vs 2.80 (n=29)+0.48n=14 2922.7 stns
ECMWF IFS HRESifs_hres2.19[2.07, 2.31]−0.52[−0.64, −0.40]75% +0.22[+0.13, +0.28]vs 2.80 (n=29)+0.34n=14 2922.7 stns
Pangu-Weather (IFS init)pangu_ifs2.46−0.4267% vs 3.48 (n=2)n=0 219 stns
FourCastNet v2 (GFS init)fourcastnet_gfs2.55−1.0977% vs 3.48 (n=2)n=0 219 stns
FourCastNet v2 (IFS init)fourcastnet_ifs2.60−1.3157% vs 3.48 (n=2)n=0 219 stns
Persistence (baseline)persistence · baseline2.83[2.48, 3.22]+0.01[−0.22, 0.26]63% 30all days
Pangu-Weather (GFS init)pangu_gfs2.93−0.5856% vs 3.48 (n=2)n=0 219 stns
GraphCast (IFS init)graphcast_ifs3.00[2.84, 3.14]−2.53[−2.73, −2.32]54% −0.08[−0.21, +0.05]vs 2.79 (n=28)+0.41n=28 2822.6 stns
NCEP GFSgfs3.19[2.97, 3.37]+1.88[1.57, 2.20]55% −0.14[−0.31, −0.03]vs 2.80 (n=29)+0.30n=14 2922.7 stns
Aurora (IFS init)aurora_ifs3.46−3.3043% vs 3.48 (n=2)n=0 219 stns
GraphCast (GFS init)graphcast_gfs3.72−3.3746% vs 3.48 (n=2)n=2 219 stns
Aurora (GFS init)aurora_gfs4.24−4.1137% vs 3.48 (n=2)n=0 219 stns

+ model too warm − model too cold bias interval includes zero ★ lowest MAE = not distinguishable from the leader ▼ worse than the leader ▲ better than the leader — all three Holm-corrected within this table, over the family of comparisons against the leader; the uncorrected verdict for every pair is published in the pairwise table of each permanent link and in pairwise_latest.csv.gzn < 30 greyed and unranked CI is a 95 % moving-block bootstrap interval on the group's own days, and reads — when it could not be computed skill reads — when fewer than 10 days are common to the model and the baseline: the ratio of two means over a handful of shared days is not a number worth printing

Lead day 3target = init + 3 d

permalink

Same sampling as lead day 1. Skill is 1 − MAE ÷ MAE(persistence) on the days both have a value.

Lead day 3, the last 30 days, 12Z, bilinear, sampled daily maximum.
RankModelMAE °FBias °FSkillnvs leader
ECMWF AIFS Singleaifs_single2.20−0.52+0.43vs 3.84 (n=27)2722.6 stns
ECMWF IFS HRESifs_hres2.41−0.19+0.37vs 3.84 (n=27)2722.6 stns
GraphCast (IFS init)graphcast_ifs3.22[2.97, 3.44]−2.60[−2.89, −2.33]+0.17[+0.05, +0.28]vs 3.89 (n=28)2822.9 stns
NCEP GFSgfs3.50+1.97+0.09vs 3.84 (n=27)2722.6 stns
Persistence (baseline)persistence · baseline3.90[3.60, 4.19]−0.01[−0.52, 0.59]30all days

Lead day 5target = init + 5 d

permalink

Same sampling as lead day 1. Skill is 1 − MAE ÷ MAE(persistence) on the days both have a value.

Lead day 5, the last 30 days, 12Z, bilinear, sampled daily maximum.
RankModelMAE °FBias °FSkillnvs leader
1GraphCast (IFS init)graphcast_ifs3.54[3.22, 3.88]−2.64[−2.91, −2.37]+0.12[−0.07, +0.28]vs 4.03 (n=30)3022.6 stns lowest MAE in this view
ECMWF AIFS Singleaifs_single2.49−0.56+0.39vs 4.05 (n=25)2522.6 stns
ECMWF IFS HRESifs_hres2.85+0.08+0.30vs 4.05 (n=25)2522.6 stns
NCEP GFSgfs3.66+1.62+0.10vs 4.05 (n=25)2522.6 stns
Persistence (baseline)persistence · baseline4.03[3.60, 4.48]+0.03[−0.68, 0.80]30all days

Lead day 7target = init + 7 d

permalink

Same sampling as lead day 1. Skill is 1 − MAE ÷ MAE(persistence) on the days both have a value.

Lead day 7, the last 30 days, 12Z, bilinear, sampled daily maximum.
RankModelMAE °FBias °FSkillnvs leader
1GraphCast (IFS init)graphcast_ifs3.84[3.39, 4.34]−2.78[−3.27, −2.32]+0.11[−0.06, +0.25]vs 4.29 (n=30)3022.6 stns lowest MAE in this view
ECMWF AIFS Singleaifs_single2.86−1.09+0.33vs 4.26 (n=23)2322.6 stns
ECMWF IFS HRESifs_hres3.11−0.28+0.27vs 4.26 (n=23)2322.6 stns
NCEP GFSgfs4.05+1.54+0.05vs 4.26 (n=23)2322.6 stns
Persistence (baseline)persistence · baseline4.29[4.00, 4.58]+0.04[−0.75, 0.80]30all days

The same four samples as a daily maximum and minimum

Lead day 1, the last 30 days, 12Z, bilinear. Here the forecast's max/min of the four samples is scored against the observation's max/min of the same four instants. Like for like: whatever the four-sample definition misses, it misses on both sides. The comparison against the true NWS daily extremes is a different question and lives on the station and permanent-link pages.

Sampled daily maximumtmax_s

RankModelMAE °FBias °FSkillnvs leader
ECMWF AIFS Singleaifs_single2.04[1.98, 2.11]−0.38[−0.49, −0.27]+0.27vs 2.80 (n=29)29
ECMWF IFS HRESifs_hres2.19[2.07, 2.31]−0.52[−0.64, −0.40]+0.22vs 2.80 (n=29)29
Pangu-Weather (IFS init)pangu_ifs2.46−0.42vs 3.48 (n=2)2
FourCastNet v2 (GFS init)fourcastnet_gfs2.55−1.09vs 3.48 (n=2)2
FourCastNet v2 (IFS init)fourcastnet_ifs2.60−1.31vs 3.48 (n=2)2
Persistence (baseline)persistence2.83[2.48, 3.22]+0.01[−0.22, 0.26]30all days
Pangu-Weather (GFS init)pangu_gfs2.93−0.58vs 3.48 (n=2)2
GraphCast (IFS init)graphcast_ifs3.00[2.84, 3.14]−2.53[−2.73, −2.32]−0.08vs 2.79 (n=28)28
NCEP GFSgfs3.19[2.97, 3.37]+1.88[1.57, 2.20]−0.14vs 2.80 (n=29)29
Aurora (IFS init)aurora_ifs3.46−3.30vs 3.48 (n=2)2
GraphCast (GFS init)graphcast_gfs3.72−3.37vs 3.48 (n=2)2
Aurora (GFS init)aurora_gfs4.24−4.11vs 3.48 (n=2)2

Sampled daily minimumtmin_s

RankModelMAE °FBias °FSkillnvs leader
FourCastNet v2 (IFS init)fourcastnet_ifs2.03−0.02vs 2.54 (n=2)2
FourCastNet v2 (GFS init)fourcastnet_gfs2.11+0.33vs 2.54 (n=2)2
Aurora (GFS init)aurora_gfs2.14−0.20vs 2.54 (n=2)2
ECMWF IFS HRESifs_hres2.18[2.02, 2.35]−0.55[−0.75, −0.35]+0.07vs 2.34 (n=29)29
GraphCast (GFS init)graphcast_gfs2.19+0.18vs 2.54 (n=2)2
Pangu-Weather (GFS init)pangu_gfs2.20+0.15vs 2.54 (n=2)2
Aurora (IFS init)aurora_ifs2.22−0.85vs 2.54 (n=2)2
GraphCast (IFS init)graphcast_ifs2.26[2.14, 2.39]−1.21[−1.39, −1.03]+0.04vs 2.34 (n=28)28
ECMWF AIFS Singleaifs_single2.28[2.09, 2.44]−1.33[−1.54, −1.14]+0.02vs 2.34 (n=29)29
Pangu-Weather (IFS init)pangu_ifs2.30−0.99vs 2.54 (n=2)2
Persistence (baseline)persistence2.34[2.15, 2.51]+0.02[−0.18, 0.21]30all days
NCEP GFSgfs2.40[2.21, 2.61]+0.17[−0.09, 0.45]−0.03vs 2.34 (n=29)29

Every model × every lead day

MAE in °F with the bias and n underneath, the last 30 days, 12Z, bilinear, sampled daily maximum. The sparkline is the same model's MAE across lead days 1–9 on a shared vertical scale.

Modeld1d2d3d4d5d6d7d8d9 lead 1–9
ECMWF AIFS Singleaifs_single2.04−0.38 · n=292.12−0.54 · n=282.20−0.52 · n=272.30−0.46 · n=262.49−0.56 · n=252.83−0.75 · n=242.86−1.09 · n=233.21−1.53 · n=223.73−2.05 · n=21ECMWF AIFS Single MAE by lead day 1 to 9
Aurora (GFS init)aurora_gfs4.24−4.11 · n=25.06−5.05 · n=1 Aurora (GFS init) MAE by lead day 1 to 9
Aurora (IFS init)aurora_ifs3.46−3.30 · n=23.81−3.50 · n=1 Aurora (IFS init) MAE by lead day 1 to 9
FourCastNet v2 (GFS init)fourcastnet_gfs2.55−1.09 · n=22.47−1.83 · n=1 FourCastNet v2 (GFS init) MAE by lead day 1 to 9
FourCastNet v2 (IFS init)fourcastnet_ifs2.60−1.31 · n=22.60−1.67 · n=1 FourCastNet v2 (IFS init) MAE by lead day 1 to 9
NCEP GFSgfs3.19+1.88 · n=293.24+1.86 · n=283.50+1.97 · n=273.54+1.75 · n=263.66+1.62 · n=253.92+1.31 · n=244.05+1.54 · n=234.38+1.36 · n=224.80+1.21 · n=21NCEP GFS MAE by lead day 1 to 9
GraphCast (GFS init)graphcast_gfs3.72−3.37 · n=24.79−4.69 · n=1 GraphCast (GFS init) MAE by lead day 1 to 9
GraphCast (IFS init)graphcast_ifs3.00−2.53 · n=283.12−2.60 · n=283.22−2.60 · n=283.34−2.51 · n=293.54−2.64 · n=303.49−2.54 · n=303.84−2.78 · n=304.32−3.11 · n=304.63−3.22 · n=30GraphCast (IFS init) MAE by lead day 1 to 9
ECMWF IFS HRESifs_hres2.19−0.52 · n=292.30−0.24 · n=282.41−0.19 · n=272.54+0.00 · n=262.85+0.08 · n=253.13+0.25 · n=243.11−0.28 · n=233.42−0.18 · n=223.80−0.63 · n=21ECMWF IFS HRES MAE by lead day 1 to 9
Pangu-Weather (GFS init)pangu_gfs2.93−0.58 · n=22.80−1.06 · n=1 Pangu-Weather (GFS init) MAE by lead day 1 to 9
Pangu-Weather (IFS init)pangu_ifs2.46−0.42 · n=22.70−0.76 · n=1 Pangu-Weather (IFS init) MAE by lead day 1 to 9
Persistence (baseline)persistence2.83+0.01 · n=303.46+0.09 · n=303.90−0.01 · n=303.93+0.04 · n=304.03+0.03 · n=304.19+0.06 · n=304.29+0.04 · n=304.30−0.04 · n=304.50−0.10 · n=30Persistence (baseline) MAE by lead day 1 to 9

Where the errors are

Both maps show a fixed quantity per station. Neither shows which model wins where: on samples this short the per-station winner is mostly noise, and a map of winners would invite a comparison the data cannot carry.

ECMWF IFS HRES bias

Mean bias of the fixed reference model ECMWF IFS HRES (lead day 1, sampled daily maximum, 30d window, 12Z, bilinear). One model everywhere, so the colours compare stations, not models. Dot area grows with the number of scored days; fill is the mean bias (warm = model too warm, cool = too cold, grey = interval includes zero). The frame is a latitude/longitude graticule on an Albers projection.

Source: CastCheck · station coordinates NWS/NOAA · Albers conic projection · stations.csv. Hover a dot for its numbers.

model too cold interval includes zero model too warm dot area ∝ scored days

ECMWF IFS HRES bias as a table
StationModelMAE °FBias °Fn
KATL Atlanta HartsfieldECMWF IFS HRES1.79+0.2929
KAUS Austin BergstromECMWF IFS HRES2.11−1.8329
KBOS Boston LoganECMWF IFS HRES2.48+0.7026
KDCA Washington ReaganECMWF IFS HRES1.95+0.2129
KDEN Denver IntlECMWF IFS HRES2.31+1.4628
KDFW Dallas-Fort WorthECMWF IFS HRES1.51+1.1329
KEWR Newark LibertyECMWF IFS HRES3.37−2.1729
KIAH Houston BushECMWF IFS HRES1.63−0.9429
KLAS Las Vegas Harry ReidECMWF IFS HRES2.79−2.4128
KLAX Los Angeles IntlECMWF IFS HRES1.37+0.3828
KMIA Miami IntlECMWF IFS HRES2.73−1.2629
KMSP Minneapolis-St PaulECMWF IFS HRES2.29−0.7029
KMSY New Orleans IntlECMWF IFS HRES1.92−1.0429
KNYC New York Central ParkECMWF IFS HRES2.50+1.0529
KOKC Oklahoma CityECMWF IFS HRES2.03−0.4729
KORD Chicago O'HareECMWF IFS HRES1.55−0.3829
KPHL Philadelphia IntlECMWF IFS HRES2.24−1.2229
KPHX Phoenix Sky HarborECMWF IFS HRES2.08−1.0628
KSAN San Diego LindberghECMWF IFS HRES3.31−3.3128
KSAT San Antonio IntlECMWF IFS HRES1.21−0.5929
KSEA Seattle-TacomaECMWF IFS HRES2.16+0.8128
KSFO San Francisco IntlECMWF IFS HRES2.59−2.1828
KTTN Trenton MercerECMWF IFS HRES2.36+1.5629

All-model mean bias

Mean bias averaged over every scored model at each station (lead day 1, sampled daily maximum, 30d window, 12Z, bilinear). A station that is cold for all of them is a station property — elevation, coastline, a grid cell that is partly sea — not a model ranking. Dot area grows with the number of scored days; fill is the mean bias (warm = model too warm, cool = too cold). The frame is a latitude/longitude graticule on an Albers projection.

Source: CastCheck · station coordinates NWS/NOAA · Albers conic projection · stations.csv. Hover a dot for its numbers.

model too cold interval includes zero model too warm dot area ∝ scored days

All-model mean bias as a table
StationModelsMAE °FBias °Fn
KATL Atlanta Hartsfield11 models2.29−0.6129
KAUS Austin Bergstrom11 models3.99−3.5129
KBOS Boston Logan11 models2.24+0.6326
KDCA Washington Reagan11 models2.33+0.6029
KDEN Denver Intl11 models1.64−0.5028
KDFW Dallas-Fort Worth11 models2.77−1.6829
KEWR Newark Liberty11 models2.38−0.6129
KIAH Houston Bush11 models1.75−0.5229
KLAS Las Vegas Harry Reid11 models4.56−4.3628
KLAX Los Angeles Intl11 models1.89−0.5728
KMIA Miami Intl11 models3.33−2.5929
KMSP Minneapolis-St Paul11 models3.96−2.8129
KMSY New Orleans Intl11 models2.98−2.7729
KNYC New York Central Park11 models2.38+1.6329
KOKC Oklahoma City11 models3.20−1.9529
KORD Chicago O'Hare11 models1.56−0.5729
KPHL Philadelphia Intl11 models2.27−0.3929
KPHX Phoenix Sky Harbor11 models5.48−5.1928
KSAN San Diego Lindbergh11 models5.51−5.4428
KSAT San Antonio Intl11 models3.38−2.9329
KSEA Seattle-Tacoma11 models3.57−2.4128
KSFO San Francisco Intl11 models2.50−1.8028
KTTN Trenton Mercer11 models3.16+2.9429

Data availability

Each model is scored only over its own available period (2026-01-01 → 2026-08-30); the windows above are therefore not identical across models. Pairwise comparisons on the permanent-link pages use common days only.

ModelPeriodScored daysCoverage
ECMWF AIFS Single2026-08-02 → 2026-08-3029
2026-01-012026-08-30
Aurora (GFS init)2026-08-29 → 2026-08-302
2026-01-012026-08-30
Aurora (IFS init)2026-08-29 → 2026-08-302
2026-01-012026-08-30
FourCastNet v2 (GFS init)2026-08-29 → 2026-08-302
2026-01-012026-08-30
FourCastNet v2 (IFS init)2026-08-29 → 2026-08-302
2026-01-012026-08-30
NCEP GFS2026-08-02 → 2026-08-3029
2026-01-012026-08-30
GraphCast (GFS init)2026-01-02 → 2026-08-3036
2026-01-012026-08-30
GraphCast (IFS init)2026-01-02 → 2026-08-30197
2026-01-012026-08-30
ECMWF IFS HRES2026-08-02 → 2026-08-3029
2026-01-012026-08-30
Pangu-Weather (GFS init)2026-08-29 → 2026-08-302
2026-01-012026-08-30
Pangu-Weather (IFS init)2026-08-29 → 2026-08-302
2026-01-012026-08-30
Persistence (baseline)2026-01-01 → 2026-08-30215
2026-01-012026-08-30

Stations23

full station table
  • KNYC — New York Central Park
  • KEWR — Newark Liberty
  • KPHL — Philadelphia Intl
  • KTTN — Trenton Mercer
  • KBOS — Boston Logan
  • KDCA — Washington Reagan
  • KATL — Atlanta Hartsfield
  • KMIA — Miami Intl
  • KORD — Chicago O'Hare
  • KMSP — Minneapolis-St Paul
  • KDFW — Dallas-Fort Worth
  • KIAH — Houston Bush
  • KAUS — Austin Bergstrom
  • KSAT — San Antonio Intl
  • KMSY — New Orleans Intl
  • KOKC — Oklahoma City
  • KDEN — Denver Intl
  • KPHX — Phoenix Sky Harbor
  • KLAS — Las Vegas Harry Reid
  • KLAX — Los Angeles Intl
  • KSAN — San Diego Lindbergh
  • KSFO — San Francisco Intl
  • KSEA — Seattle-Tacoma