Data — CastCheck

Downloads

File	Contents	Rows	Size
`scores_latest.csv.gz`	every published aggregate: station × model × init × lead × variable × method × window, with n, MAE, bias, RMSE, hit rates, skill and bootstrap intervals (values in °C).	407448	17.6 MB
`pairwise_latest.csv.gz`	paired model-vs-model MAE differences on common days, with the bootstrap interval, the bootstrap p-value and the Holm verdict (°C). Published on Hugging Face: it grows as the square of the model count and nothing on this site reads it. The station_id=ALL slice the leaderboards use is at /api/v1/pairwise/latest.json.	2134608	on Hugging Face
`daily_errors/KATL.csv.gz`	every scored value at KATL: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	978420	7.9 MB
`daily_errors/KAUS.csv.gz`	every scored value at KAUS: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	978192	7.8 MB
`daily_errors/KBOS.csv.gz`	every scored value at KBOS: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	962336	7.8 MB
`daily_errors/KDCA.csv.gz`	every scored value at KDCA: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	978676	8.0 MB
`daily_errors/KDEN.csv.gz`	every scored value at KDEN: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	930106	7.7 MB
`daily_errors/KDFW.csv.gz`	every scored value at KDFW: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	975596	7.8 MB
`daily_errors/KEWR.csv.gz`	every scored value at KEWR: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	977468	8.0 MB
`daily_errors/KIAH.csv.gz`	every scored value at KIAH: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	979892	7.6 MB
`daily_errors/KLAS.csv.gz`	every scored value at KLAS: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	930538	7.7 MB
`daily_errors/KLAX.csv.gz`	every scored value at KLAX: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	930474	7.3 MB
`daily_errors/KMIA.csv.gz`	every scored value at KMIA: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	976948	7.4 MB
`daily_errors/KMSP.csv.gz`	every scored value at KMSP: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	980500	8.1 MB
`daily_errors/KMSY.csv.gz`	every scored value at KMSY: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	979380	7.2 MB
`daily_errors/KNYC.csv.gz`	every scored value at KNYC: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	977024	7.9 MB
`daily_errors/KOKC.csv.gz`	every scored value at KOKC: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	978564	8.0 MB
`daily_errors/KORD.csv.gz`	every scored value at KORD: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	979892	8.1 MB
`daily_errors/KPHL.csv.gz`	every scored value at KPHL: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	979892	8.0 MB
`daily_errors/KPHX.csv.gz`	every scored value at KPHX: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	930922	7.5 MB
`daily_errors/KSAN.csv.gz`	every scored value at KSAN: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	929770	7.2 MB
`daily_errors/KSAT.csv.gz`	every scored value at KSAT: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	980500	7.8 MB
`daily_errors/KSEA.csv.gz`	every scored value at KSEA: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	930922	7.4 MB
`daily_errors/KSFO.csv.gz`	every scored value at KSFO: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	930922	7.4 MB
`daily_errors/KTTN.csv.gz`	every scored value at KTTN: one row per model, initialization, lead day, method, variable and climatological day (four rows a day for the pooled t2), with the forecast, the observation and the signed error (°C, 2 dp). Download all 23 to recompute anything.	973596	7.9 MB
`stations.csv`	station metadata: identifier, name, truth product, IEM archive id, fixed standard UTC offset, coordinates, station elevation, the mean elevation of the 0.25° grid cell, their difference and its first-order lapse-rate magnitude, and the temperature-contract city the station was selected for.	23	2.3 kB
`models.csv`	model registry, including the persistence baseline.	12	792 B

The parquet shards behind these CSVs are mirrored as a Hugging Face dataset: https://huggingface.co/datasets/castcheck/temperature-verification. The code that produces them is at https://github.com/ZzZTripleZzZ/castcheck (MIT); this page was built from commit 90d8bfb.

The 6-hourly instantaneous layer

Everything on this site is derived from two tables that are published in full, because a daily extreme is a definition and someone checking a claim should not have to accept ours:

forecast_values — one row per model run, valid time, station, variable and interpolation method: the raw 2 m temperature at 00/06/12/18 UTC, before any extreme is taken. Parquet shards at https://huggingface.co/datasets/castcheck/temperature-verification/tree/main under data/forecast_values/model_id=<model>/year_month=YYYY-MM.parquet; the signed daily errors derived from them are in the daily_errors/<station>.csv.gz files above — one row per model, initialization, lead day, method, variable and day, and four rows a day for the pooled t2, with valid_hour_utc naming the instant.
truth_instant — the observed 2 m temperature at those same four instants: the routine METAR nearest the synoptic hour within ±35 minutes, from the IEM ASOS archive (data/truth_instant/year=YYYY.parquet in the same dataset). This is the truth for the headline t2 variables and for tmax_s/tmin_s; the NWS Daily Climate Report in truth_daily is the truth for the secondary *_cli variables.

JSON API (v1)

Static files, CORS-open, cached one hour. See the API page for worked curl and Python examples and openapi.json.

Endpoint	Contents	Rows/files
`scores/index.json`	the index of the sharded scores export: which stations, models, leads and views exist and where each shard is	1
`/station/{station}/cards.json`	the shard the index points at: every score for one station, plus one card per model and lead day (its pairwise table and daily error series). /api/v1/scores/by-station/{station}.json points here	—
`scores/leaderboard.json`	the station_id=ALL slice used by the front page	—
`leaderboard/{window}-{init}z-{method}-{variable}.json`	one pre-built file per leaderboard view (48 of them)	—
`/station/{station}/cards.json#{model}-{lead}`	one permanent-link card inside that bundle: its pairwise table and the last 90 days of daily errors	—
`pairwise/latest.json`	model-vs-model paired bootstrap (station_id=ALL)	2134608
`stations.json`	station metadata	—
`models.json`	model metadata	—
`status.json`	pipeline completeness report	—
`diagnostics.json`	the diurnal-structure diagnostic behind /diagnostics/: bias at each of the four synoptic instants, bias against lead day, and the sampling penalty	—
`openapi.json`	OpenAPI 3.1 description of the endpoints above	—

Table schemas

forecast_values

one extracted station value per model run, valid time, variable and interpolation method (DESIGN §3.1). This is the 6-hourly instantaneous layer: the only table that is untouched by any daily-extreme definition, and the one to start from to re-derive everything else or to check a claim about diurnal amplitude.

Column	Type	Unit	Meaning
`model_id`	string	—	stable model identifier from config/models.yaml
`model_version`	string	—	upstream cycle/version string as advertised by the producer
`init_time`	timestamp	UTC	model initialization time
`valid_time`	timestamp	UTC	forecast valid time
`lead_h`	int16	hours	valid_time − init_time
`station_id`	string	—	ICAO identifier of the station the row belongs to
`variable`	string	—	the field this value was extracted from: t2 (instantaneous 2 m temperature, the only one that is scored), or one of the models' native extreme fields — mx2t3/mn2t3 and mx2t6/mn2t6 (ECMWF, 3 h and 6 h accumulations) and tmax6/tmin6 (GFS, 6 h). The native fields are diagnostic only (METHODOLOGY §2.4)
`bucket_h`	int8	hours	accumulation window of a native extreme field, 0 if instantaneous
`method`	string	—	grid-to-station interpolation: bilinear (headline) or nearest
`value_c`	float32	°C	extracted station value, NaN when missing
`missing_reason`	string	—	why a value is absent; empty when present
`source_url`	string	—	exact object or URL the value was read from
`fetched_at`	timestamp	UTC	when the value was fetched

truth_instant

the observed 2 m temperature at the four common synoptic instants — the routine METAR nearest the hour within ±35 minutes (DESIGN §10.1). This is the truth for the headline t2 variables and for tmax_s/tmin_s.

Column	Type	Unit	Meaning
`station_id`	string	—	ICAO identifier of the station the row belongs to
`valid_time`	timestamp	UTC	the synoptic instant the observation belongs to — one of 00/06/12/18 UTC, not a forecast valid time. The timestamp of the report actually used is obs_time, within ±35 min of this
`temp_c`	float32	°C	observed 2 m air temperature at the synoptic instant; empty when no usable report was found
`obs_time`	timestamp	UTC	timestamp of the report actually used
`source`	string	—	where the observation came from: ASOS_IEM (the Iowa Environmental Mesonet METAR archive) or NWS_API (api.weather.gov, used for the last week, before the archive catches up)
`n_reports`	int8	count	routine reports found inside the ±35 min window
`qc_flag`	string	—	quality note, empty when clean
`schema_version`	string	—	data-model version (DESIGN §3)
`methodology_version`	string	—	METHODOLOGY version that produced the numbers

truth_daily

one row per station-day-source with the first-final policy and QC flags (DESIGN §3.2). Truth for the secondary tmax_cli/tmin_cli.

Column	Type	Unit	Meaning
`station_id`	string	—	ICAO identifier of the station the row belongs to
`climo_date`	date	LST day	climatological day, midnight to midnight local standard time
`source`	string	—	truth product: CLI, CF6 or OBS
`tmax_f`	int16	°F	daily maximum as reported by the NWS (whole degrees)
`tmin_f`	int16	°F	daily minimum as reported by the NWS (whole degrees)
`tmax_c`	float32	°C	daily maximum converted to °C
`tmin_c`	float32	°C	daily minimum converted to °C
`issuance_time`	timestamp	UTC	issuance time of the truth product
`is_final`	bool	—	first CLI issued after local midnight (the first-final policy)
`revised`	bool	—	a later corrected report exists (never used in scores)
`revised_tmax_f`	int16	°F	latest corrected maximum, published but not scored
`revised_tmin_f`	int16	°F	latest corrected minimum, published but not scored
`qc_flag`	string	—	quality note, empty when clean
`product_id`	string	—	api.weather.gov product id or IEM archive key
`schema_version`	string	—	data-model version (DESIGN §3)
`methodology_version`	string	—	METHODOLOGY version that produced the numbers

daily_forecasts

sampled and native daily extremes per model run, station and climatological day (DESIGN §3.3).

Column	Type	Unit	Meaning
`model_id`	string	—	stable model identifier from config/models.yaml
`model_version`	string	—	upstream cycle/version string as advertised by the producer
`init_time`	timestamp	UTC	model initialization time
`station_id`	string	—	ICAO identifier of the station the row belongs to
`climo_date`	date	LST day	climatological day, midnight to midnight local standard time
`lead_day`	int8	days	target climatological date − UTC date of the initialization
`method`	string	—	grid-to-station interpolation: bilinear (headline) or nearest
`tmax_sampled_c`	float32	°C	max of the four common samples in the climatological day
`tmin_sampled_c`	float32	°C	min of the four common samples in the climatological day
`n_samples`	int8	count	how many of the four common samples were present (0–4)
`tmax_native_c`	float32	°C	daily max from the model's native extreme field (diagnostic)
`tmin_native_c`	float32	°C	daily min from the model's native extreme field (diagnostic)
`missing_reason`	string	—	why a value is absent; empty when present
`tmax_obs_s_c`	float32	°C	max of the four observed instants of the same climatological day — the truth tmax_s is scored against, so that forecast and observation are sampled identically (METHODOLOGY §2.3)
`tmin_obs_s_c`	float32	°C	min of the four observed instants of the same climatological day — the truth tmin_s is scored against
`n_obs_samples`	int8	count	how many of the four observed instants were present (0–4); tmax_s/tmin_s are only scored on a day with all four
`native_overhang_h`	int8	hours	how far past the end of the climatological day the native extreme accumulation window reaches, 0–6; it is why the native comparison is diagnostic only (METHODOLOGY §2.4)
`schema_version`	string	—	data-model version (DESIGN §3)
`methodology_version`	string	—	METHODOLOGY version that produced the numbers

scores

published aggregates with bootstrap intervals (DESIGN §3.4).

Column	Type	Unit	Meaning
`station_id`	string	—	ICAO identifier, or ALL for the cross-station aggregate
`model_id`	string	—	stable model identifier from config/models.yaml
`init_hour`	int8	UTC hour	model initialization hour, 0 or 12
`lead_day`	int8	days	target climatological date − UTC date of the initialization
`variable`	string	—	t2 (instantaneous, headline), t2_00z…t2_18z (one synoptic hour), tmax_s/tmin_s (extremes of the four samples vs the extremes of the four observed samples), tmax_cli/tmin_cli (the same forecast samples vs the NWS daily extreme, secondary) or tmax_native_cli/tmin_native_cli (native extreme fields vs the NWS daily extreme, secondary)
`method`	string	—	grid-to-station interpolation: bilinear (headline) or nearest
`window`	string	—	scoring window: 30d, 90d, 365d or all
`n`	int32	days	number of scored climatological days in the window
`n_stations`	float32	stations	mean number of stations behind each day of an ALL row (1 for a single-station row)
`n_flagged`	int32	days	how many of those days carry a QC flag on the observation
`mae`	float32	°C	mean absolute error
`bias`	float32	°C	mean signed error; positive = model too warm
`rmse`	float32	°C	root mean squared error
`hit1f`	float32	fraction	share of days with \|error\| ≤ 1 °F
`hit2f`	float32	fraction	share of days with \|error\| ≤ 2 °F
`hit3f`	float32	fraction	share of days with \|error\| ≤ 3 °F
`mae_debiased`	float32	°C	MAE after removing an out-of-sample bias estimate: the bias of the trailing 30 scored days before each day (minimum 15) applied forward to that day. Days without enough history are excluded
`n_debiased`	int32	days	days that had at least 15 of the preceding 30 scored days available to estimate the out-of-sample bias correction
`n_common`	int32	days	days used as the denominator of the skill score: days on which both the model and the persistence baseline have a value (in the pairwise table, days on which both models of the pair have a forecast)
`mae_persistence_common`	float32	°C	MAE of the persistence baseline restricted to the n_common days — the denominator of skill_persistence
`skill_persistence`	float32	fraction	1 − MAE ÷ mae_persistence_common, both computed on the same n_common days; positive is better
`skill_persistence_debiased`	float32	fraction	the same skill score computed on the out-of-sample debiased errors, so a station with a large constant offset is not scored as skill-less
`skill_ci_low`	float32	fraction	2.5th percentile of the paired skill bootstrap
`skill_ci_high`	float32	fraction	97.5th percentile of the paired skill bootstrap
`mae_ci_low`	float32	°C	2.5th percentile of the per-group moving-block bootstrap; empty when n < 28 or fewer than 4 blocks
`mae_ci_high`	float32	°C	97.5th percentile of the per-group moving-block bootstrap; empty when n < 28 or fewer than 4 blocks
`bias_ci_low`	float32	°C	2.5th percentile of the bias bootstrap
`bias_ci_high`	float32	°C	97.5th percentile of the bias bootstrap
`rmse_ci_low`	float32	°C	2.5th percentile of the RMSE bootstrap
`rmse_ci_high`	float32	°C	97.5th percentile of the RMSE bootstrap
`hit1f_ci_low`	float32	fraction	lower bound of the Wilson score 95 % interval of the ±1 °F hit rate
`hit1f_ci_high`	float32	fraction	upper bound of the Wilson score 95 % interval of the ±1 °F hit rate
`model_version`	string	—	upstream cycle/version string as advertised by the producer
`segment_start`	date	LST day	first day of the current model-version segment; scores cover only this segment
`period_start`	date	LST day	first climatological day contributing to the window
`period_end`	date	LST day	last climatological day contributing to the window
`computed_at`	timestamp	UTC	when this table was computed
`methodology_version`	string	—	METHODOLOGY version that produced the numbers
`schema_version`	string	—	data-model version (DESIGN §3)

pairwise

paired model-vs-model MAE differences (DESIGN §3.5).

Column	Type	Unit	Meaning
`station_id`	string	—	ICAO identifier, or ALL for the cross-station aggregate
`init_hour`	int8	UTC hour	model initialization hour, 0 or 12
`lead_day`	int8	days	target climatological date − UTC date of the initialization
`variable`	string	—	t2 (instantaneous, headline), t2_00z…t2_18z (one synoptic hour), tmax_s/tmin_s (extremes of the four samples vs the extremes of the four observed samples), tmax_cli/tmin_cli (the same forecast samples vs the NWS daily extreme, secondary) or tmax_native_cli/tmin_native_cli (native extreme fields vs the NWS daily extreme, secondary)
`window`	string	—	scoring window: 30d, 90d, 365d or all
`model_a`	string	—	first model of the pair
`model_b`	string	—	second model of the pair
`n_common`	int32	days	days used as the denominator of the skill score: days on which both the model and the persistence baseline have a value (in the pairwise table, days on which both models of the pair have a forecast)
`mae_diff`	float32	°C	MAE(model_a) − MAE(model_b) on their common days
`ci_low`	float32	°C	2.5th percentile of the paired bootstrap difference
`ci_high`	float32	°C	97.5th percentile of the paired bootstrap difference
`p_boot`	float32	—	two-sided bootstrap p-value of the paired MAE difference
`distinguishable_uncorrected`	bool	—	true when the single 95 % interval of the difference excludes zero, with no correction for the number of comparisons made
`distinguishable_holm`	bool	—	true after a Holm correction over the family of comparisons against the leader within one displayed table (same station, initialization, lead, variable, method and window). This is the only flag the site marks with ▼ or ▲
`method`	string	—	grid-to-station interpolation: bilinear (headline) or nearest
`computed_at`	timestamp	UTC	when this table was computed
`methodology_version`	string	—	METHODOLOGY version that produced the numbers
`schema_version`	string	—	data-model version (DESIGN §3)

Licences and attribution

CastCheck derived tables and this site — CC BY 4.0. Please cite the permanent link of the page you used.
CastCheck code — MIT, https://github.com/ZzZTripleZzZ/castcheck.
ECMWF IFS HRES and AIFS Single open data — CC BY 4.0, © ECMWF. Source: data.ecmwf.int and the AWS Open Data mirror. Changes made: the published 0.25° fields are interpolated to a station point and converted to °C; nothing else is altered. ECMWF is not the source of, and does not endorse, the scores on this site.
NCEP GFS (AWS noaa-gfs-bdp-pds) — U.S. Government work, public domain.
NOAA/CIRA AIWP model output (AWS noaa-oar-mlwp-data) — U.S. Government work, public domain. Cite: Radford, J. T., et al. (2025). Domain-agnostic evaluation of AI weather prediction models. Bulletin of the American Meteorological Society. doi:10.1175/BAMS-D-24-0057.1.
NWS Daily Climate Report (CLI), CF6 and station observations — U.S. Government work, public domain, via api.weather.gov.
Historical NWS text products — Iowa Environmental Mesonet (IEM) AFOS archive, Iowa State University.

Raw model output, no post-processing. These are values interpolated from each model's native grid, without MOS, bias correction, downscaling or any other post-processing, and they are not the products that ECMWF, NOAA, the NWS or any app vendor issue to end users.

Cite the dataset

Dataset

CastCheck (2026). Station-level verification of raw weather-model 2 m temperature forecasts [data set]. Methodology version 0.3.1, schema version 0.3, data through 2026-10-28. https://castcheck.zifanzhang.com doi:10.5281/zenodo.22212363 · doi:10.5281/zenodo.22212363

Software archive

Zenodo record 10.5281/zenodo.22212363 — the concept DOI, which always resolves to the latest release. To cite one release, use its version DOI instead (v0.3.3: 10.5281/zenodo.22237426).

For a specific number, cite its permanent link instead — every score card carries a short and a long citation with the access date.

Machine-readable citation metadata is in CITATION.cff at the repository root. Every build stamps the commit it came from — this one is 90d8bfb, also in the footer of every page — so a citation can name the exact code that produced the number.

Changelog

Rendered from the Changelog section of METHODOLOGY.md, so the two cannot diverge.

Methodology	Date	Change
v0.3.1	2026-08-31	CLI plausibility check against the instantaneous observations. §3.3 (new) / §6: the published daily extreme is checked against the four sampled observations of the same day and repaired from the corrected issuance or CF6, or dropped when it is physically impossible and nothing can replace it, or when it lies past the widest excursion three years of observations support (25 °F). First-final is unchanged; only values that contradict the station's own measurements are acted on. 14 station-days affected in the 2024-2026 archive.
v0.3	2026-08-31	external methodological review (`docs/06-external-review-v02.md`). Published scores are not comparable to v0.2. §2.3: the headline is now the instantaneous 2 m temperature at the four common instants (`t2`, plus `t2_00z…t2_18z`) against the observation at the same instant. Sampled daily extremes are scored like for like against the observed sampled extremes (`tmax_s`, `tmin_s`). The old comparison — sampled forecast extreme vs true CLI extreme — survives as the secondary `tmax_cli`/`tmin_cli`, explicitly labelled as carrying a sampling penalty whose size varies with each model's diurnal amplitude. The v0.2 claim that this penalty "applies identically to every model and is therefore fair for comparing models" is withdrawn. §3.1: new instantaneous truth table `truth_instant` (ASOS routine METAR within ±35 min of the synoptic hour). §2.4: the realised native-extreme overhang is published per row as `native_overhang_h`. §2.5: documents that the same `lead_day` is a different forecast range at different longitudes (up to 6 h across the CONUS), and that the `ALL` row mixes them. §2.6 / §1.1: station metadata gains `grid_elev_m` and `dz_m`; the station-selection rule is stated, together with what it does to external validity. §4: the persistence baseline is now the same functional as the numerator (instant vs instant, sampled extreme vs sampled extreme), which removes the sampling penalty from the skill ratio. `n_common` and `mae_persistence_common` are published so that the skill column is recomputable from its own row. `mae_debiased` becomes out of sample (trailing 30 scored days, minimum 15) with a new `n_debiased`. The `ALL` rule is restated as "each functional averaged separately". §5: the day bootstrap is drawn per group on that group's own realised dates, seeded from the date set, replacing the shared-resample-matrix approximation; no interval below 28 days or 4 blocks. Hit rates move to Wilson intervals. `skill_ci_low/high` added. `significant` renamed `distinguishable_uncorrected`; new `p_boot` and `distinguishable_holm` (Holm over the comparisons against the leader within one displayed table), and only the latter is marked on the site. §10: rewritten; adds the unattributed 18/00 UTC AI cold-bias observation with its three candidate explanations and the planned attribution analyses, and no causal claim.
v0.2	2026-08-30	methodology and statistics review. §2.4 native extremes: the "buckets entirely inside the day" rule (which yielded values only at −6 h stations) is replaced by "the contiguous overlapping run, at most one crossing bucket at each end, overhang ≤ 6 h"; the overhang is documented per station offset. Native extremes remain diagnostic. §3: preliminary CLI reports are dropped rather than ranked below the observation fallback. NWS rounding (nearest °F, mid-points up; ASOS internal tenths of °C) documented; hit-rate thresholds stated as inclusive. §4: new published columns `n_stations`, `n_flagged`, `mae_debiased`, `skill_persistence_debiased`, `rmse_ci_low/high`, `hit1f_ci_low/high`, `model_version`, `segment_start`. The lagged-persistence baseline and the day-mean `ALL` aggregate are stated as rulings with their rejected alternatives. §5: the day bootstrap is now a circular moving-block bootstrap (block 7 days). The shared-resample-matrix approximation is documented and measured against a textbook per-group bootstrap. §7: version segmentation is implemented, not merely asserted — the scoring window is truncated to the latest `model_version` segment.
v0.1	2026-08-30	initial pre-release methodology.

Data events (a source outage, a corrected observation, a model version change) are recorded in status and in the gaps list of status.json.

What the first-final policy costs

A score is computed from the first final NWS Daily Climate Report and is never rewritten when a corrected report arrives later. That keeps a published number stable and citable, at the price of a permanent, documented discrepancy wherever a correction happened. This is how often, and how large:

Station-days with a first-final CLI in the archive: **23638**, of which 35 carry a later corrected report. The rows below count only the days on which the corrected value actually differs, and measure corrected − first in whole °F.
Variable	Days changed	Share of days	Mean change °F	Median °F	Largest \|change\| °F
daily maximum	13	0.05%	-1.00	-1.0	5
daily minimum	15	0.06%	-1.67	-1.0	7

Read the largest column as the upper bound on how far any single published score can be from the currently accepted observation. The corrected values are published in truth_daily as revised_tmax_f/revised_tmin_f, so anyone can recompute the scores on corrected truth and compare.

Known limitations

The headline variable is the instantaneous temperature at four synoptic instants, so it says nothing about the daily maximum a person experiences. The like-for-like daily extremes (tmax_s/tmin_s) are the max/min of the same four samples on both sides; the comparison against the true NWS daily extremes (tmax_cli/tmin_cli) additionally carries a sampling penalty whose size depends on each model's own diurnal amplitude, which is why it is published as secondary and never ranked (METHODOLOGY §2.3).
No elevation or lapse-rate correction is applied; stations whose elevation differs sharply from the 0.25° grid cell carry a representativeness error that is charged to the model. The size of the offset is published per station on /stations/ as dz_m and a first-order lapse-rate magnitude.
Truth is the first final NWS CLI report for the CLI variables and the routine METAR nearest each synoptic hour for the instantaneous ones. Later corrections are stored but never change a published score, so a corrected observation leaves a permanent, documented discrepancy; the size of that effect is quantified on /data/.
Models enter the record on different dates, so their windows are not identical. Pairwise comparisons are computed on common days only; the leaderboard columns are not. The skill column is the one leaderboard column that is a common-day comparison, and it names its own denominator and n.
Groups with fewer than 30 scored days are published but greyed out and excluded from every ranking, and no confidence interval is computed below 28 days or 4 bootstrap blocks; early in a model's record most windows are in that state.
The same lead day covers different forecast hours at different longitudes: a climatological day begins at local midnight, so at the easternmost station (UTC−5) a lead day starts six hours earlier in forecast time than at the westernmost (UTC−8). Every pooled ALL row mixes forecast ranges that differ by up to six hours across the station set (§2.5).
The 0.25° AIWP archive is a research product whose GFS-initialized models are produced on alternating cycles; an initialization the archive never produced is marked as such on /status/ and is not counted as downtime.