Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library holds four geophysical sweeps of the guidance weight under a stated convention (Lopez-Gomez et al., 2025; Martin et al., 2025; Schusterbauer et al., 2026; Suresh Babu et al., 2025) and two whose convention cannot be recovered (Hua et al., 2024; Millard et al., 2025). The episode adds a fifth stated sweep from outside the library. James et al. (2026) generate thirty-member six-hour ensembles of convective available potential energy over the contiguous United States from a deterministic forecast, with a guidance variant whose weaker branch drops only the forecast field rather than the whole condition. Swept from zero to 1.5 on one summer, the root-mean-square error has a broad minimum near plain conditional sampling, the ensemble spread falls throughout, and the summer-average spread-skill ratio stays below one at every weight; the authors choose 0.6, below conditional sampling, for high skill and a reasonable spread-skill ratio, and their curves show it giving about half again the spread of conditional sampling for about a tenth more error. No continuous ranked probability score is reported per weight, and the sweep summer lies inside the fine-tuning period.

Reading the full texts adds conditions that the library's account leaves implicit. The day-seven Z500 sweep of Martin et al. (2025) is verified against a withheld simulation of the emulator it was distilled from, not against observations; the downscaling weight of Lopez-Gomez et al. (2025) was selected from four values jointly with the probability of dropping the condition in training, learning rate and batch size, with no score per weight shown; and the optimum of Suresh Babu et al. (2025) is stated without published error values. For precipitation, Schusterbauer et al. (2026) report that CasCast's extreme values under guidance soon exceed the strongest precipitation in SEVIR, and relate the upward shift to the dominance of light precipitation in the data.

• A CAPE ensemble study outside the library finds its lowest error near plain conditional sampling and chooses a weight below it, which widens an ensemble that on summer average is under-dispersed at every weight (James et al., 2026, preprint).
• The day-seven Z500 calibration result is a perfect-model test, and two of the other optimal weights are reported without a score at each weight.

Where they differ from the library

The library describes the optimal setting as lying at or near conditional sampling in the calibration studies, and the full texts support this. Two weather sources move the weight below conditional sampling to widen an under-dispersed ensemble, James et al. (2026) and the stated practice of Martin et al. (2025), whose own optimum was nonetheless plain conditional sampling; this is the reverse of image generation (Ho & Salimans, 2022). Other studies placed the weight at or above it, 1.1 in Lopez-Gomez et al. (2025), 1.25 in Suresh Babu et al. (2025) and 2, one full step past conditional sampling, in Foo et al. (2025), each supported by error, CRPS or threshold scores. James et al. (2026) also state that spread stays flat above 0.5, while the spread curve in their Figure 3 falls by about three fifths between 0.5 and 1.5 and the error curve in their Figure 2 falls to a minimum near 1 and rises only slightly after it, so the decline in their spread-skill ratio above 0.5 comes from the spread.

The scores disagree with one another, and the library records this only in part. Guidance raises categorical scores on SEVIR, the Heidke skill score up to a weight near 1.5 (Schusterbauer et al., 2026) and every critical success index at a weight one full step past conditional sampling (Foo et al., 2025), while the continuous ranked probability score worsens with every step in the one study that reports both. The precipitation result of Millard et al. (2025) worsens with the guidance value under either of the paper's two formulas, so its direction can be counted even though its position cannot; the authors report 0.5 as best for most other variables and 0.7 for geopotential, while their tables show 0.3 best for the 10 m winds at longer leads and 0.7 best for geopotential only at the last lead, so only the rain result has a clear direction.

• Where a conditional weather ensemble is too narrow, the weight has been moved below conditional sampling to loosen it, the reverse of its use in image generation (James et al., 2026; Martin et al., 2025).
• On SEVIR precipitation, guidance improves threshold scores while it worsens the continuous ranked probability score, so a study's verdict depends on which score it reports.

How they bear on the open questions

On which criterion should select the weight, the sources read here show that the choice of score decides the verdict on precipitation: hit-based scores reward the intensity shift that the distribution score penalizes, and the rank histogram of CasCast, whose published results use guidance, shows strong overconfidence in Schusterbauer et al. (2026). The question remains open, because only two studies score rain probabilistically against the weight, neither reports reliability at heavy-rain thresholds, and no study tests whether a weight selected in one region or season transfers to another.

On whether the image-side controls change geophysical fields, the one test remains the projected update of Sadat et al. (2025), which on the two SEVIR nowcasting models of Schusterbauer et al. (2026) left the worsening of the continuous ranked probability score in place; in the sources found here, the interval, learned, scheduled and weaker-model alternatives have been scored only on images and video. On reporting, the episode confirms that Nath et al. (2023), Ren et al. (2025) and Springenberg et al. (2025) give no sweep and either no weight or no formula, and that Hua et al. (2024) give no formula, so the reporting gap the library records persists in the sources read here.

• How the weight changes heavy-rain reliability and how often extremes occur has not been measured in any source read here; one study shows only that a model's largest values grow.
• Of the image-side alternatives to a constant weight, only projected guidance has been tried on weather in the sources found here, and it did not stop the probabilistic score from worsening.

References

1. Foo, S. Q., Wong, C.-H., Gao, Z., Yeung, D.-Y., Wong, K.-H., & Wong, W.-K. (2025). STLDM: Spatio-temporal latent diffusion model for precipitation nowcasting. Transactions on Machine Learning Research. https://doi.org/10.48550/arXiv.2512.21118
2. Ho, J., & Salimans, T. (2022). Classifier-free diffusion guidance (arXiv:2207.12598). arXiv. https://doi.org/10.48550/arXiv.2207.12598
3. Hua, Z., He, Y., Ma, C., & Anderson-Frey, A. (2024). Weather prediction with diffusion guided by realistic forecast processes (arXiv:2402.06666). arXiv. https://doi.org/10.48550/arXiv.2402.06666
4. James, Z., Guinness, J., & DeGaetano, A. (2026). Improving ensemble CAPE forecasts with a diffusion model incorporating aerosol information (arXiv:2605.24009). arXiv. https://doi.org/10.48550/arXiv.2605.24009
5. Lopez-Gomez, I., Wan, Z. Y., Zepeda-Núñez, L., Schneider, T., Anderson, J., & Sha, F. (2025). Dynamical-generative downscaling of climate model ensembles. Proceedings of the National Academy of Sciences, 122(17), e2420288122. https://doi.org/10.1073/pnas.2420288122
6. Martin, S. A., Brenowitz, N., Durran, D. R., & Pritchard, M. S. (2025). Long-range distillation: Distilling 10,000 years of simulated climate into long timestep AI weather models (arXiv:2512.22814). arXiv. https://doi.org/10.48550/arXiv.2512.22814
7. Millard, D., Carr, A., Gaudreault, S., & Baheri, A. (2025). DEF: Diffusion-augmented ensemble forecasting (arXiv:2506.07324). arXiv. https://doi.org/10.48550/arXiv.2506.07324
8. Nath, P., Shukla, P., Wang, S., & Quilodrán-Casas, C. (2023). Forecasting tropical cyclones with cascaded diffusion models. ICLR 2024 Workshop on Tackling Climate Change with Machine Learning. https://doi.org/10.48550/arXiv.2310.01690
9. Ren, Z., Nath, P., & Shukla, P. (2025). Improving tropical cyclone forecasting with video diffusion models. ICLR 2025 Workshop on Tackling Climate Change with Machine Learning. https://doi.org/10.48550/arXiv.2501.16003
10. Sadat, S., Hilliges, O., & Weber, R. M. (2025). Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2410.02416
11. Schusterbauer, J., Wiese, J., Stracke, N., Phan, T., & Ommer, B. (2026). Probabilistic precipitation nowcasting with rectified flow transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.48550/arXiv.2605.31204
12. Springenberg, M., Otero, N., Xue, Y., & Ma, J. (2025). DiffScale: Continuous downscaling and bias correction of subseasonal wind speed forecasts using diffusion models (arXiv:2503.23893). arXiv. https://doi.org/10.48550/arXiv.2503.23893
13. Suresh Babu, A. N., Sadam, A., & Lermusiaux, P. F. J. (2025). Guided unconditional and conditional generative models for super-resolution and inference of quasi-geostrophic turbulence (arXiv:2507.00719). arXiv. https://doi.org/10.48550/arXiv.2507.00719

Read into the library after publication:

• Improving Ensemble CAPE Forecasts with a Diffusion Model Incorporating Aerosol Information arXiv:2605.24009

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library holds three sources that tie a forecasting design choice to a property of the data (Liu et al., 2024; Nie et al., 2023; Shao et al., 2025) and records that each rests on two to four datasets. The episode adds a fourth, larger source from outside the library. Qiu et al. (2024) evaluate fourteen multivariate methods on twenty five datasets from ten domains, compute trend and seasonality strength from an STL decomposition and a shifting score from the position of high values in the series, and conclude that linear-based methods excel on datasets with increasing trend or significant shifts while Transformer-based methods do better with marked seasonality and weak trend. The finding is read by eye from six datasets per characteristic, using the best error within each model family at one horizon per dataset, without repeated runs or a statistical test, and the weak-trend end of the trend panel is occupied by PEMS04 and PEMS08, the datasets Shao et al. (2025) class as stable.

The episode also adds two synthetic benchmarks, both preprints, that bear on whether the rules have been tested under control. Forootani and Khosravi (2025) compare three Transformer families on ten fixed univariate signals and train no linear model. Janßen et al. (2025) compare RLinear with iTransformer, Autoformer and S-Mamba on synthetic data with 800 variates under controlled waveform, frequency, noise type and noise level, with nine runs per setting; adding a trend component improved every model and left the ranking unchanged, with RLinear's error two and a half to five times that of the two best models. Trend in that study is present or absent with a random weight rather than set at graded strengths, and the authors attribute RLinear's weakness to the many cross-dependent variates.

• A twenty-five-dataset benchmark with computed characteristics reports that linear methods do better with strong trend or shifting and Transformers with weak trend and marked seasonality (Qiu et al., 2024).
• That finding is read by eye from six datasets per characteristic at one horizon per dataset, without repeated runs.
• The one controlled synthetic test found with a linear model switched trend on and off at 800 variates, and every model improved without the ranking changing (Janßen et al., 2025, preprint).

Where they differ from the library

The library already records that Shao et al. (2025) assign temporal classes by visualisation, use two datasets per class and report no seeds, and that pattern type is confounded there with domain, variate count and sampling interval. Reading the full texts narrows the library's account of the other two rules. The variate-count rule of Liu et al. (2024) compares iTransformer with its own variant without attention across variates, and the training-size rule of Nie et al. (2023) compares channel-independent with channel-mixing PatchTST on one dataset; neither is a rule for choosing between a Transformer and a linear model. The direct comparison with a linear model in Liu et al. (2024) is its main table, where RLinear is level on the ETT datasets and far behind on Traffic and PEMS, the same contrast that Shao et al. (2025) report. The same table also has iTransformer about a fifth ahead of RLinear on Electricity, a dataset with many variates that Shao et al. (2025) treat as lacking stable patterns and on which their channel-independent PatchTST is level with the linear models; across the two papers, this is the one place where variate count comes apart from pattern type.

Qiu et al. (2024) at first appear to supply the independent replication the library lacks, since their characteristics are computed rather than judged. The overlap in data qualifies this: the weak-trend and weak-shift ends of their panels hold the PEMS traffic networks, and the strong ends hold short daily stock series with five variables, so the trend and shifting contrasts coincide with a contrast in domain, sampling interval, length and variate count. The agreement between the two benchmarks therefore rests partly on shared datasets and does not separate the candidate properties.

• Of the three selection rules in the library, only Shao et al. (2025) state a rule for choosing between Transformers and linear models; the other two compare Transformer variants.
• The new benchmark's agreement with Shao et al. (2025) rests partly on the same PEMS datasets, so it does not separate pattern type from domain, sampling interval or variate count.

How they bear on the open questions

On the library's open question whether Shao's temporal classes can be assigned by a quantitative measure and whether the Transformer-versus-linear ordering follows it outside PeMS and ETT, Qiu et al. (2024) supply such measures over a wider set of datasets, and the ordering they report points the same way. The question remains open in substance, because the extremes of their measured scales are the same PeMS networks and a few stock series, and no source read here varies trend, shift or periodicity within one dataset while comparing both families.

On whether the variate-count rule and the spatial-indistinguishability rule pick the same model on the same data, the sources read here show that both rules, and the trend rule, separate the same datasets except Electricity, where many variates meet low spatial indistinguishability and patterns Shao et al. (2025) treat as unstable; there the cross-paper results lean toward variate count combined with a model that mixes variables (Liu et al., 2024, Table 1; Shao et al., 2025, Table VI), but the two results come from different pipelines. The only controlled synthetic test with a linear model, Janßen et al. (2025), fixes the variate count at 800 and varies trend only by presence, so it does not decide the matter. The broader question of validating selection guidance across horizons, noise levels and data volume stays open, and the reporting of Shao et al. (2025) and Qiu et al. (2024) without repeated runs, set against the five-seed spreads in Ilbert et al. (2024), means that the small margins favouring linear models need repeated runs before they support a rule.

• Whether the Transformer-versus-linear ordering follows a measurable data property stays open, because the candidate properties line up on the same benchmark datasets, with Electricity the one cross-paper hint that variate count matters.
• The experiment that would decide it, changing one property within one dataset and retraining both families over several seeds, is absent from the sources read here.

References

1. Forootani, A., & Khosravi, M. (2025). Synthetic time series forecasting with transformer architectures: Extensive simulation benchmarks (arXiv:2505.20048). arXiv. https://doi.org/10.48550/arXiv.2505.20048
2. Ilbert, R., Odonnat, A., Feofanov, V., Virmaux, A., Paolo, G., Palpanas, T., & Redko, I. (2024). SAMformer: Unlocking the potential of transformers in time series forecasting with sharpness-aware minimization and channel-wise attention. In Proceedings of the 41st International Conference on Machine Learning (PMLR 235). https://doi.org/10.48550/arXiv.2402.10198
3. Janßen, N., Schaller, M., & Rosenhahn, B. (2025). Benchmarking M-LTSF: Frequency and noise-based evaluation of multivariate long time series forecasting models (arXiv:2510.04900). arXiv. https://doi.org/10.48550/arXiv.2510.04900
4. Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., & Long, M. (2024). iTransformer: Inverted transformers are effective for time series forecasting. In International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2310.06625
5. Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2023). A time series is worth 64 words: Long-term forecasting with transformers. In International Conference on Learning Representations. https://doi.org/10.48550/arXiv.2211.14730
6. Qiu, X., Hu, J., Zhou, L., Wu, X., Du, J., Zhang, B., Guo, C., Zhou, A., Jensen, C. S., Sheng, Z., & Yang, B. (2024). TFB: Towards comprehensive and fair benchmarking of time series forecasting methods. Proceedings of the VLDB Endowment, 17(9), 2363-2377. https://doi.org/10.14778/3665844.3665863
7. Shao, Z., Wang, F., Xu, Y., Wei, W., Yu, C., Zhang, Z., Yao, D., Sun, T., Jin, G., Cao, X., Cong, G., Jensen, C. S., & Cheng, X. (2025). Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 37(1), 291-305. https://doi.org/10.1109/TKDE.2024.3484454

Read into the library after publication:

• TFB: Towards Comprehensive and Fair Benchmarking of Time Series Forecasting Methods 10.14778/3665844.3665863
• Synthetic Time Series Forecasting with Transformer Architectures: Extensive Simulation Benchmarks arXiv:2505.20048
• Benchmarking M-LTSF: Frequency and Noise-Based Evaluation of Multivariate Long Time Series Forecasting Models arXiv:2510.04900

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library records the node-definition problem through four applied studies that compress fields into nodes, two reviews and three studies whose nodes are single grid cells or station series, but it holds no controlled comparison of extraction methods. The episode adds a benchmark built for that purpose, which the library lacked. Tibau et al. (2022) generate synthetic fields with known modes and lagged links and find that plain principal component analysis recovers the true mode weights systematically worse than Varimax rotation, with Varimax followed by PCMCI described as seemingly the most robust pipeline. In those main experiments each mode is a bivariate Gaussian inside its own non-overlapping square box, many grid points belong to no mode, and every method is truncated at the true number of modes; the authors themselves state that their spatially concentrated definition of modes entails the Varimax advantage. In grid-level experiments whose modes are taken from reanalysis surface pressure and overlap, the gap between the two extractions narrows to a slight Varimax advantage, which the authors suggest may reflect large, sparse networks, and the true modes of those fields were themselves estimated with a Varimax variant.

Two later preprints extend the comparison while keeping the property that favours Varimax in the benchmark's main test, that each observed point belongs to at most one mode. Brouillard et al. (2024) learn latent nodes and graph jointly under a single-parent assumption; on their own synthetic data, unordered observed variables each assigned to one latent, Varimax-PCMCI is slightly better in most fully linear settings and their method is much better under nonlinear decoding, and on reanalysis sea level pressure they run only their own method and judge it against a published graph and standard regions. Ninad et al. (2025) take the grouping of grid points as given, propose consistency scores that test how each group is summarised against the full field without a ground-truth graph, and evaluate them on random synthetic models and on a SAVAR field of three expert-defined boxes, with no real data.

• A controlled comparison of extraction methods exists, and on synthetic fields with compact, non-overlapping, correctly counted modes Varimax recovers mode weights better than plain principal components (Tibau et al., 2022).
• Where the benchmark's modes come from reanalysis and overlap, the Varimax advantage shrinks to slight, and those true modes were estimated with a Varimax variant.
• The two later comparisons found, both preprints, still test on data where each observed variable belongs to one mode and run no extraction comparison on real data (Brouillard et al., 2024; Ninad et al., 2025).

Where they differ from the library

The library's account of this question states that no source compares extraction methods on the same fields. That holds for real fields but not for synthetic ones: Tibau et al. (2022) compare PCA and Varimax extraction under a known truth, and Brouillard et al. (2024) compare Varimax-PCMCI with a jointly learned representation. The library also records the benchmark only through Camps-Valls et al. (2023), where SAVAR appears among the synthetic surrogates offered for the lack of expert-labelled causal ground truth; reading the benchmark itself narrows its reach further, since its authors state that they do not claim Varimax is a valid estimator of the model's weights and that the model may not capture the dependencies of real teleconnections.

On the applied side the episode agrees with what the library already holds and does not overturn it. The library already records that Nowack et al. (2020) and Debeire et al. (2025) test robustness only within one Varimax pipeline, that Debeire et al. (2025) report a larger uncertainty reduction with ERA5 than with NCEP/NCAR components and frame their networks as a metric of the data, that Karmouche et al. (2023) use packaged EOF and box-average indices, and that Falkena et al. (2025) attribute a wrong-signed link and a largely absent one partly to node definitions.

• The library's statement that no source compares extraction methods holds for real fields only; on synthetic fields such comparisons exist.
• The benchmark's own authors scope its result more narrowly than the applied studies use it: they do not claim Varimax is a valid estimator of the true weights.

How they bear on the open questions

On the library's first open question, whether Varimax components are legitimate causal nodes, the evidence now includes a controlled test, but it remains open in substance. The test favours Varimax only on fields whose modes are compact, separate and correctly counted, or whose true modes were themselves drawn with a Varimax variant (Tibau et al., 2022); overlapping and propagating modes, which the benchmark's authors name as untested, are the conditions under which the question is decided for real climate fields.

On the second open question, what a natively spatio-temporal method recovers on the same fields compared with extract-then-discover, Brouillard et al. (2024) supply a joint method that has been run on reanalysis sea level pressure, but they ran Varimax-PCMCI only on synthetic data, so the comparison on a real field has not been made and the question stays open. Ninad et al. (2025) supply a check, needing no ground truth, of how predefined groups of grid points are summarised; it does not judge where the groups lie, so it could serve such a comparison only for nodes built as groups, and it has been tested only on synthetic fields. The question also bears on the library's model-evaluation work, since every causal fingerprint score in Nowack et al. (2020), Debeire et al. (2025) and Karmouche et al. (2023) is conditioned on one node definition.

• Whether Varimax components are legitimate causal nodes stays open, because the conditions that decide it on real fields, overlapping and moving modes, are untested.
• How a causal graph on a real field changes with the extraction method stays open, since no study read here runs two extractions on the same real field and compares the graphs.

References

1. Brouillard, P., Lachapelle, S., Kaltenborn, J., Gurwicz, Y., Sridhar, D., Drouin, A., Nowack, P., Runge, J., & Rolnick, D. (2024). Causal representation learning in temporal data via single-parent decoding (arXiv:2410.07013). arXiv. https://doi.org/10.48550/arXiv.2410.07013
2. Camps-Valls, G., Gerhardus, A., Ninad, U., Varando, G., Martius, G., Balaguer-Ballester, E., Vinuesa, R., Diaz, E., Zanna, L., & Runge, J. (2023). Discovering causal relations and equations from data. Physics Reports, 1044, 1-68. https://doi.org/10.1016/j.physrep.2023.10.005
3. Debeire, K., Bock, L., Nowack, P., Runge, J., & Eyring, V. (2025). Constraining uncertainty in projected precipitation over land with causal discovery. Earth System Dynamics, 16, 607-630. https://doi.org/10.5194/esd-16-607-2025
4. Falkena, S. K. J., Dijkstra, H. A., & von der Heydt, A. S. (2025). Causal mechanisms of subpolar gyre variability in CMIP6 models. Earth System Dynamics, 16, 1833-1844. https://doi.org/10.5194/esd-16-1833-2025
5. Karmouche, S., Galytska, E., Runge, J., Meehl, G. A., Phillips, A. S., Weigel, K., & Eyring, V. (2023). Regime-oriented causal model evaluation of Atlantic-Pacific teleconnections in CMIP6. Earth System Dynamics, 14(2), 309-344. https://doi.org/10.5194/esd-14-309-2023
6. Ninad, U., Wahl, J., Gerhardus, A., & Runge, J. (2025). Causal discovery on vector-valued variables and consistency-guided aggregation (arXiv:2505.10476). arXiv. https://doi.org/10.48550/arXiv.2505.10476
7. Nowack, P., Runge, J., Eyring, V., & Haigh, J. D. (2020). Causal networks for climate model evaluation and constrained projections. Nature Communications, 11, 1415. https://doi.org/10.1038/s41467-020-15195-y
8. Tibau, X.-A., Reimers, C., Gerhardus, A., Denzler, J., Eyring, V., & Runge, J. (2022). A spatiotemporal stochastic climate model for benchmarking causal discovery methods for teleconnections. Environmental Data Science, 1, e12. https://doi.org/10.1017/eds.2022.11

Read into the library after publication:

• A spatiotemporal stochastic climate model for benchmarking causal discovery methods for teleconnections 10.1017/eds.2022.11
• Causal representation learning in temporal data via single-parent decoding arXiv:2410.07013
• Causal discovery on vector-valued variables and consistency-guided aggregation arXiv:2505.10476

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library already holds every study the episode rests on except one. The addition is FuXiWeather2 (Xu et al., 2026), which withholds the same satellite instruments over a one-year test from two assimilation models that differ in their training data, one trained on real observations alone and one with simulated observations filling historical gaps. Removing GNSS radio occultation and the FY-3E microwave sounders degrades the second model much more than the first, and in the first some variables even improve. The authors attribute the first model's weak response to its failing to learn from instruments that were often missing in training, and the second's strong response to the idealised simulated observations, and conclude that observation importance cannot be simply evaluated within a machine-learning assimilation framework. This is a measurement under removal in a weather system that varies the training while holding the removal fixed, which the library did not hold; whether the models were retrained for each denial is not stated, and no model trained without an instrument is compared.

• In one machine-learning assimilation system, withholding the same instruments from two differently trained models gave different importance, and the authors judge observation importance not simply evaluable within such a framework (Xu et al., 2026).

Where they differ from the library

The episode's account of the three same-architecture comparisons agrees with the library's reading, with one shift of emphasis drawn from the full texts. In the Earth-observation comparison (Ulku et al., 2026), the per-scenario models are not monotone in the inputs available on any of the three image sets: on Potsdam the model trained on the false-colour infrared image alone reaches an IoU of 0.767 against 0.671 for the model trained on all three inputs, while the dropout-trained model is monotone everywhere. The dropout model's lead in 17 of 21 pairs therefore partly reflects a weak reference, a reading the authors do not offer; for the simpler baseline architecture, read across the same tables, the dropout model leads in only 11 of 21. The medical comparison (Wang et al., 2023) runs the other way in 11 of 12 cells, and the synthetic comparison (Jethani et al., 2021) shows a small deficit in all 12 cells; none of the three reports spread across repeated trainings.

Gao et al. (2022) qualify the use of retraining as the reference in a way the episode makes explicit: when predictors are correlated, the retrained importance of each tends toward zero because its neighbours substitute for it. Their CESM-LENS result therefore shows that mean-fill removal over-credits the nearly collinear Niño indices relative to retraining, and leaves open what the shared Niño signal is worth.

• The per-scenario reference in the Earth-observation comparison is non-monotone in its inputs, so the dropout model's lead there partly reflects a weak reference.
• Retraining is itself a biased reference for correlated predictors, whose retrained importance tends toward zero (Gao et al., 2022).

How they bear on the open questions

On the open question of whether a network trained with inputs randomly dropped gives the same importance ordering and effect sizes as networks retrained without each input, the evidence remains three comparisons, none reporting a defined spread across repeated trainings, with outcomes of different sign (Jethani et al., 2021; Wang et al., 2023; Ulku et al., 2026), backed by a population-level argument that the two arms share one target when masks are independent of the data (Covert et al., 2021; Le Morvan et al., 2021). None reads a source's value from both arms on purpose. The question stays open: the disagreement is unresolved, the finite-sample behaviour is untested, and the reference arm is itself unreliable in the one study designed for the comparison.

For machine-learning weather forecasting, the two arms exist only in separate systems. Aardvark's encoder was retrained on five reduced observation sets (Allen et al., 2025); a probabilistic fine-tune of the same system was read under inference-time zeroing of a stream, with predictive variance as the readout and no skill or retrained control (Almeida et al., 2026); a global forecaster trains with autoregressive input modalities masked at a probability of 0.9 and reports no experiment varying which inputs are present (Rasp et al., 2026); and FuXiWeather2 shows the removal readout changing with the training data (Xu et al., 2026). The question whether data-denial values read from one trained forecaster agree with retraining is open for weather: no study among those read sets the two side by side in one system, and the one study that varies training finds the answer moves.

The same question bears on the library's open question about running a retrieval from whatever sensors are present. With sensor dropout, Earth-observation models lose a smaller share of their own skill when a sensor is removed (Mena et al., 2024), but the authors cannot say whether the retained skill reflects adaptation, disregard of the sensor or its irrelevance, which is the measurement problem stated by Hooker et al. (2019) seen from the robustness side.

• Whether a dropout-trained network reproduces per-subset retraining is open: three comparisons disagree in sign and none reports a defined spread across repeated trainings.
• Weather forecasting has the retrained arm and the switch-off arm only in separate systems, so data-denial values from one ML forecaster have no tested reference.
• Robustness to a missing sensor does not tell whether the sensor was informative, which is the same measurement problem seen from the other side.

References

1. Allen, A., Markou, S., Tebbutt, W., Requeima, J., Bruinsma, W. P., Andersson, T. R., Herzog, M., Lane, N. D., Chantry, M., Hosking, J. S., & Turner, R. E. (2025). End-to-end data-driven weather prediction. Nature, 641, 1172-1179. https://doi.org/10.1038/s41586-025-08897-0
2. Almeida, R., Otero, N., Arndt, J., Baur, S., Samek, W., & Ma, J. (2026). Uncertainty-aware end-to-end AI weather forecasting: Disentangling observation and model contributions (arXiv:2608.30795). arXiv. https://doi.org/10.48550/arXiv.2608.30795
3. Covert, I. C., Lundberg, S., & Lee, S.-I. (2021). Explaining by removing: A unified framework for model explanation. Journal of Machine Learning Research, 22(209), 1-90. https://doi.org/10.48550/arXiv.2011.14878
4. Gao, Y., Stevens, A., Willett, R., & Raskutti, G. (2022). Lazy estimation of variable importance for large neural networks. In Proceedings of the 39th International Conference on Machine Learning. https://doi.org/10.48550/arXiv.2207.09097
5. Hooker, S., Erhan, D., Kindermans, P.-J., & Kim, B. (2019). A benchmark for interpretability methods in deep neural networks. In Advances in Neural Information Processing Systems 32. https://doi.org/10.48550/arXiv.1806.10758
6. Jethani, N., Sudarshan, M., Aphinyanaphongs, Y., & Ranganath, R. (2021). Have we learned to explain?: How interpretability methods can learn to encode predictions in their interpretations. In Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (PMLR 130). https://doi.org/10.48550/arXiv.2103.01890
7. Le Morvan, M., Josse, J., Scornet, E., & Varoquaux, G. (2021). What's a good imputation to predict with missing values? In Advances in Neural Information Processing Systems 34. https://doi.org/10.48550/arXiv.2106.00311
8. Mena, F., Arenas, D., & Dengel, A. (2024). Increasing the robustness of model predictions to missing sensors in Earth observation. MACLEAN workshop, ECML PKDD 2024. https://doi.org/10.1007/978-3-032-25311-8_14
9. Rasp, S., Babenko, B., Masters, D., El-Kadi, A., et al. (2026). WeatherNext 3: Increasing resolution and performance of global weather models with raw observations (arXiv:2609.03582). arXiv. https://doi.org/10.48550/arXiv.2609.03582
10. Ulku, I., Akagündüz, E., & Tanrıöver, Ö. Ö. (2026). Robust multispectral semantic segmentation under missing or full modalities via structured latent projection (arXiv:2604.15856). arXiv. https://doi.org/10.48550/arXiv.2604.15856
11. Wang, H., Chen, Y., Ma, C., Avery, J., Hull, L., & Carneiro, G. (2023). Multi-modal learning with missing modality via shared-specific feature modelling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://doi.org/10.1109/CVPR52729.2023.01524
12. Xu, X., Sun, X., Zhu, S., Zhong, X., Huang, Y., Zhu, Z., Liu, J., & Li, H. (2026). FuXiWeather2: Learning accurate atmospheric state estimation for operational global weather forecasting (arXiv:2603.15358). arXiv. https://doi.org/10.48550/arXiv.2603.15358

Read into the library after publication:

• FuXiWeather2: Learning accurate atmospheric state estimation for operational global weather forecasting arXiv:2603.15358

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library records the loss of strong echoes in latent compression as an author statement (Gong et al., 2024) and holds the tokenizer-loss experiment of Franch et al. (2025), but it does not hold the magnitudes needed to set the compression step against the forecast step. Read off Figure 4 of Gong et al. (2024), the reconstruction critical success index on SEVIR stays above 0.9 at the lowest VIL threshold for every latent width tested, while at the highest threshold it reaches 0.57, 0.71 and 0.77 for latents of 48 × 48 × 1, 48 × 48 × 2 and 48 × 48 × 4; each doubling of the channels raises the top-threshold score by 0.14 and then 0.07, against 0.03 and 0.01 at the lowest threshold. The same paper reports a forecast critical success index of 0.19 at that threshold. The paper does not state which width its forecaster used, and the comparison sets a pixel-matching score on reconstructions against the same score on forecasts, where displacement is also penalised.

Zhu et al. (2026) report the only experiment found that runs one generator in latent and in pixel space with the rest of the framework unchanged. On SEVIR coarsened to 128 × 128, conditional flow matching in pixel space reached an overall critical success index of 0.200, against 0.212 and 0.213 with latents of 16² and 32², a deficit the authors attribute to optimisation collapse; Leinonen et al. (2023) had found earlier that their pixel-space attempts trained less stably. No per-threshold score is given for these variants. Leinonen et al. (2023) also document intensity-related design facts the library does not record: a layer normalisation inherited from image models often shifted forecast magnitudes, and removing it let the model reproduce intensity better; training samples were drawn equally from eleven intensity classes; and every reported score follows probability matching to the observed distribution, which removes much of any intensity bias before scoring.

• On SEVIR, an autoencoder's reconstruction score at the highest echo threshold is 0.57 to 0.77 depending on latent width, against 0.19 for the forecasts built on it (Gong et al., 2024).
• The one within-paper swap of latent for pixel space, for a flow-matching generator on coarsened SEVIR, scored the pixel version slightly lower on an all-threshold average and reported no heavy-rain score (Zhu et al., 2026).

Where they differ from the library

The library's gap record states that latent compression systematically attenuates extreme radar intensities and caps the extreme-precipitation skill of latent diffusion. The sources read here qualify both parts. The attenuation varies with latent width (Gong et al., 2024) and with the reconstruction loss: for one tokenizer and dataset, a magnitude-weighted error raised the frequency bias at 50 mm h⁻¹ from 0.22 to 0.92 (Franch et al., 2025). On the cap, the top-threshold reconstruction score in Gong et al. (2024) lies far above the forecast score, and two models with improved first stages still under-forecast (Franch et al., 2025; Schusterbauer et al., 2026). For those models the binding limit therefore sits after compression, with the reservation that the scores mix displacement with intensity and that neither study reports a heavy-rain bias for its forecasts.

The library's evidence that pixel-space generation avoids the attenuation is cross-paper: residual diffusion in pixel space beat the latent PreDiff on overall scores, with different architectures and training, and kept high-value echoes better than its own deterministic backbone (Yu et al., 2024). The one within-paper swap points the other way on an average over all thresholds (Zhu et al., 2026), while that paper's preferred pixel-space generator, which beats the latent baselines, was never run in latent space. The two sources disagree on the direction of the effect of the space alone, and neither measures it at heavy-rain thresholds.

Schusterbauer et al. (2026) attribute the residual underestimation to the predominance of light rain in the training data, and the library records that account. The same paper describes SEVIR as sampled toward storms, and Leinonen et al. (2023), training on intensity-balanced samples, obtained mixed threshold skill against an adversarial baseline and offered their logarithmic training target as a possible factor, and they related their shortfall at 10 mm h⁻¹ on German radar partly to that region's climatic similarity to the baseline's British training data. In the sources read, the rarity account rests on data already enriched in heavy precipitation and has not been tested by intervention.

• The loss of strong echoes in compression depends on latent width and reconstruction loss, so in the tests read it is a design and training property of the autoencoder.
• In two models whose first stage was improved, the forecasts still came out too low, which points to a limit after compression in those models, though neither measured a heavy-rain bias for its forecasts.
• Cross-paper and within-paper comparisons disagree on whether pixel space helps, and neither measures heavy rain directly.

How they bear on the open questions

The gap's first open question asks whether the attenuation can be removed by higher-dimensional or intensity-weighted latents or is intrinsic to compressed-latent generation. For reconstruction, two independent tests answer part of it within their conditions: wider latents on SEVIR (Gong et al., 2024) and a magnitude-weighted loss on Emilia-Romagna radar (Franch et al., 2025) both raise the reconstruction of strong echoes substantially, so the size of the attenuation is set by design and training. Neither test removes it. The reconstruction critical success index reaches about 0.77 at the highest threshold in Gong et al. (2024) and 0.44 at 50 mm h⁻¹ in Franch et al. (2025), and only Franch et al. (2025) measure intensity apart from placement: the weighted loss restores almost the full count of pixels above 50 mm h⁻¹, with many of them misplaced. Whether the same remedies raise heavy rain in forecasts has not been tested in the sources read, so the question stays open, in part for reconstruction and wholly for forecasts. The second open question, how much of generative heavy-rain decay comes from compression and how much from training imbalance, stays open on every count in the sources read: none reports a heavy-rain frequency bias for decoded observations and for forecasts on the same events, the within-paper swap reports no heavy-rain score (Zhu et al., 2026), and none varies the share of heavy rain in training with the model held fixed. The related question in the library's record of heavy-rain skill at long lead, whether intensity decay is a sampling artefact or an effect of imbalance, remains open for the same reasons.

The diagnostic that would separate the two steps exists in a neighbouring task. For tropical cyclone forecasting with a latent rectified flow, Zannat and Ahsan (2026) used a reconstruction ceiling to place the error budget in the generator, though for average fidelity, not for the intensity tail, and an early bounded decoder of theirs clipped more than half of the pressure range. Applied to heavy-rain frequency, the same ceiling would split the radar problem. Because classifier-free guidance shifts forecast intensity in both FREUD and CasCast (Schusterbauer et al., 2026), the library's open question on guidance weights bears directly on this one: any heavy-rain bias needs to be reported at a stated guidance setting and before probability matching. The rarity mechanism also links to the precipitation domain: a conditional diffusion retrieval over the southeastern United States matched the radar intensity distribution except for a slight underestimation above 30 mm h⁻¹, in patches selected for widespread rain (Guilloteau et al., 2025), and the same split of decoder and generator contributions would serve any retrieval that generates in a latent space.

• Whether the missing heavy rain comes mainly from compression or from training imbalance remains open, because none of the studies read reports a heavy-rain bias for decoded observations and forecasts on the same events.
• A reconstruction-ceiling check already used in tropical cyclone forecasting would split the radar problem if it were scored on heavy-rain frequency at a stated guidance setting.

References

1. Franch, G., Tomasi, E., Wanjari, R., Poli, V., Cardinali, C., Alberoni, P. P., & Cristoforetti, M. (2025). GPTCast: A weather language model for precipitation nowcasting. Geoscientific Model Development, 18, 5351–5371. https://doi.org/10.5194/gmd-18-5351-2025
2. Gong, J., Bai, L., Ye, P., Xu, W., Liu, N., Dai, J., Yang, X., & Ouyang, W. (2024). CasCast: Skillful high-resolution precipitation nowcasting via cascaded modelling. In Proceedings of the 41st International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 235, pp. 15809–15822). PMLR. https://proceedings.mlr.press/v235/gong24a.html
3. Guilloteau, C., Kerrigan, G., Nelson, K., Migliorini, G., Smyth, P., Li, R., & Foufoula-Georgiou, E. (2025). A generative diffusion model for probabilistic ensembles of precipitation maps conditioned on multisensor satellite observations. IEEE Transactions on Geoscience and Remote Sensing, 63, 5402415. https://doi.org/10.1109/TGRS.2025.3548518
4. Leinonen, J., Hamann, U., Nerini, D., Germann, U., & Franch, G. (2023). Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2304.12891
5. Schusterbauer, J., Wiese, J., Stracke, N., Phan, T., & Ommer, B. (2026). Probabilistic precipitation nowcasting with rectified flow transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026). https://doi.org/10.48550/arXiv.2605.31204
6. Yu, D., Li, X., Ye, Y., Zhang, B., Luo, C., Dai, K., Wang, R., & Chen, X. (2024). DiffCast: A unified framework via residual diffusion for precipitation nowcasting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). https://doi.org/10.1109/CVPR52733.2024.02622
7. Zannat, M., & Ahsan, S. M. M. (2026). Tropical cyclone forecasting via latent rectified flow using satellite imagery and atmospheric fields [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2608.08354
8. Zhu, Y., Shi, C., Feng, Y., & Niu, D. (2026). PixelFlowCast: Latent-free precipitation nowcasting via pixel mean flows [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2605.10046

Read into the library after publication:

• PixelFlowCast: Latent-Free Precipitation Nowcasting via Pixel Mean Flows arXiv:2605.10046

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library holds the development of STP (Thompson et al., 2003), its effective-layer application to a 2003–2011 mode-stratified sample (Thompson et al., 2012), and the one test outside the United States (Zhang et al., 2025), but it holds no skill score for STP on a large sample built after its development. Coffer et al. (2019) supply one. Recalculating parameters for 20,194 right-moving supercells from the Storm Prediction Center's mesoanalysis archive, tornadic events from 2005 to 2017 and significantly severe non-tornadic events from 2005 to 2015, they report a maximum true skill statistic of 0.364 for effective-layer STP and 0.425 for the fixed-layer version in discriminating significantly tornadic from non-tornadic supercells, both below the 0.46 that Thompson et al. (2003) obtained on their development sample, while 0–500 m storm-relative helicity alone reaches 0.529 and exceeds every STP variant.

Thompson et al. (2012) report no skill score for STP, but the cumulative frequencies in their Table 5 allow one to be computed. For right-moving supercells from March 2005 to 2011, a threshold of one is exceeded by 0.20 of non-tornadic events and by about 0.70 of EF2 and stronger events weighted by count, implying a true skill statistic near 0.50 for that pairing, with weak tornadoes excluded, unlike the development score, which set significant tornadoes against all other supercells; thresholds of 0.5 and 2 give about 0.48 and 0.40. This is a calculation made for the present episode from published frequencies, on a sample whose effective-layer years follow the 1999–2001 and 2003–2004 development samples (Thompson et al., 2003; Thompson et al., 2004).

• On 20,194 later American supercells, effective-layer STP scores a true skill of 0.36, below the 0.46 of its development sample, and 0–500 m helicity alone scores higher than any STP variant (Coffer et al., 2019).
• The frequencies published by Thompson et al. (2012) imply a skill near 0.5 at a threshold of one on 2005–2011 cases with weak tornadoes left out, though the paper itself reports no score and the larger recalculation by Coffer et al. (2019) gives less.

Where they differ from the library

The library's reading of Zhang et al. (2025) takes the United States comparator from the paper itself, which states that STP's skill there is usually higher than 0.45 (Thompson et al., 2003, 2004). Both sources score the index on the samples from which its constants and threshold were derived. On the larger later sample of Coffer et al. (2019) the reference falls to 0.36–0.43, so the gap between United States and Chinese skill (0.29 and 0.14) is smaller than the library's synthesis implies, though still substantial. Coffer et al. (2025) add that dataset construction alone shifts the score within the United States: the same formula reaches 0.368 on the Storm Prediction Center dataset and 0.528 on GridRad-Severe supercells with HRRR inflow soundings, a difference the authors attribute partly to a null sample that is almost entirely significantly severe in the former. The comparison between countries therefore confounds region with analysis system, null construction and season, as Zhang et al. (2025) themselves suggest in noting the low spatial and temporal overlap of their tornadic and non-tornadic cases.

The library's gap record treats the overlap between the development and verification samples of Thompson et al. (2012) as unresolved. The paper's archive notes resolve the timing: effective-layer parameters exist only from March 2005, after the 1999–2001 and 2003–2004 development years. Independence in time is thus largely met, while independence of authorship, analysis lineage and null definition is not, and the authors do not present the sample as a verification.

• The American benchmark of about 0.45 that the China comparison relies on is an in-sample score, so the regional drop is smaller than usually stated but still large.
• Within the United States, changing how the sample is built moves STP's score from 0.37 to 0.53, so the US–China contrast cannot separate region from data and null construction.
• The years of the 2012 sample follow the development years, so its independence in time is largely met, while authorship, analysis lineage and null definition are shared.

How they bear on the open questions

On the open question of whether the STP threshold of one holds on independent multi-year samples, the evidence now gives a qualified yes for discrimination at the analysis hour: Coffer et al. (2019) find an optimal threshold of 0.86 for effective-layer STP with lower skill than the development figure, and the frequencies in Thompson et al. (2012) place the best threshold near one. On the second requirement named by Doswell and Schultz (2006), skill as a function of lead time, no source read here reports anything; every score uses an analysis from about the time of the event, the closest analysis hour in Coffer et al. (2019) and the closest prior hour in the other studies that state it. The two estimates of independent-sample skill also disagree: the frequencies in Thompson et al. (2012) imply about 0.50, whereas Coffer et al. (2019), on overlapping years and the same kind of null sample with recalculated parameters, report 0.364. On whether a region-refit composite holds outside its calibration sample, the refit STP300cn of Zhang et al. (2025) has been scored only on the Chinese cases it was calibrated on, 57 significantly tornadic against 143 non-tornadic supercells, where it exceeds 0–300 m helicity or shear by 0.03. Under the criterion that work on a question does not settle it, the question of STP's forecast value remains open in both countries, and the Chinese case list and code published by Zhang et al. (2025) make an out-of-sample test on later Chinese cases feasible.

Across the three studies read here, four samples in all, the best low-level kinematic ingredient matches or exceeds the composite (Coffer et al., 2019, 2025; Zhang et al., 2025), which bears on the library's question of how vertical structure, rather than bulk products, determines outcomes. The depth of the best layer is not settled: with Bunkers motion estimates the shallowest layers win, whereas with observed storm motion Coffer et al. (2025) find 0–3 km helicity overtaking 0–500 m, so a forecast-time composite and a diagnostic one may need different kinematic terms.

• No study read here reports STP's skill at any lead time before the event, so its forecast value in Doswell and Schultz's sense remains unmeasured.
• The Chinese refit has been scored only on its calibration cases, and the published case list makes an independent test on later Chinese storms feasible.

References

1. Coffer, B. E., Parker, M. D., Coniglio, M. C., & Homeyer, C. R. (2025). Supercell environments using GridRad-Severe and the HRRR: Addressing discrepancies between prior tornado datasets. Weather and Forecasting, 40(8), 1405–1428. https://doi.org/10.1175/WAF-D-24-0251.1
2. Coffer, B. E., Parker, M. D., Thompson, R. L., Smith, B. T., & Jewell, R. E. (2019). Using near-ground storm relative helicity in supercell tornado forecasting. Weather and Forecasting, 34(5), 1417–1435. https://doi.org/10.1175/WAF-D-19-0115.1
3. Doswell, C. A., III, & Schultz, D. M. (2006). On the use of indices and parameters in forecasting severe storms. Electronic Journal of Severe Storms Meteorology, 1(3), 1–22.
4. Thompson, R. L., Edwards, R., Hart, J. A., Elmore, K. L., & Markowski, P. (2003). Close proximity soundings within supercell environments obtained from the Rapid Update Cycle. Weather and Forecasting, 18(6), 1243–1261. https://doi.org/10.1175/1520-0434(2003)018<1243:CPSWSE>2.0.CO;2
5. Thompson, R. L., Edwards, R., & Mead, C. M. (2004). An update to the supercell composite and significant tornado parameters [Preprint P8.1]. 22nd Conference on Severe Local Storms, Hyannis, MA, United States. American Meteorological Society. https://www.spc.noaa.gov/publications/thompson/stp_scp.pdf
6. Thompson, R. L., Smith, B. T., Grams, J. S., Dean, A. R., & Broyles, C. (2012). Convective modes for significant severe thunderstorms in the contiguous United States. Part II: Supercell and QLCS tornado environments. Weather and Forecasting, 27(5), 1136–1154. https://doi.org/10.1175/WAF-D-11-00116.1
7. Zhang, R., Xue, M., & Yu, X. (2025). Environments of tornadic and non-tornadic supercells in China and optimized significant tornado parameter for China region. Quarterly Journal of the Royal Meteorological Society, 151(772), e5027. https://doi.org/10.1002/qj.5027

Read into the library after publication:

• Using Near-Ground Storm Relative Helicity in Supercell Tornado Forecasting 10.1175/WAF-D-19-0115.1

Against the library

How this episode sits against the listener's own paper library. Coverage: partial.

What the sources add

The library records that Earth-system causal discovery lacks benchmarks with known ground truth, but it holds no attempt to supply one. Stein et al. (2025) provide what is, by their own survey of existing datasets, the largest real-data benchmark for time-series causal discovery, set in hydrology: fifteen-minute discharge from 666 gauges in eastern Germany and 494 in Bavaria for 2019 to 2023, with a ground-truth graph of hydrological connectivity assembled from agency metadata, remote sensing and river descriptions and checked edge by edge by hand. The data carry hidden confounding by rainfall, flow-dependent and therefore variable lags, nonstationarity and missing values. On randomly sampled connected subgraphs of three and five stations, a linear vector autoregression reached an AUROC of 0.82 and 0.80, PCMCI with a linear conditional independence test 0.65 on both, and a baseline that orients each link from the smaller to the larger mean discharge 0.79 and 0.75; two nonlinear deep-learning methods showed no advantage over linear ones. Each score is the mean at the best hyperparameter combination, selected with the labels, which the authors note may overestimate performance.

Das et al. (2026), a preprint, supply the only comparison in the sources read here in which the same methods are scored on synthetic systems and on this benchmark. On a thirty-variable synthetic structural vector autoregression the five leading methods scored an AUROC of about 0.85 to 0.91 for lagged links, whereas on river subgraphs of ten and twenty stations every method fell between about 0.49 and 0.64, and the method ranked fifth on the synthetic system ranked first at twenty stations. Standard deviations of up to about 0.13 across five subgraph draws make that reordering indistinguishable from sampling noise, so the comparison establishes the drop in level more firmly than any change in rank.

• A real-data benchmark with physically known causal structure now exists for geoscientific time series, built from more than a thousand German river gauges (Stein et al., 2025).
• On its random three- and five-station subgraphs, a linear vector autoregression scored best among formal methods, PCMCI scored clearly lower, and a rule based only on mean discharge came within a few hundredths of the top.
• The one comparison of the same methods on synthetic and river data shows a large drop in scores on real rivers but too much seed-to-seed noise to establish a change in ranking (Das et al., 2026).

Where they differ from the library

Five of the library's applied studies validate their real-data networks without known truth: by recognition of models and model families together with two artificial reference datasets (Nowack et al., 2020), by the physical plausibility of the strongest links (Debeire et al., 2025), by a linear model refitted to the recovered graph that reproduces most lagged correlations but not the Atlantic index's own autocorrelation (Karmouche et al., 2023), against a hypothesised mechanism (Falkena et al., 2025), and against plant-physiological expectation after calibration on a synthetic model fitted to 72 flux sites (Krich et al., 2020). All five use the PCMCI family, which Stein et al. (2025) found below both a linear Granger model and a magnitude rule on randomly sampled subgraphs of these rivers. The conditions do not match, however: the river test used three to five variables and very long records, whereas the library's studies use dozens of pressure modes or a handful of yearly indices or flux variables, and PCMCI was designed for high-dimensional, autocorrelated settings (Runge et al., 2019). PCMCI also scored second best of all methods on one subgraph set. The benchmark therefore qualifies the assumption that a method validated on synthetic data performs as expected on real data, without amounting to a verdict on the networks these studies report.

A real-data benchmark is not free of features that favour particular methods. Discharge increases downstream, so orientation by magnitude alone scores well, and an unreviewed audit reports that the linear autoregression's AUROC falls from about 0.77 on raw discharge to 0.59 after standardisation (Matin, n.d.). Stein et al. (2025) motivate their benchmark by the inflated performance that synthetic benchmarks can report; the same risk has a counterpart in real data whose structure carries a simple cue.

• Five applied causal network studies in the library validate by consistency, plausibility or synthetic surrogates, and none checks a real link against a known truth.
• The river result qualifies trust in synthetic validation, but it was obtained with three to five variables and settings chosen with the labels, so it is no verdict on the PCMCI networks in the library.
• Real-data benchmarks carry their own shortcuts: downstream growth in discharge lets magnitude alone score well.

How they bear on the open questions

On the open question of whether real or modelled Earth-system datasets with confidently known causal structure have accumulated since the call in Runge et al. (2019), Stein et al. (2025) give a partial answer: one large real dataset now exists, for a one-way, acyclic hydrological network in which almost every gauge has a single downstream neighbour and whose ground truth records hydrological connectivity without delays rather than a graph of contemporaneous links, which is why Das et al. (2026) scored lagged links only. On the second open question, whether method rankings on synthetic surrogates transfer to real observations, the evidence remains insufficient. Stein et al. (2025) run no synthetic comparison, and the one side-by-side, by Das et al. (2026), cannot separate a change in ranking from noise. The question stays open, and the field-level statement of absence itself rests on two reviews from overlapping author groups (Camps-Valls et al., 2023; Runge et al., 2019).

The benchmark cannot address the structures that define the library's neighbouring domains. The subpolar gyre mechanism tested by Falkena et al. (2025) contains a fast negative and a delayed positive feedback loop. The lagged link from the gyre back to salinity that closes the delayed loop was found in no model, which the authors attribute in part to the lagged regression behind the theory, proposing instead a positive loop on a shorter timescale through the gyre's own memory; the density-to-gyre link mostly runs against the theory, which they attribute to eddy parameterisation and the choice of gyre index. With no known truth, the study cannot decide among these explanations. Krich et al. (2020) state that their synthetic test model omits feedback loops although they are important in natural systems, and the library's land-atmosphere domain is organised around feedback from the surface onto convection. The fifteen-minute resolution of the river data would allow the question of how time aggregation turns lagged links contemporaneous to be tested against a known graph, but Stein et al. (2025) report only each method's best resolution. Whether rankings obtained on synthetic systems or on rivers carry into real feedback systems therefore remains untested in the sources read. Synthetic systems with known loops already appear in these sources: Das et al. (2026) also score methods on the Lorenz-96 system, in which neighbouring variables drive one another, yet a ranking obtained there is itself a ranking on synthetic data, the kind whose transfer is in question. Settling it needs a real system, or a realistic model, whose feedback is known with confidence.

• The river benchmark partly answers whether real datasets with known causal structure exist, but only for a one-way network without feedback.
• Whether method rankings transfer from synthetic data to real systems, and in particular to feedback systems such as the subpolar gyre or land-atmosphere coupling, remains open.

References

1. Camps-Valls, G., Gerhardus, A., Ninad, U., Varando, G., Martius, G., Balaguer-Ballester, E., Vinuesa, R., Diaz, E., Zanna, L., & Runge, J. (2023). Discovering causal relations and equations from data. Physics Reports, 1044, 1–68. https://doi.org/10.1016/j.physrep.2023.10.005
2. Das, S., Chakraborty, D., & Maulik, R. (2026). SC3D: Dynamic and differentiable causal discovery for temporal and instantaneous graphs (arXiv:2602.02830v3) [Preprint]. arXiv. https://arxiv.org/abs/2602.02830
3. Debeire, K., Bock, L., Nowack, P., Runge, J., & Eyring, V. (2025). Constraining uncertainty in projected precipitation over land with causal discovery. Earth System Dynamics, 16, 607–630. https://doi.org/10.5194/esd-16-607-2025
4. Falkena, S. K. J., Dijkstra, H. A., & von der Heydt, A. S. (2025). Causal mechanisms of subpolar gyre variability in CMIP6 models. Earth System Dynamics, 16, 1833–1844. https://doi.org/10.5194/esd-16-1833-2025
5. Karmouche, S., Galytska, E., Runge, J., Meehl, G. A., Phillips, A. S., Weigel, K., & Eyring, V. (2023). Regime-oriented causal model evaluation of Atlantic–Pacific teleconnections in CMIP6. Earth System Dynamics, 14, 309–344. https://doi.org/10.5194/esd-14-309-2023
6. Krich, C., Runge, J., Miralles, D. G., Migliavacca, M., Perez-Priego, O., El-Madany, T., Carrara, A., & Mahecha, M. D. (2020). Estimating causal networks in biosphere–atmosphere interaction with the PCMCI approach. Biogeosciences, 17, 1033–1061. https://doi.org/10.5194/bg-17-1033-2020
7. Matin. (n.d.). causal-rivers-lab [Computer software]. GitHub. https://github.com/Mathindata/causal-rivers-lab
8. Nowack, P., Runge, J., Eyring, V., & Haigh, J. D. (2020). Causal networks for climate model evaluation and constrained projections. Nature Communications, 11, 1415. https://doi.org/10.1038/s41467-020-15195-y
9. Runge, J., Bathiany, S., Bollt, E., Camps-Valls, G., Coumou, D., Deyle, E., Glymour, C., Kretschmer, M., Mahecha, M. D., Muñoz-Marí, J., van Nes, E. H., Peters, J., Quax, R., Reichstein, M., Scheffer, M., Schölkopf, B., Spirtes, P., Sugihara, G., Sun, J., . . . Zscheischler, J. (2019). Inferring causation from time series in Earth system sciences. Nature Communications, 10, 2553. https://doi.org/10.1038/s41467-019-10105-3
10. Stein, G., Shadaydeh, M., Blunk, J., Penzel, N., & Denzler, J. (2025). CausalRivers – Scaling up benchmarking of causal discovery for real-world time-series. In International Conference on Learning Representations (ICLR 2025). https://arxiv.org/abs/2503.17452

Read into the library after publication:

• CausalRivers - Scaling up benchmarking of causal discovery for real-world time-series arXiv:2503.17452
• SC3D: Dynamic and Differentiable Causal Discovery for Temporal and Instantaneous Graphs arXiv:2602.02830

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library holds the storm-scale diffusion route and one verification study of spread placement, but none of the three global papers that define the other two routes. Bülte et al. (2026) show that three initial-condition perturbation schemes applied to Pangu-Weather leave the ensemble clearly under-dispersed for near-surface temperature and wind over 2022, while post-processing methods are better calibrated but return marginal distributions rather than spatial fields. Mahesh et al. (2025) add an ablation the library lacks: in an SFNO ensemble, independently trained checkpoints alone are under-dispersed at early leads and bred vectors alone at three to five days, and only their combination brings the spread-error ratio towards one at longer leads, with the full ensemble still under-dispersed at early leads for every verified variable except 500 hPa geopotential.

Lang et al. (2024) supply the CRPS-trained route and complicate it. AIFS-CRPS is over-dispersive in the extratropics, most visibly for 500 hPa geopotential in the early medium range, and under-spread relative to the IFS ensemble in the tropics; because its members are initialised from the operational IFS ensemble, the authors attribute the excess to inherited singular-vector perturbations and report improved reliability in first tests with a revised perturbation amplitude. The deployed CRPS-trained system therefore combines learned noise with the initial-condition route, and its dispersion is set in part by an initial-perturbation amplitude.

• In the papers read, initial-condition perturbation alone left Pangu-Weather under-dispersed near the surface (Bülte et al., 2026).
• In one SFNO ensemble, model checkpoints and bred vectors supplied spread at different lead times, and only their combination approached a ratio of one (Mahesh et al., 2025).
• The CRPS-trained AIFS ensemble is over-dispersive in the extratropics because it inherits operational initial perturbations, so its spread is partly set by an initial-perturbation amplitude (Lang et al., 2024).

Where they differ from the library

The new sources qualify the library's framing of dispersion as a storm-scale generative deficit. Under-dispersion is equally reported for global ensembles built by initial-condition perturbation (Bülte et al., 2026; Mahesh et al., 2025), and over-dispersion appears once a CRPS-trained model inherits operational initial perturbations (Lang et al., 2024). The direction of the miscalibration thus depends on how the members are made and on tuned amplitudes, not only on whether a diffusion sampler is involved, which the library's account of StormCast (Pathak et al., 2025), CorrDiff (Mardani et al., 2025) and HRRRCast (Abdi et al., 2026) does not by itself establish.

Mahesh et al. (2025) also extend what counts as controlled attribution. The library's only controlled evidence was the SFNO lagged-ensemble ablation of Brenowitz et al. (2025); the checkpoint and bred-vector ablation is a second controlled decomposition, from an overlapping author group and again on a global SFNO. Both decompose the total by lead time or training choice, and neither examines the spatial placement of spread, so the library's single placement test (Bonte et al., 2026) stands without a counterpart among the global or storm-scale systems.

• In the global systems read here, the direction of miscalibration depends on how members are made and how amplitudes are tuned, so under-dispersion is reported beyond storm-scale diffusion ensembles.
• A second controlled decomposition of spread exists outside the library, but like the first it decomposes totals and leaves spatial placement untested.

How they bear on the open questions

On the open question of why noise-only storm-scale diffusion ensembles are under-dispersed, the episode's sources leave it open. Four such ensembles in the library are under-dispersed, StormCast, CorrDiff, HRRRCast and the radar-nowcast refinement of Park et al. (2026), from three author groups, and none varies the suspected cause; the training-side mechanism isolated by Brenowitz et al. (2025) concerns a global model whose spread comes from initialisation time. The question of initial-condition versus model-error origin raised for physics-based storm-scale ensembles is not touched by any of these papers.

On whether ensemble spread is located where the error is, the question stays open. Bonte et al. (2026) find, for STEPS and an out-of-domain LDCast over ten convective and ten stratiform Belgian events, a variance-to-error ratio near one above 5 km but an error projection onto the members' span only slightly above that of phase-randomised surrogates; this is one preprint, one region, nowcasts to 100 minutes and a model never retrained there. None of the global or storm-scale papers tests placement. Almeida et al. (2026) attribute an end-to-end ensemble's variance to observation and model components, which bears on where spread comes from but again as domain totals. A comparison against phase-randomised surrogates, applied to a medium-range or storm-scale machine-learning ensemble, is the untested condition.

• Why noise-only storm-scale diffusion ensembles are under-dispersed remains open: four instances, three groups, no experiment that varies the cause.
• Whether spread sits where the error is has been tested once, for radar nowcasts over Belgium, and remains untested for medium-range and storm-scale machine-learning ensembles.

References

1. Abdi, D., Jankov, I., Madden, P., Vargas, V., Smith, T. A., Frolov, S., Flora, M., & Potvin, C. (2026). HRRRCast: A data-driven emulator for regional weather forecasting at convection allowing scales. Artificial Intelligence for the Earth Systems. https://doi.org/10.1175/aies-d-25-0061.1
2. Almeida, R., Otero, N., Arndt, J., Baur, S., Samek, W., & Ma, J. (2026). Uncertainty-aware end-to-end AI weather forecasting: Disentangling observation and model contributions [Preprint]. arXiv. https://arxiv.org/abs/2608.30795
3. Bonte, M., De Cruz, L., Debal, F., & Vannitsem, S. (2026). Spread/Error relationship and spatial error structure of precipitation ensemble nowcasting: Comparison of STEPS and generative AI [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2601.19298
4. Brenowitz, N. D., Cohen, Y., Pathak, J., Mahesh, A., Bonev, B., Kurth, T., Durran, D. R., Harrington, P., & Pritchard, M. S. (2025). A practical probabilistic benchmark for AI weather models. Geophysical Research Letters. https://doi.org/10.1029/2024gl113656
5. Bülte, C., Horat, N., Quinting, J., & Lerch, S. (2026). Uncertainty quantification for data-driven weather models. Artificial Intelligence for the Earth Systems, 5(1). https://doi.org/10.1175/aies-d-24-0049.1
6. Lang, S., Alexe, M., Clare, M. C. A., Roberts, C., Adewoyin, R., Ben Bouallègue, Z., Chantry, M., Dramsch, J., Dueben, P. D., Hahner, S., Maciel, P., Prieto-Nemesio, A., O'Brien, C., Pinault, F., Polster, J., Raoult, B., Tietsche, S., & Leutbecher, M. (2024). AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score [Preprint]. arXiv. https://arxiv.org/abs/2412.15832
7. Mahesh, A., Collins, W. D., Bonev, B., Brenowitz, N., Cohen, Y., Elms, J., Harrington, P., Kashinath, K., Kurth, T., North, J., O'Brien, T., Pritchard, M., Pruitt, D., Risser, M., Subramanian, S., & Willard, J. (2025). Huge ensembles – Part 1: Design of ensemble weather forecasts using spherical Fourier neural operators. Geoscientific Model Development, 18, 5575–5603. https://doi.org/10.5194/gmd-18-5575-2025
8. Mardani, M., Brenowitz, N., Cohen, Y., Pathak, J., Chen, C.-Y., Liu, C.-C., Vahdat, A., Nabian, M. A., Ge, T., Subramaniam, A., Kashinath, K., Kautz, J., & Pritchard, M. (2025). Residual corrective diffusion modeling for km-scale atmospheric downscaling. Communications Earth & Environment. https://doi.org/10.1038/s43247-025-02042-5
9. Park, D., Song, C., Chang, T.-Y., Ham, Y.-G., & Hong, Y. (2026). Diffusion-based refinement for kilometer-scale probabilistic precipitation nowcasting [Preprint]. arXiv. https://arxiv.org/abs/2608.30205
10. Pathak, J., Cohen, Y., Garg, P., Harrington, P., Brenowitz, N., Durran, D., Mardani, M., Vahdat, A., Xu, S., Kashinath, K., & Pritchard, M. (2025). Kilometer-scale convection allowing model emulation using generative diffusion modeling. Science Advances. https://doi.org/10.1126/sciadv.adv0423

Read into the library after publication:

• Uncertainty quantification for data-driven weather models 10.1175/aies-d-24-0049.1
• Huge ensembles – Part 1: Design of ensemble weather forecasts using spherical Fourier neural operators 10.5194/gmd-18-5575-2025
• AIFS-CRPS: Ensemble forecasting using a model trained with a loss function based on the Continuous Ranked Probability Score arXiv:2412.15832

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library already holds all seven papers behind this episode and an open question that states its subject, so the episode adds a re-reading rather than new sources. Three details from the full texts sharpen what the library records. First, the pixel-based, neighbourhood-based and patch-based infrared estimators were already compared in Hong et al. (2004): at daily, 0.25-degree scale on 8 July 1999 over the western United States, the correlation with radar rose from 0.36 for a fixed threshold to 0.54 for PERSIANN and 0.75 for PERSIANN-CCS. That is two days in one region, and the three estimators were also calibrated on different rain data (a fixed threshold, microwave estimates, and radar over the same region), so the ladder is an early indication of what context adds beyond the co-located pixel, the quantity Ombadi et al. (2021) hold fixed by assumption.

Second, So and Shin (2018) separate two things that the phrase 'warm-rain insensitivity' merges. A water-vapour minus window brightness temperature difference distinguishes shallow from non-shallow precipitating clouds with a Heidke skill score of 0.82 against the GPM radar, while the rain rate within the shallow class shows no inferable relation to brightness temperature. Detection of the regime and estimation within it therefore fail to different degrees. Third, the only retrieval paper here that claims to recover warm-cloud rain (Hayatbini et al., 2019) supports the claim with maps for a single half hour and with overall scores for July 2018, in which the false alarm ratio is 0.68 against 0.62 for PERSIANN-CCS; no score conditioned on cloud-top temperature is given.

• An early comparison already pointed to context: on one July day in 1999, correlation with radar rose from 0.36 for a pixel threshold to 0.75 for a cloud-patch method, with calibration data differing between the methods (Hong et al., 2004).
• Extra channels can identify a shallow raining cloud with high skill while leaving its rain rate undetermined (So & Shin, 2018).
• The one claim of recovered warm-cloud rain rests on a single half hour of maps and overall scores (Hayatbini et al., 2019).

Where they differ from the library

The episode narrows two statements. The library's summary describes Oya as scoring above IMERG Early and close to IMERG Final (Brempong et al., 2025). The paper's Africa ablation shows where that margin comes from: a model given only the 10.7-micron channel reaches a critical success index of 0.32 for light rain against 0.49 with all channels, a difference larger than the margin over either IMERG version. Because training target and evaluation reference are the same GPM combined product, the ranking measures agreement with that product at overpass times.

The episode also counts the sources on warm rain differently. Hong et al. (2004), Hayatbini et al. (2019) and Ombadi et al. (2021) come from one group, so they are treated as one source alongside So and Shin (2018) and Pfreundschuh et al. (2026). Hong et al. (2004) do not themselves state that warm rain is missed; the paper shows a segmentation that stops at 253 K and a class of small patches with minimum temperature above 245 K whose temperature-rain slope is about zero, and the explicit statement is made fifteen years later by Hayatbini et al. (2019). The two neural retrievals are also not fully independent of each other or of the review that frames the problem, since one author is common to Brempong et al. (2025), Pfreundschuh et al. (2026) and Kidd et al. (2021).

• Oya's margin over IMERG for light rain is smaller than what its own extra channels contribute, and it is measured against the product Oya was trained on (Brempong et al., 2025).
• The warm-rain weakness is supported by three independent groups of authors, with the PERSIANN and information-content papers counting as one.

How they bear on the open questions

The library's open question asks how much of the accuracy gap to microwave and radar estimates learned infrared retrievals can close, and in which regimes they cannot. On the first half the evidence has moved: two retrievals with different inputs report scores at or above parts of IMERG under stated conditions, and both are preprints. On the second half the question stays open in substance. Shallow warm rain is reported as a weakness by three sets of authors and is scored separately by none of the retrievals; in Oya's own account heavy rain remains below IMERG Final and skill declines with latitude (Brempong et al., 2025); and GPROF-IR stays below the microwave estimates over Korea (Pfreundschuh et al., 2026). These are untested or unresolved conditions, so the verdict is partly addressed, with the regime half unsettled.

On the sub-question of how much information context, channels and time add beyond the pixel, the sources give three partial measurements that do not share a metric: the 1999 correlation ladder in Hong et al. (2004), the channel ablation in Brempong et al. (2025), and the temporal ablation in Pfreundschuh et al. (2026), whose authors describe the gain from three time steps as robust and the gain overall as moderate. No study has recomputed an information measure of the kind used by Ombadi et al. (2021) with multi-channel or neighbourhood inputs, which would put these on one scale. One further point is an inference of this episode and is not tested in any source: Kidd et al. (2021) note that spaceborne radars cannot retrieve precipitation within about 500 to 2,000 m of the surface, and the shallow clouds in So and Shin (2018) have mean storm heights near 2.8 km, so the training and scoring reference used by both neural retrievals may be least reliable for the regime in question.

• How far learned infrared retrievals close the gap is partly answered, by two preprints under stated conditions.
• Where they cannot close it stays open: no retrieval read here reports a score for shallow warm rain alone.
• An information measure recomputed with multi-channel and neighbourhood inputs would put the separate ablations on one scale; none of the sources does this.

References

1. Brempong, E. A., Hassen, M. A., MohamedKhair, M., Dube, V., Hincapie Potes, S., Graham, O., Brik, A., McGovern, A., Huffman, G. J., & Hickey, J. (2025). Oya: Deep learning for accurate global precipitation estimation. arXiv. https://arxiv.org/abs/2511.10562
2. Hayatbini, N., Kong, B., Hsu, K., Nguyen, P., Sorooshian, S., Stephens, G., Fowlkes, C., Nemani, R., & Ganguly, S. (2019). Conditional generative adversarial networks (cGANs) for near real-time precipitation estimation from multispectral GOES-16 satellite imageries—PERSIANN-cGAN. Remote Sensing, 11(19), 2193. https://doi.org/10.3390/rs11192193
3. Hong, Y., Hsu, K.-L., Sorooshian, S., & Gao, X. (2004). Precipitation estimation from remotely sensed imagery using an artificial neural network cloud classification system. Journal of Applied Meteorology, 43(12), 1834–1853. https://doi.org/10.1175/JAM2173.1
4. Kidd, C., Huffman, G., Maggioni, V., Chambon, P., & Oki, R. (2021). The global satellite precipitation constellation: Current status and future requirements. Bulletin of the American Meteorological Society, 102(10), E1844–E1861. https://doi.org/10.1175/BAMS-D-20-0299.1
5. Ombadi, M., Nguyen, P., Sorooshian, S., & Hsu, K. (2021). How much information on precipitation is contained in satellite infrared imagery? Atmospheric Research, 256, 105578. https://doi.org/10.1016/j.atmosres.2021.105578
6. Pfreundschuh, S., Kummerow, C. D., Tan, J., & Huffman, G. J. (2026). GPROF-IR: An improved single-channel infrared precipitation retrieval for merged satellite precipitation products. arXiv. https://arxiv.org/abs/2605.07167
7. So, D., & Shin, D.-B. (2018). Classification of precipitating clouds using satellite infrared observations and its implications for rainfall estimation. Quarterly Journal of the Royal Meteorological Society, 144(S1), 133–144. https://doi.org/10.1002/qj.3288

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library holds one paper on this question, the review by Morrison et al. (2020), and its record of the question rests on that review alone. The episode adds the position that review answers. Khain et al. (2015) argue that bin schemes describe microphysical processes with more consistent accuracy, on two grounds: locally observed size distributions are frequently bimodal and often cannot be fitted by the gamma functions bulk schemes assume, and process rates depend so steeply on particle size that integrating over the spectrum reduces a scheme's sensitivity. They support the argument with case simulations in which a bin scheme agreed better with observations than bulk comparators, among them a Florida rain event, a squall line, Hurricane Irene and month-long regional simulations.

The episode also adds a controlled separation of the two things a bin scheme and a bulk scheme differ in. Hu and Igel (2023) ran a three-moment bulk scheme that calls the process routines of a bin scheme, so that structure is the only difference, in a one-dimensional kinematic warm-rain column over twenty-five combinations of aerosol concentration and updraft strength, with two different underlying bin schemes. Mean liquid water path agreed within four percent and mean surface precipitation was about thirty-five percent lower in the bulk configuration, which they traced to autoconversion; lowering the cloud-rain threshold from 80 to 50 micrometres reduced the largest difference to about ten percent, below the difference between the two bin schemes. Grabowski et al. (2019) add the numerical argument against treating bin schemes as a benchmark: diffusion across bins and in physical space, the cost of adding particle attributes, and the deterministic character of the collection equation.

• The case for bin schemes rests on observed size distributions that the assumed gamma shape cannot fit and on case studies where one bin scheme family agreed better with observations (Khain et al., 2015).
• When a bulk scheme uses a bin scheme's own process formulas in a warm-rain column, the remaining structural difference is smaller than the difference between two bin schemes (Hu and Igel, 2023).

Where they differ from the library

Khain et al. (2015) and Morrison et al. (2020) disagree on whether bin schemes have been shown to perform better, and the disagreement is unresolved. Morrison et al. (2020) state that evidence of consistently better agreement with observations is lacking; Khain et al. (2015) present case studies in which a bin scheme was closer to observations. The two statements differ in what they count as evidence. The cases in Khain et al. (2015) are single events, several of the bulk comparators are one-moment schemes, and most use bin schemes of one family, whereas Morrison et al. (2020) ask for consistency across schemes and cases. Khain et al. (2015) themselves write that the variability among bin schemes is nearly as large as that among bulk schemes and that the hundreds of forecasts needed to establish an advantage have not been performed, which narrows the distance between the two reviews on the systematic question while leaving their recommendations opposed. The same review also reports one intercomparison that went the other way: for Arctic mixed-phase stratiform clouds simulated by seven models with the ice concentration tuned to the observed one, the two bin schemes produced similar results (Khain et al., 2015).

Hu and Igel (2023) qualify the library's account of why schemes diverge. The library records that inter-scheme spread on an identical squall-line setup is forty to sixty kilometres within both the bin and bulk groups (Morrison et al., 2020), which shows that the schemes of one family do not converge on one result but cannot say how much of any bin-bulk difference is structural. Hu and Igel (2023) measure that share directly and find it smaller than the difference between two bin schemes, under conditions the squall-line case does not share: warm rain only, prescribed dynamics, no ice and a three-moment bulk scheme.

• The two reviews disagree on whether bin schemes have been shown to be better, and they differ in what they accept as evidence: single cases in one, consistency across schemes and cases in the other.
• Khain et al. (2015) concede that bin schemes vary nearly as much as bulk schemes and that the many-forecast comparison has not been run.

How they bear on the open questions

The library's open question on whether more detailed schemes give demonstrably better results remains open in substance. The sources read here add a second review that concedes the missing systematic test, and a controlled experiment confined to warm rain; none compares scheme families across many cases against observations, and none applies the controlled design to ice, where Morrison et al. (2020) and Grabowski et al. (2019) locate the largest uncertainty. The library's two subquestions, whether inter-scheme spread exceeds realistic initial-condition spread across many cases and which observable best discriminates scheme families, are not addressed by any of the four.

The open question on observing process rates directly gains independent support. Khain et al. (2015), writing from the opposite side of the bin-bulk argument, list in-cloud supersaturation as unmeasured, report laboratory values of the condensation coefficient spanning 0.01 to 1, and describe the forces binding ice aggregates as unknown, a list that overlaps the one in Morrison et al. (2020). Two author groups with opposed recommendations therefore agree on the measurement gap, and the question stays open. On the library's question about unquantified structural uncertainty, Hu and Igel (2023) supply one way to separate structural from parameterization differences, demonstrated for warm rain in one dimension; whether it extends to mixed-phase clouds with dynamical feedback is untested.

The result bears on the library's machine-learning forecasting domain through the training-target problem: Morrison et al. (2020) note that no benchmark model exists from which to generate training data for a learned microphysics scheme, and the spread among bin schemes reported by Grabowski et al. (2019) and Hu and Igel (2023) shows what an emulator trained on any one of them would inherit.

• Whether detailed schemes give better results stays open: no source read compares the families across many cases with observations as judge, and the controlled design has not been applied to ice.
• Both reviews, from opposed positions, name overlapping sets of unmeasured quantities behind the process rates.

References

1. Grabowski, W. W., Morrison, H., Shima, S.-I., Abade, G. C., Dziekan, P., & Pawlowska, H. (2019). Modeling of cloud microphysics: Can we do better? Bulletin of the American Meteorological Society, 100(4), 655-672. https://doi.org/10.1175/BAMS-D-18-0005.1
2. Hu, A. Z., & Igel, A. L. (2023). A bin and a bulk microphysics scheme can be more alike than two bin schemes. Journal of Advances in Modeling Earth Systems, 15(3), e2022MS003303. https://doi.org/10.1029/2022MS003303
3. Khain, A. P., Beheng, K. D., Heymsfield, A., Korolev, A., Krichak, S. O., Levin, Z., Pinsky, M., Phillips, V., Prabhakaran, T., Teller, A., van den Heever, S. C., & Yano, J.-I. (2015). Representation of microphysical processes in cloud-resolving models: Spectral (bin) microphysics versus bulk parameterization. Reviews of Geophysics, 53(2), 247-322. https://doi.org/10.1002/2014RG000468
4. Morrison, H., van Lier-Walqui, M., Fridlind, A. M., Grabowski, W. W., Harrington, J. Y., Hoose, C., Korolev, A., Kumjian, M. R., Milbrandt, J. A., Pawlowska, H., Posselt, D. J., Prat, O. P., Reimel, K. J., Shima, S.-I., van Diedenhoven, B., & Xue, L. (2020). Confronting the challenge of modeling cloud and precipitation microphysics. Journal of Advances in Modeling Earth Systems, 12, e2019MS001689. https://doi.org/10.1029/2019MS001689

Read into the library after publication:

• Representation of microphysical processes in cloud-resolving models: Spectral (bin) microphysics versus bulk parameterization 10.1002/2014RG000468
• A Bin and a Bulk Microphysics Scheme Can Be More Alike Than Two Bin Schemes 10.1029/2022MS003303
• Modeling of Cloud Microphysics: Can We Do Better? 10.1175/BAMS-D-18-0005.1

Against the library

How this episode sits against the listener's own paper library. Coverage: partial.

What the sources add

The library holds the warn-on-forecast proposal (Stensrud et al., 2009) and no measurement of how tornado warnings have performed over time. Brooks and Correia (2018) supply one from the National Weather Service verification record for 1986–2016. They separate the official lead time into the fraction of tornadoes warned in advance and the mean lead time of those tornadoes. The fraction rose from 0.26 in 1986 to 0.72 in 2003, while the mean lead time of tornadoes warned in advance showed no statistically significant trend from 1986 to 2011 and averaged 18.8 min, with values between 18.7 and 19.3 min across damage ratings (E)F0 to (E)F3. In 2012 the fraction warned in advance, the false alarm ratio and the lead time fell together, to about 0.50 by 2016 and 15.6 min for 2012–2016, which the authors attribute to a shorter default warning duration and a higher decision threshold at approximately constant skill.

The library also holds no evidence on what lead time does for the people warned. Simmons and Sutter (2008) fitted a Poisson model of fatalities to 18,216 tornado segments in the contiguous United States for 1986–2002, with controls for F scale, path length, population density, mobile homes, income, time of day, season and year. Relative to an unwarned tornado, expected fatalities were 19%, 41% and 19% lower at lead times of 1–5, 6–10 and 11–15 min, with only the middle interval significant, and 60%, 37% and 1% higher at 16–20, 21–30 and more than 30 min, the first two significant. With five tornadoes omitted, all six interval estimates were negative and the three beyond 15 min lay within about 5% of zero (converted here from the printed coefficients). Simmons and Sutter (2009) re-estimated the model for 1986–2004 with a local false alarm ratio added, defined over the previous one or two years for the state, forecast office or television market. The lead-time pattern recurred: fatalities were 46% and 29% lower at 6–10 and 11–15 min in the one-year state specification, both significant, and 60–70% higher at 16–20 min in all six specifications, which the authors again attributed to a handful of tornadoes and excluded from their own calculations. A one-standard-deviation increase in the false alarm ratio was associated with 12–29% more expected fatalities and 14–32% more expected injuries. Chatwin et al. (2024), an undergraduate research report that has not been peer reviewed, tabulated the share of warned tornadoes with at least one death by lead time through 2023, without controls, and found no significant relationship.

Hoekstra et al. (2011) add stated preferences. Among 320 visitors to the National Weather Center in 2009, the mean preferred lead time was 34.3 min; 79% named taking shelter as an action under a 15-min lead time and 61% under a 1-h lead time; and 44% said a 1-h lead time would make the situation feel less life threatening, against 47% who reported no change.

• In the 1986–2016 record, tornadoes warned in advance received about 19 min of lead time until 2011, and the rise in official lead time came from warning on a larger share of tornadoes (Brooks & Correia, 2018).
• In 2012 a higher warning threshold and a shorter default duration lowered the false alarm ratio, the share of tornadoes warned in advance and the lead time together.
• The controlled casualty study read here associates lead time with fewer deaths up to about 15 min for 1986–2002, its estimate beyond 15 min changes sign when five tornadoes are removed, and the authors' rerun for 1986–2004 repeats the pattern (Simmons & Sutter, 2008, 2009).
• Survey respondents preferred about 34 min on average and named sheltering less often under a 1-h lead time, as stated intentions from 320 visitors (Hoekstra et al., 2011).

Where they differ from the library

The library's open question on lead time and false alarms takes its baseline from Stensrud et al. (2009): a national average lead time of 13 min and a false alarm rate near 75%. Brooks and Correia (2018) show that an average of this kind is a product. In the official definition plotted in their Figure 3, which stood near 13 min in the late 2000s, about 70% of tornadoes were warned in advance with a mean near 19 min and the remainder counted as zero. The 13-min figure therefore describes how often tornadoes were warned in advance as much as how early, and the lead time available under detection to a tornado that was warned was already about 19 min.

The library records no evidence addressing that question and treats the trade-off as a risk attached to a future paradigm, with warning fatigue as the stake (Camps-Valls et al., 2025). Brooks and Correia (2018) document the trade-off inside the detection-based system. When the threshold for warning rose in 2012, the false alarm ratio fell, and the fraction of tornadoes warned in advance and their lead time fell with it; their signal detection model implies that near a false alarm ratio of 0.75 the probability of detection changes about twice as much as the false alarm ratio. Simmons and Sutter (2009) put the same exchange in casualties: moving along the curve they take from Brooks (2004), from a probability of detection of 0.4 and a false alarm ratio of 0.7 to 1.0 and 0.9, the gain from warning more tornadoes and the loss from more false alarms nearly offset for fatalities in four of six definitions of the false alarm ratio and gave net increases of 20% and 37% in the other two. These are measurements of the exchange under detection for one national system, and they do not bear on whether forecast-based warnings escape it.

Stensrud et al. (2009) present longer lead times as a route to lives saved. Simmons and Sutter (2008) narrow that premise: for 1986–2002 the association between lead time and fewer fatalities holds up to about 15 min, and beyond that the sign of the estimate depends on five tornadoes among 311 that were fatal. Their 2009 re-estimation for 1986–2004 shares authors, model and most of the data with the first, so it is a rerun and not an independent test, and it leaves the estimate beyond 15 min where it was. The grounds for caution are the small number of fatal events, a lead time measured at the first county in the path, and the absence of any measure of whether warnings were received or acted on.

• A national mean lead time of 13 min is the product of how often tornadoes are warned in advance and how early, so it understates the roughly 19 min a warned tornado received.
• The exchange between false alarms and lead time has been measured within detection-based warning, in the 2012 policy change and in a casualty regression where the two effects roughly offset (Simmons & Sutter, 2009), and has not been measured for forecast-based warning in the sources read.
• The premise that longer lead times save lives is supported to about 15 min in one study of 1986–2002 and its rerun to 2004 by the same authors, and is unsupported in either direction beyond that.

How they bear on the open questions

The library asks whether any warning paradigm can extend lead time beyond the limits of detection without raising false alarms. The sources read here document the trade-off under detection in one policy change and leave the forecast-based case untested, so on this reading the question stays open; evaluations of the Warn-on-Forecast System after 2009 were not read for this episode. The sources also expose a prior question that the library does not hold: whether lead time beyond about a quarter of an hour benefits the people warned. On the studies read, that question is open in substance: the one controlled estimate among them, in its 2008 and 2009 versions, has no stable sign beyond 15 min, the proposed mechanism for a declining benefit, that residents discount a warning when no tornado follows quickly, is not tested in them, and none of them measures the response of institutional users to long lead times.

The library also carries from Stensrud et al. (2009) the question of how forecasters and the public can use probabilistic hazard information. Hoekstra et al. (2011) offer a stated-preference data point for the public on lead time, drawn from a convenience sample of visitors, and Brooks and Correia (2018) note that they know of no work estimating the costs of missed detections and false alarms on which a warning threshold could be set. Simmons and Sutter (2009) is one such estimate in casualties, and it finds no gain from moving the threshold in either direction; it measures deaths and injuries only and has no direct evidence on how false alarms change the response to a warning. On the studies read, the value side of that question remains open.

Flora et al. (2026) identify lead times of two to six hours, between watch and warning, as largely unexplored for machine-learning guidance. The episode's sources bear on that window from the user's side: skill at those lead times and the benefit of acting on them are separate questions, and the second has no measurement in the studies read here beyond the institutional needs that Stensrud et al. (2009) list. For the verification literature in the machine-learning forecasting domain, the 2012 discontinuity is a worked case of why probability of detection, false alarm ratio and lead time have to be read jointly.

• Whether forecast-based warnings can add lead time without adding false alarms remains open, because the evidence read here concerns detection only.
• Whether lead time beyond a quarter of an hour benefits those warned is open in substance on the studies read: the estimate is unstable, the proposed mechanism is untested and institutional users are unmeasured.
• Skill at watch-to-warning lead times and the value of acting at those lead times are separate questions, and only the first has work in the library.

References

1. Brooks, H. E. (2004). Tornado-warning performance in the past and future: A perspective from signal detection theory. Bulletin of the American Meteorological Society, 85, 837–843.
2. Brooks, H. E., & Correia, J., Jr. (2018). Long-term performance metrics for National Weather Service tornado warnings. Weather and Forecasting, 33(6), 1501–1511. https://doi.org/10.1175/WAF-D-18-0120.1
3. Camps-Valls, G., Fernández-Torres, M.-A., Cohrs, K.-H., Höhl, A., Castelletti, A., Pacal, A., Robin, C., Martinuzzi, F., Papoutsis, I., Prapas, I., Pérez-Aracil, J., Weigel, K., Gonzalez-Calabuig, M., Reichstein, M., Rabel, M., et al. (2025). Artificial intelligence for modeling and understanding extreme weather and climate events. Nature Communications. https://doi.org/10.1038/s41467-025-56573-8
4. Chatwin, R. M., Brooks, H. E., & Klockow McClain, K. (2024). Tornado warnings, lead times, and death [Undergraduate research report]. National Weather Center Research Experiences for Undergraduates Program. https://reu.caps.ou.edu/reu24/finalpapers/Chatwin_FinalPaper.pdf
5. Flora, M. L., Varga, S., Potvin, C. K., & Lang, N. (2026). Developing machine learning-based watch-to-warning severe weather guidance from the Warn-on-Forecast System. arXiv. https://doi.org/10.48550/arXiv.2603.20250
6. Hoekstra, S., Klockow, K., Riley, R., Brotzge, J., Brooks, H., & Erickson, S. (2011). A preliminary look at the social perspective of warn-on-forecast: Preferred tornado warning lead time and the general public's perceptions of weather risks. Weather, Climate, and Society, 3(2), 128–140. https://doi.org/10.1175/2011WCAS1076.1
7. Simmons, K. M., & Sutter, D. (2008). Tornado warnings, lead times, and tornado casualties: An empirical investigation. Weather and Forecasting, 23(2), 246–258. https://doi.org/10.1175/2007WAF2006027.1
8. Simmons, K. M., & Sutter, D. (2009). False alarms, tornado warnings, and tornado casualties. Weather, Climate, and Society, 1(1), 38–53. https://doi.org/10.1175/2009WCAS1005.1
9. Stensrud, D. J., Xue, M., Wicker, L. J., Kelleher, K. E., Foster, M. P., Schaefer, J. T., Schneider, R. S., Benjamin, S. G., Weygandt, S. S., Ferree, J. T., & Tuell, J. P. (2009). Convective-scale warn-on-forecast system: A vision for 2020. Bulletin of the American Meteorological Society, 90(10), 1487–1499. https://doi.org/10.1175/2009BAMS2795.1

Read into the library after publication:

• Long-term performance metrics for National Weather Service tornado warnings 10.1175/WAF-D-18-0120.1
• Tornado warnings, lead times, and tornado casualties: An empirical investigation 10.1175/2007WAF2006027.1
• False alarms, tornado warnings, and tornado casualties 10.1175/2009WCAS1005.1
• A preliminary look at the social perspective of warn-on-forecast: Preferred tornado warning lead time and the general public's perceptions of weather risks 10.1175/2011WCAS1076.1

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

Gale et al. (2002) supply the only observational test of the cold pool–shear balance against dissipation among the sources considered here. In 47 nocturnal mesoscale convective systems over Iowa and the surrounding states during May–August 1998–1999, with dissipation defined as the disappearance of all radar echoes above 35 dBZ, the ratio of cold-pool strength to line-normal low-level shear averaged 3.8 and was generally highest early in the life cycle and lower as dissipation approached, so the ratio did not correlate well with dissipation in that sample. Surface and elevated system-relative inflow declined during the final six hours, the surface decline being due mostly to slower system motion, and of 19 systems previously affected by a low-level jet and measured within three hours of dissipation, 13 were no longer affected by one. Among the Eta Model fields examined, only 850-hPa equivalent potential temperature advection showed moderate potential, in a test that concerned the model's own convective systems. The authors describe the results as preliminary and document systems that dissipated without slowing and systems that slowed and persisted.

Coniglio et al. (2007) add a proximity-sounding comparison for warm-season quasi-linear systems, drawn from May–August 1998–2004 and supplemented with 79 derecho proximity soundings from an earlier study. Among systems whose leading line moved at 10 m/s or faster, 96 soundings near mature lines and 115 near weakening lines differed by 1.1 m/s in the 0–2-km shear vector magnitude, a difference exceeded in 52 of 1000 random regroupings, and by 11.7 m/s in the 0–10-km shear, almost three times the largest difference among the regroupings. The maximum deep shear, the 3–8-km lapse rate, convective available potential energy and the 3–12-km mean wind together classified over 80% of the same soundings correctly. The study measured no cold pools, defines dissipation as a stage within a three-hour window and assigns it no time, reports no verification statistics, and attributes the decline of upper-level wind and shear in large part to systems moving away from the jet streams that accompanied their initiation.

• In 47 nocturnal systems over Iowa and neighbouring states, the cold pool–shear ratio did not track dissipation, while system-relative inflow and the influence of the low-level jet declined in the final hours (Gale et al., 2002).
• In warm-season proximity soundings for quasi-linear systems, 0–10-km shear separated mature from weakening lines far better than 0–2-km shear (Coniglio et al., 2007).
• Both results are associations within the sample that produced them; neither paper reports skill at a stated lead time on independent cases.

Where they differ from the library

The library's account of dissipation starts from Schumacher and Rasmussen (2020), who state that the processes have received relatively little attention and then name two routes: the mid-morning weakening of low-level jets, and cold pools that become strong enough to overwhelm the shear. Reading the cited sources narrows both statements. The low-level-jet route is referenced to Gale et al. (2002) alone, a single regional sample of 47 systems whose authors call it preliminary; the library held the review's sentence without that source. The cold-pool route is referenced to Weisman and Rotunno (2004), whose simulations produced significant updrafts through six hours at every shear considered, including zero shear, and who attribute the abrupt demise of weakly sheared lines in earlier simulations to constrained model domains. Rotunno et al. (1988) likewise describe a weaker, upshear-tilted state that lasted to the end of an eight-hour integration. The review's sentence on the cold-pool route therefore states more than its source demonstrates. The library already reads both papers this way, as explaining a weaker state in idealized lines without a time of decay, so on the cold-pool route the episode adds only the link from the review's sentence to its source.

The result of Gale et al. (2002) limits where the vorticity-balance concept can be applied. Their cold-pool strength was estimated from surface temperature differences and a pressure-derived depth, the measurements began more than six hours after initiation in most cases, and the majority of the systems were probably elevated for part of their lives, whereas the simulations of Rotunno et al. (1988) and Weisman and Rotunno (2004) treat surface-based lines from initiation. A similar limit is drawn by Coniglio et al. (2012), who suggest from the 8 May 2009 derecho that the theory be applied with much caution to complex nocturnal systems and that the shear above 3 km be examined. Coniglio et al. (2007) and Coniglio et al. (2012) share authors, so the independent pairing is the Iowa sample of Gale et al. (2002) against the soundings of Coniglio et al. (2007); the two reach compatible conclusions about low-level shear by different methods, and only the former tests the balance ratio.

• The two dissipation routes named by Schumacher and Rasmussen (2020) trace to one sample of 47 systems and to simulations in which every line persisted to the end of the run.
• Rotunno et al. (1988) and Weisman and Rotunno (2004) describe a weaker, upshear-tilted state under cold-pool dominance and attach no time to its end, a reading the library already holds.
• The observational result of Gale et al. (2002) limits where the balance concept applies, since their systems were probably mostly elevated and were sampled late in life.

How they bear on the open questions

The library's open question is whether dissipation can be predicted from diagnostics of the inflow supply ahead of the decay of the radar echo. Gale et al. (2002) are the direct evidence: declines in system-relative inflow and the loss of the low-level jet appear within three to six hours of dissipation in one sample of 47 systems, with documented exceptions in both directions and with part of the slowing arising from tracking the stratiform centroid once the convective line had faded. Coniglio et al. (2007) offer a probability of the mature stage conditioned on the environment, with no time of dissipation. The question remains open in substance: none of the five sources reports skill at a stated lead time on independent cases, none establishes whether slowing precedes decay or accompanies it, and none confirms the cold-pool route in observations.

Two neighbouring questions are touched. On the applicability of cold pool–shear balance to observed systems, Gale et al. (2002) add a multi-case sample in which lines with ratios of three or more kept their strength for many hours, which supports the caution urged by Coniglio et al. (2012); the question stays open because cold-pool strength was inferred from surface data, and its direct estimation is itself unresolved. On the transition between elevated and surface-based inflow, Gale et al. (2002) found that the 27 systems north of a boundary and the 20 in the warm sector differed in how inflow declined before dissipation, so a predictor built on inflow depends on diagnosing the inflow source, a diagnosis the library's sources leave without a real-time method (Coniglio et al., 2012; Schumacher & Rasmussen, 2020). For the neighbouring domain of machine-learning weather forecasting, the radar definition of dissipation used by Gale et al. (2002) is a verifiable target for evaluating the timing of decay; this is an inference of the present analysis, untested in the five sources.

• Whether dissipation is predictable ahead of echo decay remains open: the inflow and jet signals appear in the final three to six hours of one sample and are unverified on independent cases in the sources read.
• The applicability of cold pool–shear balance to observed systems remains open because the one multi-case test inferred cold-pool strength from surface data.
• A dissipation predictor built on inflow depends on knowing whether that inflow is elevated or surface-based, which the library's sources cannot yet diagnose in real time.

References

1. Coniglio, M. C., Brooks, H. E., Weiss, S. J., & Corfidi, S. F. (2007). Forecasting the maintenance of quasi-linear mesoscale convective systems. Weather and Forecasting, 22(3), 556–570. https://doi.org/10.1175/WAF1006.1
2. Coniglio, M. C., Corfidi, S. F., & Kain, J. S. (2012). Views on applying RKW theory: An illustration using the 8 May 2009 derecho-producing convective system. Monthly Weather Review, 140, 1023–1043. https://doi.org/10.1175/MWR-D-11-00026.1
3. Gale, J. J., Gallus, W. A., Jr., & Jungbluth, K. A. (2002). Toward improved prediction of mesoscale convective system dissipation. Weather and Forecasting, 17(4), 856–872. https://doi.org/10.1175/1520-0434(2002)017<0856:TIPOMC>2.0.CO;2
4. Rotunno, R., Klemp, J. B., & Weisman, M. L. (1988). A theory for strong, long-lived squall lines. Journal of the Atmospheric Sciences, 45(3), 463–485. https://doi.org/10.1175/1520-0469(1988)045<0463:ATFSLL>2.0.CO;2
5. Schumacher, R. S., & Rasmussen, K. L. (2020). The formation, character and changing nature of mesoscale convective systems. Nature Reviews Earth & Environment, 1, 300–314. https://doi.org/10.1038/s43017-020-0057-7
6. Weisman, M. L., & Rotunno, R. (2004). “A theory for strong long-lived squall lines” revisited. Journal of the Atmospheric Sciences, 61(4), 361–382. https://doi.org/10.1175/1520-0469(2004)061<0361:ATFSLS>2.0.CO;2

Read into the library after publication:

• Toward improved prediction of mesoscale convective system dissipation 10.1175/1520-0434(2002)017<0856:TIPOMC>2.0.CO;2
• Forecasting the maintenance of quasi-linear mesoscale convective systems 10.1175/WAF1006.1

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library holds this question through one source, the review of polarimetric precipitation estimation by Ryzhkov et al. (2022). From it the library records the two polarimetric snow relations, their sensitivity to particle shape, orientation and riming, an instability threshold in specific differential phase and differential reflectivity, one end-to-end radar test on a nor'easter observed at a single radar, and the authors' verdict that the snow algorithms are still in their infancy. The review was not reopened for this episode and is cited here from the library's record. The episode adds the two primary papers on which the review's snow section rests. Bukovčić et al. (2018) derive the combined relation from about 7000 one-minute disdrometer spectra in sixteen Oklahoma snow events, with both radar variables computed from the spectra under a fixed axis ratio of 0.65 and zero canting; the correlation with disdrometer snowfall rate is 0.995, against 0.862 for a reflectivity relation fitted to the same data. Bukovčić et al. (2020) generalise the relation over aspect ratio and canting width, introduce a second relation based on the ratio of differential phase to a differential-reflectivity term, and apply both to WSR-88D data in three storms.

The 2020 paper supplies quantities the library's record lacks. Across canting widths of 0 to 40 degrees and aspect ratios of 0.5 to 0.7 the multiplier of the combined relation rises by a factor of up to about 3.5. Specific differential phase in aggregated snow at S band is typically below 0.05 degrees per kilometre, and the stated accuracy of 0.01 degrees per kilometre is reached by averaging over a full azimuth circle. At the Baltimore gauge the combined relation finished very close to the gauge total of about 46.5 mm, against about 42 mm for the second relation and 29.5 mm for reflectivity; in Colorado the gauge, combined-relation and reflectivity totals were 22.9, 16 and 8 mm; in Oklahoma the two estimates bracketed a reconstructed reference of 11.5 mm, and at the Dulles gauge beside the Virginia radar they lay within a few millimetres of each other. A change of 0.025 dB in the assumed differential-reflectivity bias altered the second relation's storm total by about 20 percent (Bukovčić et al., 2020).

Three further sources extend the record beyond the library's holding. Capozzi et al. (2020) test differential-phase estimators at X band in the southern Apennines against a laser disdrometer: in the three cases, out of fifteen, that survived a wind screening, correlation rose from between 0.06 and 0.19 for reflectivity-only estimators to between 0.53 and 0.60, while the normalised error of the phase-based estimators stayed near 1. Wu et al. (2026) fit a reflectivity relation to thirty sea-effect snowfalls at one S-band radar on the Shandong Peninsula and find, with the exponent fixed at 1.72, a prefactor between 109.0 for a rimed event and 567.8 for an event of unrimed aggregates. Zhang et al. (2025) review the American, Chinese and French operational systems and report that polarimetric estimators are applied only to liquid precipitation in all three, with a single reflectivity relation given for snow in the MRMS system.

• The combined phase and reflectivity relation was derived and first scored entirely on disdrometer spectra with an assumed particle shape and no canting (Bukovčić et al., 2018).
• Its S-band radar verification in the 2020 paper is three storms, with a clear advantage over reflectivity in two and parity in the third (Bukovčić et al., 2020).
• At one S-band radar the prefactor of the reflectivity relation ranged fivefold over thirty sea-effect snowfalls, lowest in a rimed event and highest in one of unrimed aggregates (Wu et al., 2026).
• A 2025 review finds polarimetric estimators applied only to liquid precipitation in the American, Chinese and French operational systems (Zhang et al., 2025).

Where they differ from the library

The library records the review's statement that both polarimetric relations become unstable when specific differential phase falls below 0.01 to 0.03 degrees per kilometre and differential reflectivity below 0.2 to 0.3 dB (Ryzhkov et al., 2022). The 2020 paper gives 0.01 degrees per kilometre and 0.3 to 0.4 dB for the second relation, and a typical value below 0.05 degrees per kilometre in aggregated snow (Bukovčić et al., 2020). The typical signal therefore lies in the same range as the stated floor, and the practical floor depends on how much spatial averaging is accepted, which neither source fixes. Wu et al. (2026) report differential phase above 0.125 degrees per kilometre in the lowest kilometre of an aggregate-dominated sea-effect storm, more than twice the typical value of 0.05 and inside the maxima of 0.3 to 0.4 degrees per kilometre that the 2020 paper also reports; Wu et al. add that secondary ice production or non-uniform beam filling may contribute to the low-level increase. At least one aggregate storm therefore exceeds the typical value, and how often that happens is unmeasured in the sources read here.

On riming the sources disagree in sign. Bukovčić et al. (2018) derive a multiplier proportional to the riming factor raised to the power of minus 0.57, so that rimed snow lowers it, and state that no typical range of the factor is known. Bukovčić et al. (2020) report a preliminary analysis in which light to moderate riming may raise the multipliers by 50 percent, with riming not examined explicitly. The library's record states only that the second relation is somewhat sensitive to the degree of riming (Ryzhkov et al., 2022). Wu et al. (2026) associate the prefactor of the reflectivity relation with riming in observed storms, and in the same events with number concentration, so their result does not separate the two; in the 2018 derivation the reflectivity multiplier depends almost entirely on the size-distribution intercept and changes by 11 percent across the full range of the riming factor.

The library describes the verification as a single storm at one site. Counting the three storms of the 2020 paper, the S-band record known to the library and this episode is four storms, and Zhang et al. (2025) also count four evaluation cases, citing the 2020 paper and a conference presentation without listing them, with good agreement in three and severe underestimation in one. The X-band test adds three cases with a different reference instrument and reverses one expectation from the simulations: on real data the phase-only estimator outperformed the combined one, which the authors attribute to attenuation of reflectivity by a melting layer along the path (Capozzi et al., 2020).

• The typical differential phase of aggregated snow at S band, below 0.05 degrees per kilometre, lies in the same range as the instability floor the library records, and one aggregate-dominated sea-effect storm showed more than twice that value near the surface (Bukovčić et al., 2020; Wu et al., 2026).
• The effect of riming on the polarimetric multiplier is given with opposite signs in the 2018 and 2020 papers.
• The S-band verification record is four storms, of which the library had recorded one.

How they bear on the open questions

The library's open question on the maturity of polarimetric snow estimation remains open. The sources read here settle that the combined relation reduces the error arising from size-distribution variability when the radar variables are computed from measured spectra (Bukovčić et al., 2018), and that in two of three S-band storms it brought the storm total closer to a gauge than a fixed reflectivity relation did (Bukovčić et al., 2020). In three X-band cases the phase-based estimators raised the correlation with a disdrometer while the normalised error stayed near 1 (Capozzi et al., 2020). They leave three matters unsettled, each a question of substance: the sign and size of the riming effect, a means of obtaining shape and orientation factors other than by assumption, and the frequency with which the signal in near-surface snow falls below a usable level. The first subquestion the library poses, the multi-season frequency of sub-floor conditions at S band, is addressed by none of the sources read here. Nor is the second, retrieval of shape and orientation from the radar itself: the 2020 paper states that the multipliers could be obtained experimentally from radar data and leaves this to further study.

The library's open question on the degradation of radar precipitation estimates at long range is affected directly. Both primary papers propose estimating snowfall in the dendritic growth layer aloft, where differential phase is larger, and projecting it to the surface under an assumption of conserved mass flux (Bukovčić et al., 2018, 2020). That moves the measurement to the heights where the review held by the library locates the loss of accuracy (Ryzhkov et al., 2022), and Zhang et al. (2025) state that knowledge of the vertical profiles of polarimetric variables in ice is limited and that correction methods have yet to be developed. The 2020 paper notes in addition that a flake falling from about 4 km can drift 80 to 90 km in strong wind. For the neighbouring domain of machine-learning weather forecasting the consequence, drawn here as an inference, concerns labels: where MRMS precipitation fields serve as training targets, their snow values carry the error of a reflectivity relation, and Wu et al. (2026) show the fitted relation to vary by a factor of about 2.6 in rate between storms at a single site.

• The maturity question stays open because the riming effect, the source of shape and orientation factors, and the frequency of sub-floor signal are unresolved in substance.
• Estimating snow aloft in the dendritic growth layer transfers the problem to the long-range, above-surface regime the library already holds as an open question.
• MRMS snow fields used as machine-learning targets inherit the storm-to-storm error of a reflectivity relation.

References

1. Bukovčić, P., Ryzhkov, A., & Zrnić, D. (2020). Polarimetric relations for snow estimation—Radar verification. Journal of Applied Meteorology and Climatology, 59(5), 991–1009. https://doi.org/10.1175/JAMC-D-19-0140.1
2. Bukovčić, P., Ryzhkov, A., Zrnić, D., & Zhang, G. (2018). Polarimetric radar relations for quantification of snow based on disdrometer data. Journal of Applied Meteorology and Climatology, 57(1), 103–120. https://doi.org/10.1175/JAMC-D-17-0090.1
3. Capozzi, V., Montopoli, M., Bracci, A., Adirosi, E., Baldini, L., Vulpiani, G., & Budillon, G. (2020). Retrieval of snow precipitation rate from polarimetric X-band radar measurements in Southern Italy Apennine mountains. Atmospheric Research, 236, 104796. https://doi.org/10.1016/j.atmosres.2019.104796
4. Ryzhkov, A., Zhang, P., Bukovčić, P., Zhang, J., & Cocks, S. (2022). Polarimetric radar quantitative precipitation estimation. Remote Sensing, 14(7), 1695. https://doi.org/10.3390/rs14071695
5. Wu, Y., Li, H., Gao, W., & Ruan, Z. (2026). Radar observations of cold air outbreak snowfall in East China: Variations of Z-S parameterizations and microphysical impacts. Journal of Geophysical Research: Atmospheres, 131(16), e2026JD046447. https://doi.org/10.1029/2026JD046447
6. Zhang, Z., Zhao, Z., Qi, Y., & Xiong, M. (2025). Dual-polarization radar quantitative precipitation estimation (QPE): Principles, operations, and challenges. Remote Sensing, 17(21), 3619. https://doi.org/10.3390/rs17213619

Read into the library after publication:

• Polarimetric Radar Relations for Quantification of Snow Based on Disdrometer Data 10.1175/JAMC-D-17-0090.1
• Polarimetric Relations for Snow Estimation—Radar Verification 10.1175/JAMC-D-19-0140.1
• Dual-Polarization Radar Quantitative Precipitation Estimation (QPE): Principles, Operations, and Challenges 10.3390/rs17213619
• Retrieval of snow precipitation rate from polarimetric X-band radar measurements in Southern Italy Apennine mountains 10.1016/j.atmosres.2019.104796
• Radar Observations of Cold Air Outbreak Snowfall in East China: Variations of Z-S Parameterizations and Microphysical Impacts 10.1029/2026JD046447

Against the library

How this episode sits against the listener's own paper library. Coverage: partial.

What the sources add

The library holds the critique that opened the question, the finding that a one-layer linear model outperformed five Transformer forecasters on nine benchmarks (Zeng et al., 2023), together with a survey statement that no architecture family had proved clearly superior (Benidis et al., 2022). It holds none of the eight later papers read for this episode. Two papers restored a Transformer lead by redefining the token. Nie et al. (2023) used overlapping 16-step patches and processed each variable independently through a shared encoder; with a 336-step look-back at horizon 96 they report errors 14, 10 and 7 percent below DLinear on Weather, Traffic and Electricity and 0 to 5 percent below it on the four ETT sets. Liu et al. (2024) embedded each variable's whole look-back window as one token, applied attention across variables and a feed-forward network along time, and accepted that attention had been improperly applied on the temporal dimension. Neither design alters the attention mechanism, and the main ablation of Nie et al. (2023) varies only patching and channel independence.

Five author groups have removed or neutralised attention with tokens and training held fixed. Liu et al. (2024) report in their own ablation that removing variate attention raises MSE averaged over horizons by 8 percent on Electricity, 8 on Traffic, 3 on Weather and 12 on Solar-Energy, and that the vanilla arrangement with attention over time steps doubles the Traffic error. Lee et al. (2024) replace the PatchTST encoder with a patch-independent MLP under the same patching and channel independence and obtain an average supervised MSE of 0.304 against the 0.307 published for PatchTST over seven datasets. Guo et al. (2024) retrain PatchTST and iTransformer without the attention module over five runs: for PatchTST the change is within 1 percent on the ETT sets and between 0.2 and 5.4 percent on Weather, Traffic and Electricity; for iTransformer it is 8 to 14 percent on Traffic and Electricity and an improvement of 1 to 5 percent on the ETT sets. Liang et al. (2025) set the PatchTST attention matrix to zero, to the identity or to a uniform average; with the zero matrix the error is equal or lower on the ETT sets and Weather. Ilbert et al. (2024) report 11.9 percent lower MSE for a one-layer channel-attention Transformer trained with sharpness-aware minimisation than for the same model with attention fixed to the identity.

Three further results bear on how those numbers should be read. Shao et al. (2025) rerun more than 45 methods under one pipeline and show that the ranking of linear and Transformer forecasters reverses with dataset character: on the PEMS04 and PEMS08 traffic-flow sets the best linear model's MAE is 35 to 39 percent above the best of Informer, Autoformer and FEDformer, while on ETTh2 and ETTm2 it is 4 to 8 percent below; removing the spatial component from two models costs 10 to 15 percent on two road-speed datasets and improves the forecasts on Exchange-Rate and ETTm1. Tan et al. (2024) remove the pretrained language model from three LLM-based forecasters, or replace it with one attention layer or one Transformer block, and find the ablations ahead of Time-LLM in 26 of 26 dataset and metric cases, of CALF in 22 and of OneFitsAll in 19. Liang et al. (2025) report that zeroing attention in the pretrained model Timer raises MSE by 16 percent on Electricity and 13 percent on Traffic, whereas a uniform average raises it by at most 4 percent; that table carries no error bars and the retraining protocol is not stated. The same appendix reports changes in both directions for Lag-Llama, scored by CRPS on ten datasets, and a 2 percent change for Moirai on one dataset.

• With tokens and training held fixed, removing attention over time from PatchTST raises error by at most 5 percent on the standard benchmarks and lowers it on some (Lee et al., 2024; Guo et al., 2024; Liang et al., 2025).
• Removing attention across variables from iTransformer costs 8 to 14 percent on Traffic and Electricity and improves the seven-series ETT sets (Liu et al., 2024; Guo et al., 2024).
• The two Transformer comebacks changed the token, to patches or to whole variables, and left the attention mechanism as it was (Nie et al., 2023; Liu et al., 2024).
• Pretrained language models can be removed from three LLM-based forecasters without loss of accuracy in most dataset and metric cases (Tan et al., 2024).

Where they differ from the library

The library records Zeng et al. (2023) as evidence that permutation-invariant self-attention loses temporal order and carries that reading across to attention-based nowcasters as a shared failure mode. The later work, and the paper's own tables, narrow the claim in three ways. The shuffle test shows order-blindness for all three Transformers only on Exchange-Rate; on ETTh1, FEDformer and Autoformer degrade by 73 and 57 percent, against 81 percent for the linear model, and only Informer is unaffected (Zeng et al., 2023). The linear model's lead does not survive a change of data: under one pipeline Informer itself has an MAE about a quarter below the linear models on PEMS04, and Shao et al. (2025) attribute the 2023 result to evaluation on datasets with distribution drift or unclear patterns. And the objection concerns attention over time: attention across variables contributes 8 to 14 percent on the two benchmarks with hundreds of series in two independent ablations (Liu et al., 2024; Guo et al., 2024).

The survey statement that no approach is clearly superior (Benidis et al., 2022) is consistent with these results and less specific than they now allow. Under one pipeline the choice between a linear and a Transformer forecaster follows from whether the series show clear and stable patterns and whether samples are spatially indistinguishable (Shao et al., 2025), which is a conditional recommendation. A disagreement inside the new material concerns a single model: Ilbert et al. (2024) find a 12 percent MSE penalty for identity attention in SAMformer, whereas Liang et al. (2025) find RMSE changes of 2 percent or less when its attention is zeroed. A second concerns a single dataset: the spatial-indistinguishability measure of Shao et al. (2025) places Electricity with the ETT sets, where they expect cross-series modelling to be unnecessary, yet removing variate attention from iTransformer costs 8 to 14 percent there (Liu et al., 2024; Guo et al., 2024). The Traffic benchmark is not among the 14 datasets on which Shao et al. (2025) report, so their account reaches the loss on Traffic only by analogy with METR-LA, PEMS-BAY, PEMS04 and PEMS08.

• The 2023 shuffle test shows order-blindness for all three Transformers on Exchange-Rate only; on ETTh1 two of them degrade by 73 and 57 percent, against 81 for the linear model (Zeng et al., 2023).
• Under one pipeline the ranking of linear and Transformer forecasters reverses with dataset character, so the linear model's lead belongs to benchmarks with distribution drift or unclear patterns (Shao et al., 2025).
• The value of learnable attention in SAMformer is disputed between its authors and one replication (Ilbert et al., 2024; Liang et al., 2025).

How they bear on the open questions

The library's open question asks whether forecasting gains credited to self-attention come from the mechanism or from training strategy and data scale. On the mechanism, the question is answered in substance for supervised models on the standard long-horizon benchmarks: with tokens and training fixed, attention over time is worth 0 to 5 percent, attention across series is worth 8 to 14 percent where series number in the hundreds, and the gains over the earlier Transformers arrived with patching, channel independence and variate tokens (Nie et al., 2023; Liu et al., 2024; Lee et al., 2024; Guo et al., 2024; Liang et al., 2025). Three parts remain open. The cause is contested among four accounts, order loss (Zeng et al., 2023), convergence to sharp minima (Ilbert et al., 2024), an unstructured embedding space (Liang et al., 2025) and dataset character (Shao et al., 2025), which prescribe different remedies; among the papers read here only two of them are tested by another group, order loss when Shao et al. (2025) reran the linear-versus-Transformer comparison on other data, and the sharp-minima account when Liang et al. (2025) zeroed the attention of SAMformer and found RMSE changes of 2 percent or less. The scale half rests on three appendix tables without error bars in one preprint, a 13 to 16 percent loss from zeroed attention for Timer, mixed changes for Lag-Llama and one dataset for Moirai (Liang et al., 2025), while pretraining on language did not help the three forecasters tested by Tan et al. (2024). And none of the papers read here tests the result on gridded fields.

That last part is the bridge to the library's nowcasting holdings. Earthformer reports a mean CSI 5.6 percent above ConvLSTM on SEVIR and 0.179 against 0.131 for the best baseline at the highest threshold, with ablations that compare attention patterns, global vectors, hierarchy and decoder type but never replace the attention, and with three convolutional downsampling stages ahead of the first attention block (Gao et al., 2022). Its comparison is therefore between whole models, as the time-series comparisons were before the removal experiments. The library's card for StormDiT records a two-billion-parameter diffusion Transformer initialised from a video foundation model, a statement that global attention lets the model learn non-local rules of convection, and no ablation separating architecture from pretraining (Sun et al., 2026); its PDF exceeded the fetch limit, its full text was not read for this episode, and the narration does not rely on it. The library's proposed validation for nowcasting, a matched linear or lightweight convolutional baseline with an input-shuffle test, can be sharpened by the designs read here: replace the attention by zero, the identity or a uniform average inside the same model, separately for the temporal and spatial axes, over several seeds (Guo et al., 2024; Liang et al., 2025). On the library's separate question of how to choose an architecture, Shao et al. (2025) supply a conditional rule for multivariate series, and the question remains open for gridded fields.

• The mechanism half of the attribution question is answered in substance for supervised models on the standard benchmarks; cause, scale and transfer to gridded fields remain open.
• Four explanations for why attention adds little prescribe four different remedies, and among the papers read here only the order-loss and sharp-minima accounts are tested by another group (Shao et al., 2025; Liang et al., 2025).
• Earthformer's published comparisons are between whole models, and no attention-replacement ablation on its temporal or spatial axes is reported (Gao et al., 2022).

References

1. Benidis, K., Rangapuram, S. S., Flunkert, V., Wang, Y., Maddix, D., Turkmen, C., Gasthaus, J., Bohlke-Schneider, M., Salinas, D., Stella, L., Aubet, F.-X., Callot, L., & Januschowski, T. (2022). Deep learning for time series forecasting: Tutorial and literature survey. ACM Computing Surveys. https://doi.org/10.1145/3533382
2. Gao, Z., Shi, X., Wang, H., Zhu, Y., Wang, Y., Li, M., & Yeung, D.-Y. (2022). Earthformer: Exploring space-time Transformers for Earth system forecasting. 36th Conference on Neural Information Processing Systems (NeurIPS 2022). https://doi.org/10.48550/arXiv.2207.05833
3. Guo, S., Deng, J., Wei, Y., Dou, H., Shen, F., & Zhao, J. (2024). RAM: Replace attention with MLP for efficient multivariate time series forecasting (arXiv:2410.24023v2). arXiv. https://doi.org/10.48550/arXiv.2410.24023
4. Ilbert, R., Odonnat, A., Feofanov, V., Virmaux, A., Paolo, G., Palpanas, T., & Redko, I. (2024). SAMformer: Unlocking the potential of Transformers in time series forecasting with sharpness-aware minimization and channel-wise attention. Proceedings of the 41st International Conference on Machine Learning, PMLR 235, 20924–20954. https://proceedings.mlr.press/v235/ilbert24a.html
5. Lee, S., Park, T., & Lee, K. (2024). Learning to embed time series patches independently. International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2312.16427
6. Liang, Z., Zhu, J., & Sun, W. (2025). Why attention fails: The degeneration of Transformers into MLPs in time series forecasting (arXiv:2509.20942). arXiv. https://doi.org/10.48550/arXiv.2509.20942
7. Liu, Y., Hu, T., Zhang, H., Wu, H., Wang, S., Ma, L., & Long, M. (2024). iTransformer: Inverted Transformers are effective for time series forecasting. International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2310.06625
8. Nie, Y., Nguyen, N. H., Sinthong, P., & Kalagnanam, J. (2023). A time series is worth 64 words: Long-term forecasting with Transformers. International Conference on Learning Representations (ICLR 2023). https://arxiv.org/abs/2211.14730
9. Shao, Z., Wang, F., Xu, Y., Wei, W., Yu, C., Zhang, Z., Yao, D., Sun, T., Jin, G., Cao, X., Cong, G., Jensen, C. S., & Cheng, X. (2025). Exploring progress in multivariate time series forecasting: Comprehensive benchmarking and heterogeneity analysis. IEEE Transactions on Knowledge and Data Engineering, 37(1), 291–305. https://doi.org/10.1109/TKDE.2024.3484454
10. Sun, H., Yang, Y., Han, W., Huang, W., Chen, H., Gao, Z., Li, Z., Huo, Z., & Niu, Z. (2026). StormDiT: A generative AI model bridges the 2-6 hour 'gray zone' in precipitation nowcasting (arXiv:2601.20342). arXiv. https://doi.org/10.48550/arXiv.2601.20342
11. Tan, M., Merrill, M. A., Gupta, V., Althoff, T., & Hartvigsen, T. (2024). Are language models actually useful for time series forecasting? 38th Conference on Neural Information Processing Systems (NeurIPS 2024). https://arxiv.org/abs/2406.16964
12. Zeng, A., Chen, M., Zhang, L., & Xu, Q. (2023). Are Transformers effective for time series forecasting? Proceedings of the AAAI Conference on Artificial Intelligence, 37(9), 11121–11128. https://doi.org/10.1609/aaai.v37i9.26317

Read into the library after publication:

• A Time Series is Worth 64 Words: Long-term Forecasting with Transformers arXiv:2211.14730
• iTransformer: Inverted Transformers Are Effective for Time Series Forecasting arXiv:2310.06625
• Learning to Embed Time Series Patches Independently arXiv:2312.16427
• RAM: Replace Attention with MLP for Efficient Multivariate Time Series Forecasting arXiv:2410.24023
• Why Attention Fails: The Degeneration of Transformers into MLPs in Time Series Forecasting arXiv:2509.20942
• SAMformer: Unlocking the Potential of Transformers in Time Series Forecasting with Sharpness-Aware Minimization and Channel-Wise Attention arXiv:2402.10198
• Exploring Progress in Multivariate Time Series Forecasting: Comprehensive Benchmarking and Heterogeneity Analysis 10.1109/TKDE.2024.3484454
• Are Language Models Actually Useful for Time Series Forecasting? arXiv:2406.16964

Against the library

How this episode sits against the listener's own paper library. Coverage: partial.

What the sources add

The library records the shear dependence of precipitation efficiency as an unresolved conflict between a modelled increase and an observed decrease (Weisman & Klemp, 1982), and records efficiency itself as an ingredient whose microphysical and entrainment controls are unobservable operationally (Doswell et al., 1996). The observational record behind that conflict is what the library lacks, and the episode's sources supply it. Fankhauser (1988) measured seven Montana thunderstorms during CCOPE in 1981 with two aircraft flying within 100 to 200 m of cloud base for more than an hour each and a 10 cm radar for rainout, and found efficiencies from just under 20 to near 50 percent under the paper's middle radar rain relation, with no inverse dependence on shear and a slightly positive tendency if any; 38 percent of one storm's vapour inflow arrived in updrafts weaker than 2 m/s, below the threshold of the earlier aircraft studies, and the choice of radar rain relation alone shifted an efficiency by 10 to 25 points. Market et al. (2002, 2003) computed efficiency over six-hour windows for Midwestern mesoscale convective systems as radar rainfall over analysed precipitable water and found a negative correlation with shear over the warm cloud depth, about -0.38 across twenty periods of six hours in 2000 at the 90 percent level, together with a positive correlation with low-level relative humidity and a negative one with convective inhibition; the journal version reports the negative correlations with shear and with convective inhibition for 24 systems. Kain (2004) reports that the original Kain-Fritsch scheme set precipitation efficiency from shear and cloud-base height, with the inverse shear relation taken from Fritsch and Chappell (1980), that the inverse relation did not seem valid over a wide range of conditions, and that a new downdraft formulation eases some of those problems.

On the definitional side, Sui et al. (2007) separate the large-scale efficiency of observational work, rain over vapour convergence plus surface evaporation, from the cloud-microphysics efficiency of models, rain over condensation plus deposition, and show in a two-dimensional cloud-resolving simulation of 21 days of TOGA COARE that both exceed 100 percent when storage terms are dropped, that the large-scale form can be negative, and that with storage included the two correlate at 0.85 over hourly 96 km means with a root-mean-square difference of 18.5 points. Jo and Lasher-Trapp (2022) then compute two efficiencies in eight CM1 supercells at 100 m spacing with 0 to 6 km shear from 15 to 38 m/s: a core condensation efficiency, condensate produced inside the updraft core per unit of vapour the core ingests, which rises from 63 to 81 percent as the core widens from 4.5 to 10 km and entrains less relative to its size, and a surface efficiency, rain over vapour flowing into the storm, near 48 percent in the weakest-shear run and between 20 and 30 percent in the other seven, held down by detrainment through the storm top and evaporation downwind, which the text describes as largely steady with no clear relationship to shear and the abstract as a decrease. Mulholland et al. (2024) add that doubling the 1 to 6 km shear in 250 m supercell simulations gives two to three times the area-averaged rain rate and accumulation, the paper's 40 to 70 percent differences being taken against the stronger-shear runs, grows the core updraft area by a similar factor, raises every term of the vapour budget outside the core, evaporation and sublimation included, and moves the near-surface reflectivity centroid from about 14 to 22 km from the updraft, with the same pattern in dry and moist free tropospheres and under two microphysics schemes; no efficiency is computed. Kukulies et al. (2026) give an observational efficiency by storm type, rain over condensation estimated from satellite ice water path growth and from a 4 km CONUS simulation over 2000 to 2022: mesoscale convective systems centred on 0.6 to 0.7 with smaller convective systems a little lower in the satellite estimate, tropical cyclones above 0.8, no clear rise with organisation, methods validated only on mesoscale convective systems, and no stratification by shear.

• Seven CCOPE thunderstorms measured with cloud-base aircraft show no inverse dependence of precipitation efficiency on shear, a slightly positive one if any, and 38 percent of one storm's inflow below the 2 m/s threshold of earlier studies (Fankhauser, 1988).
• Twenty periods, each of six hours, from Midwestern mesoscale convective systems in 2000 give a negative correlation of about -0.38 between column-water efficiency and shear, and the 24-system journal study reports the same sign, while Kain (2004) argued the Kain-Fritsch scheme's shear-based efficiency away in the same period (Market et al., 2002, 2003).
• Observational and model definitions of efficiency correlate at 0.85 with an 18.5-point root-mean-square difference in a two-dimensional tropical simulation (Sui et al., 2007).
• In eight simulated supercells the core condensation efficiency rises from 63 to 81 percent with shear while surface rain over inflowing vapour does not rise, near 48 percent in the weakest-shear run and 20 to 30 percent in the other seven (Jo & Lasher-Trapp, 2022).
• Satellite efficiency by storm type now exists for 2000 to 2022 over the United States, under the condensate definition and without stratification by shear (Kukulies et al., 2026).

Where they differ from the library

The library's statement of the conflict needs two qualifications from the sources. The modelled increase holds only in the higher-buoyancy, split-storm part of Weisman and Klemp's (1982) parameter space; at their lowest buoyancy the same model shows efficiency falling from 45 percent to zero with shear, which they attribute to entrainment, so the model already carried both signs. And the observed decrease that the card sets against it rests on Marwitz (1972) and the storms cited in 1982; the observational record since then is split rather than uniformly negative, with Fankhauser (1988) finding a slightly positive tendency in seven storms under an inflow-flux definition and Market et al. (2003) recovering a negative correlation in mesoscale systems under a column-water definition. The disagreement therefore runs through the definitions and the storm types as much as between models and observations.

The library's estimation gap, drawn from Doswell et al. (1996), locates the problem in inputs a forecaster cannot observe. Sui et al. (2007) and Jo and Lasher-Trapp (2022) relocate part of it to the definition: even with every process rate known, as in a model, the sign of the shear response depends on where along the chain the ratio is taken. Condensate made per unit of ingested vapour rises with shear while rain per unit of inflowing vapour does not, because anvil detrainment lowers the surface ratio and never enters the core ratio. Two attributions also narrow under the texts. Doswell et al. (1996) treat shear with care: they write that it is among the factors that can alter efficiency and refer the reader to Fankhauser (1988), whose result is the null one. The weak-to-moderate shear item in flash-flood forecasting descends from the 151-event survey of Maddox et al. (1979), as Schumacher (2017) traces it, which characterises the environments of flash floods rather than measuring efficiency.

• Weisman and Klemp's (1982) model carried both signs, a decrease with shear at low buoyancy and an increase once storms split, so the conflict was never simply model against observation.
• The observational record is split by definition and storm type: an inflow-flux definition on discrete thunderstorms gives no inverse relation (Fankhauser, 1988) and a column-water definition on mesoscale systems gives one (Market et al., 2003).
• Doswell et al. (1996) refer shear to Fankhauser (1988) rather than asserting a rule, and the weak-shear item in flash-flood forecasting descends from the environments survey of Maddox et al. (1979).

How they bear on the open questions

The open question on the sign of the shear dependence stays open, and the sources make its closure condition explicit: the inflow ratio and the condensate ratio computed across shear on the same real storms, so that the field and model results can be mapped onto each other. Jo and Lasher-Trapp (2022) pair a surface inflow ratio with a core condensation efficiency for one model and one sounding and find the two diverging, which shows that the 1982 conflict can arise between two well-defined quantities without either side being wrong. The card's subquestion, whether the conflict is an artefact of warm-rain microphysics or of the isolated-cell scope, is narrowed rather than answered: the ice-including supercell runs raise condensate made per unit of ingested vapour with shear, a related but different quantity from Weisman and Klemp's rain-to-condensate ratio, which no ice-including study here recomputes across shear, while the scope question also remains, since Market et al. (2003) measured mesoscale systems and Fankhauser (1988) discrete thunderstorms, unicellular and multicellular, none solidly a supercell. No source here supplies a field measurement of the inflow ratio across shear since 1988, and both supercell studies call for observations, Jo and Lasher-Trapp (2022) for that ratio in particular. Fankhauser's expectation that such a dataset was unlikely ever to be assembled has held for 38 years.

For the estimation gap's subquestion, whether satellite retrievals can proxy efficiency operationally, Kukulies et al. (2026) answer for the condensate definition only: ice-water-path growth gives a bulk efficiency that agrees with a 4 km model by storm type, validated on mesoscale convective systems, while the inflow ratio of the rate equation has no direct observational route beyond cloud-base aircraft; Sui et al. (2007) note that it can be estimated from assimilation data, the route Market et al. (2003) approximated with analysed column water. The definitional split bears on the precipitation domain's process-rate gap (Morrison et al., 2020): efficiency is a ratio of process rates, and the observable ratio and the modelled ratio are ratios of different rates. Kain's (2004) case against the shear rule is an instance of the convective-parameterization inadequacy that Fritsch and Carbone (2004) name, answered by replacing an empirical closure with a downdraft formulation.

• The shear-dependence question stays open, and its closure condition is now explicit: both ratios on the same real storms across shear, which no field study in these sources has done and which Jo and Lasher-Trapp (2022) approximate in a model with a core condensation efficiency beside a surface inflow ratio.
• The warm-rain subquestion stays open: the ice-including supercell runs raise condensate made per unit of ingested vapour, not rain per unit of condensate, so Weisman and Klemp's rising ratio has not been reproduced with ice in these sources.
• Satellite ice water path can proxy the condensate definition of efficiency by storm type, so the estimation gap narrows to the inflow ratio the rate equation uses.

References

1. Doswell, C. A., III, Brooks, H. E., & Maddox, R. A. (1996). Flash flood forecasting: An ingredients-based methodology. Weather and Forecasting, 11(4), 560–581. https://doi.org/10.1175/1520-0434(1996)011<0560:FFFAIB>2.0.CO;2
2. Fankhauser, J. C. (1988). Estimates of thunderstorm precipitation efficiency from field measurements in CCOPE. Monthly Weather Review, 116(3), 663–684. https://doi.org/10.1175/1520-0493(1988)116<0663:EOTPEF>2.0.CO;2
3. Fritsch, J. M., & Carbone, R. E. (2004). Improving quantitative precipitation forecasts in the warm season: A USWRP research and development strategy. Bulletin of the American Meteorological Society, 85(7), 955–965. https://doi.org/10.1175/BAMS-85-7-955
4. Fritsch, J. M., & Chappell, C. F. (1980). Numerical prediction of convectively driven mesoscale pressure systems. Part I: Convective parameterization. Journal of the Atmospheric Sciences, 37(8), 1722–1733. https://doi.org/10.1175/1520-0469(1980)037<1722:NPOCDM>2.0.CO;2
5. Jo, E., & Lasher-Trapp, S. (2022). Entrainment in a simulated supercell thunderstorm. Part II: The influence of vertical wind shear and general effects upon precipitation. Journal of the Atmospheric Sciences, 79(5), 1429–1443. https://doi.org/10.1175/JAS-D-21-0289.1
6. Kain, J. S. (2004). The Kain–Fritsch convective parameterization: An update. Journal of Applied Meteorology, 43(1), 170–181. https://doi.org/10.1175/1520-0450(2004)043<0170:TKCPAU>2.0.CO;2
7. Kukulies, J., Prein, A. F., Done, J. M., Stansfield, A. M., & Rios-Berrios, R. (2026). Precipitation efficiency by storm type in km-scale climate simulations and satellite observations. Journal of Geophysical Research: Atmospheres, 131(5), e2025JD044800. https://doi.org/10.1029/2025JD044800
8. Maddox, R. A., Chappell, C. F., & Hoxit, L. R. (1979). Synoptic and meso-α scale aspects of flash flood events. Bulletin of the American Meteorological Society, 60(2), 115–123. https://doi.org/10.1175/1520-0477-60.2.115
9. Market, P. S., Allen, S. N., Scofield, R., Gruber, A., & Kuligowski, R. (2002). A statistical expression for the estimation of MCS precipitation efficiency (Paper 7B.4). 19th Conference on Weather Analysis and Forecasting / 15th Conference on Numerical Weather Prediction, American Meteorological Society. https://ams.confex.com/ams/pdfpapers/47631.pdf
10. Market, P., Allen, S., Scofield, R., Kuligowski, R., & Gruber, A. (2003). Precipitation efficiency of warm-season Midwestern mesoscale convective systems. Weather and Forecasting, 18(6), 1273–1285. https://doi.org/10.1175/1520-0434(2003)018<1273:PEOWMM>2.0.CO;2
11. Marwitz, J. D. (1972). Precipitation efficiency of thunderstorms on the High Plains. Journal de Recherches Atmosphériques, 6, 367–370.
12. Morrison, H., van Lier-Walqui, M., Fridlind, A. M., Grabowski, W. W., Harrington, J. Y., Hoose, C., Korolev, A., Kumjian, M. R., Milbrandt, J. A., Pawlowska, H., Posselt, D. J., Prat, O. P., Reimel, K. J., Shima, S.-I., van Diedenhoven, B., & Xue, L. (2020). Confronting the challenge of modeling cloud and precipitation microphysics. Journal of Advances in Modeling Earth Systems, 12, e2019MS001689. https://doi.org/10.1029/2019MS001689
13. Mulholland, J. P., Nowotarski, C. J., Peters, J. M., Morrison, H., & Nielsen, E. R. (2024). How does vertical wind shear influence updraft characteristics and hydrometeor distributions in supercell thunderstorms? Monthly Weather Review, 152(7), 1663–1687. https://doi.org/10.1175/MWR-D-23-0166.1
14. Schumacher, R. S. (2017). Heavy rainfall and flash flooding. In Oxford Research Encyclopedia of Natural Hazard Science. Oxford University Press. https://doi.org/10.1093/acrefore/9780199389407.013.132
15. Sui, C.-H., Li, X., & Yang, M.-J. (2007). On the definition of precipitation efficiency. Journal of the Atmospheric Sciences, 64(12), 4506–4513. https://doi.org/10.1175/2007JAS2332.1
16. Weisman, M. L., & Klemp, J. B. (1982). The dependence of numerically simulated convective storms on vertical wind shear and buoyancy. Monthly Weather Review, 110(6), 504–520. https://doi.org/10.1175/1520-0493(1982)110<0504:TDONSC>2.0.CO;2

Read into the library after publication:

• Estimates of Thunderstorm Precipitation Efficiency from Field Measurements in CCOPE 10.1175/1520-0493(1988)116<0663:EOTPEF>2.0.CO;2
• On the Definition of Precipitation Efficiency 10.1175/2007JAS2332.1
• Entrainment in a Simulated Supercell Thunderstorm. Part II: The Influence of Vertical Wind Shear and General Effects upon Precipitation 10.1175/JAS-D-21-0289.1
• How Does Vertical Wind Shear Influence Updraft Characteristics and Hydrometeor Distributions in Supercell Thunderstorms? 10.1175/MWR-D-23-0166.1
• Precipitation Efficiency by Storm Type in km-Scale Climate Simulations and Satellite Observations 10.1029/2025JD044800
• The Kain-Fritsch Convective Parameterization: An Update 10.1175/1520-0450(2004)043<0170:TKCPAU>2.0.CO;2
• Precipitation Efficiency of Warm-Season Midwestern Mesoscale Convective Systems 10.1175/1520-0434(2003)018<1273:PEOWMM>2.0.CO;2
• Numerical Prediction of Convectively Driven Mesoscale Pressure Systems. Part I: Convective Parameterization 10.1175/1520-0469(1980)037<1722:NPOCDM>2.0.CO;2
• Synoptic and Meso-α Scale Aspects of Flash Flood Events 10.1175/1520-0477-60.2.115

Against the library

How this episode sits against the listener's own paper library. Coverage: partial.

What the sources add

The library holds cold pools as a forecasting input whose state is unobservable (Corfidi, 2003) and as one side of the cold-pool and shear balance (Schumacher & Rasmussen, 2020; Weisman & Klemp, 1982); it holds no direct measurement of a cold pool. The two campaign papers supply that measurement. Kirsch et al. (2022) deployed 103 low-cost stations at about 1.85 km spacing over Hamburg in summer 2020, recorded 37 cold pool events, and in the 10 August 2020 case they describe in detail found a temperature perturbation of about -12 K with a gradient of 12 K over 11 km, against a stated conventional network resolution of about 25 km and 10 min. Kirsch et al. (2024) turned 42 events from the 99-station FESSTVaL network at Lindenberg into objects and report a median equivalent diameter of 8.5 km as a lower bound, aspect ratios of 1.5 to 1.6 independent of size and strength, and a linear radial temperature structure whose gradient steepens from about -0.06 K/km in the weakest quartile to -0.23 K/km in the strongest. For four events observed from birth, cold pool area scaled linearly with radar-accumulated rainfall (r of at least 0.976), about 80% of the cooling came from the first half of the rain, and the radial expansion velocity decreased as the object-mean deficit strengthened, with correlations between observed and density-current-predicted expansion speed of -0.87 to -0.31.

Grant et al. (2024) add the first comparison of this variability with simulations. Variograms over all 42 FESSTVaL cold pools show enhanced temperature variance at 5 to 15 km and suppressed variance below about 4 km. ICON large-eddy simulations of one of the strongest events at 625, 312, 156 and 75 m grid spacing reproduce the mean deficit and its evolution, while their variogram magnitudes are about one third of the observed, with no systematic dependence on resolution; pre-cold-pool variance is adequate at small scales and nocturnal variance lies within the observed interquartile range, which the authors use to locate the shortfall in processes internal to the cold pool. Across the other RAMS simulations, a tropical maritime case study and idealised continental and sheared runs, the variance differs by an order of magnitude between the tropical and a midlatitude continental case, is larger at night than by day and larger on the upshear side, and none of the runs reaches the observed median variogram, including runs with larger mean deficits than observed.

The two Monthly Weather Review papers add the observational record of RKW theory (Rotunno et al., 1988) against measured cold pools. Bryan and Parker (2010) launched nine rawinsondes in 3 h through the 15 May 2009 Oklahoma squall line and found a cold pool 3.6 to 4.0 km deep 8 km behind the gust front, with C of 34 to 38 m/s against a low-to-midlevel shear of about 10 m/s, an upshear tilt consistent with the theory, a depth twice the 1 to 2 km usually quoted, and no basis for saying whether such values are common. Coniglio et al. (2012) document that the theory leaves the shear depth undetermined, 2.5 km in the original simulations, 5 km in the revisit (Weisman & Rotunno, 2004) and possibly 6 to 8 km for cold pools 3 to 4 km deep; that the observational checks they cite are nine BAMEX cases with paired soundings, the list of observed squall lines that two of the theory's authors compiled in a 2005 reply to critics, of which only one is the strong midlatitude type relevant to their case, and the Bryan and Parker line; and that in the 8 May 2009 derecho the strong bow echo formed where the 0.5 to 3 km line-normal shear was 6 to 9 m/s and the 3 to 6 km shear was largest, with 5 to 7 K surface deficits, modest evaporative potential and an unknown cold pool depth.

• Dense low-cost networks put cold pool structure at scales of about 10 km and below: 12 K over 11 km in the Hamburg case of 10 August 2020 and a linear radial gradient of 0.06 to 0.23 K/km at Lindenberg, against a conventional spacing of about 25 km (Kirsch et al., 2022, 2024).
• In four cold pools observed from birth, area grew linearly with accumulated rain and expansion slowed as the deficit deepened, the opposite of the density-current relation (Kirsch et al., 2024).
• Large-eddy simulations of one observed cold pool reproduce its mean deficit but about a third of its temperature variance, at every grid spacing from 625 to 75 m (Grant et al., 2024).
• Direct soundings through one Oklahoma squall line give a cold pool about 4 km deep with C about three times the shear, consistent with the RKW tilt prediction and twice the depth the literature quotes (Bryan & Parker, 2010).
• The observational checks of RKW theory that Coniglio et al. (2012) cite are one nine-case sounding study, one relevant case in the theory's own list and one squall line, and the theory does not fix the depth of shear to use.

Where they differ from the library

The library's statement of the estimation gap, drawn from Corfidi (2003), reads that direct measurement of cold pool strength is impractical. The campaign papers narrow that to an operational rather than a technical impossibility: low-cost stations at about 2 km spacing measured the surface structure of dozens of cold pools in each of two summers (Kirsch et al., 2022, 2024). What they did not measure is as instructive. Most Lindenberg events were only partly sampled in space and time, the filtered sample is biased toward small and weak objects and gives sizes only as lower bounds, and neither campaign observed cold pool depth (Kirsch et al., 2024), which is the quantity C requires.

Schumacher and Rasmussen (2020) record that the applicability of RKW theory to observed squall lines has been questioned because the most intense lines do not always occur in the optimal state. The new sources place the difficulty earlier, in the inputs. Bryan and Parker (2010) find the theory's tilt prediction consistent with a measured C much larger than the shear, and Coniglio et al. (2012) find a strong bow echo under weak low-level shear with the cold pool depth unknown; neither can test its intensity claim, because C was measured at one stage in the first case and not at all in the second, and the derecho reads as consistent with the theory only if its cold pool was deep enough to bring the 3 to 6 km shear into play. Corfidi's (2003) caveat that DCAPE is unreliable under substantial shear, which the library carries, is joined by two further breaks in the proxy chain: Coniglio et al. (2012) report modest evaporative potential alongside 5 to 7 K surface deficits, and Kirsch et al. (2024) find that in growing cold pools the expansion speed follows area rather than density excess. Weisman and Klemp (1982) framed storm mode as a balance of outflow and inflow in simulations where the outflow strength is a model output; the observational sources show that the same quantity remains a guess when the framework is turned around and used for prediction.

• Direct measurement of a cold pool's surface structure is a matter of deployment rather than technique: 2 km station spacing captured dozens of cold pools per summer, while its strength still waits on depth, which neither campaign observed (Kirsch et al., 2022, 2024).
• The proxy chain from environment to gust-front speed breaks at two links: DCAPE misreads strength under shear and modest evaporative potential coexisted with 5 to 7 K deficits (Corfidi, 2003; Coniglio et al., 2012), and strength did not set expansion speed in the four growing pools measured (Kirsch et al., 2024).
• The question over RKW theory is one of unmeasured inputs before it is one of theory: the tilt check passes with C measured at one stage of one storm, and the intensity claim has not had, in these sources, the repeated measurements it needs (Bryan & Parker, 2010; Coniglio et al., 2012).

How they bear on the open questions

The open question of estimating cold pool state in real time stays open in substance. Its first subquestion, whether radar gust-front position and speed plus mesonet thermodynamics could invert cold pool intensity, meets four counterexamples in Kirsch et al. (2024): in those four young cold pools the expansion speed was set by area growth, itself set by rainfall, and was anticorrelated with the temperature deficit, so a speed-based inversion would recover the rain supply rather than the density excess. The station-level relation between local temperature drop and wind gust does hold in their data, so the local and the whole-object problems separate. The second subquestion, what replaces DCAPE under strong shear, has no answer in these sources; Corfidi's (2003) momentum rule gives gust-front speed without intensity, and the ten misses of the technique built on it include systems steered by an outside boundary, carried by a fast front, or containing supercells or rear-inflow jets. The cold pool audit that Bryan and Parker (2010) proposed, a climatology of depth and integrated buoyancy, is not reported by any later source here.

For the observation gap on boundary-layer thermodynamics, the library's synthesis holds that historical limitations do not establish that present-day kilometre-scale measurement is impossible. FESST@HH and FESSTVaL supply the demonstration for near-surface temperature and pressure over campaign domains of 30 to 50 km (Kirsch et al., 2022, 2024); coverage, permanence and the moisture field remain the open part. For the representation of organised convection in NWP, Grant et al. (2024) bear on the subquestion of whether convection-permitting resolution alone fixes the representation: at large-eddy grid spacings the mean cold pool is reproduced and its internal variability is not, and refining the grid from 625 to 75 m does not change that, in one case with one model. Pathak et al. (2025) judge StormCast's cold pool structures against HRRR analyses; Grant et al.'s finding names one property such an emulator would inherit from its parent and that the design cannot detect. Grant et al.'s attribution of the missing variance to surface fluxes and entrainment inside cold pools, and their finding that daytime cold pools in the haboob ensemble are less variable than nocturnal ones, which they put down to faster dissipation by surface heating, touch the land-atmosphere coupling domain and the open question of MCS dissipation without settling either.

• Real-time cold pool state estimation stays open: in the four growing cold pools measured, a gust-front-speed inversion would recover rain supply rather than density excess, and no source offers a replacement for DCAPE.
• Kilometre-scale surface thermodynamic measurement is demonstrated for campaign domains, so the boundary-layer observation gap is now about coverage and moisture rather than feasibility.
• Resolution alone does not fix cold pool representation in one large-eddy case, which bears on whether convection-permitting grids can close the organised-convection gap and on what a km-scale emulator inherits from its parent model.

References

1. Bryan, G. H., & Parker, M. D. (2010). Observations of a squall line and its near environment using high-frequency rawinsonde launches during VORTEX2. Monthly Weather Review, 138(11), 4076–4097. https://doi.org/10.1175/2010MWR3359.1
2. Coniglio, M. C., Corfidi, S. F., & Kain, J. S. (2012). Views on applying RKW theory: An illustration using the 8 May 2009 derecho-producing convective system. Monthly Weather Review, 140(3), 1023–1043. https://doi.org/10.1175/MWR-D-11-00026.1
3. Corfidi, S. F. (2003). Cold pools and MCS propagation: Forecasting the motion of downwind-developing MCSs. Weather and Forecasting, 18(6), 997–1017. https://doi.org/10.1175/1520-0434(2003)018<0997:CPAMPF>2.0.CO;2
4. Grant, L. D., Kirsch, B., Bukowski, J., Falk, N. M., Neumaier, C. A., Sakradzija, M., van den Heever, S. C., & Ament, F. (2024). How variable are cold pools? Geophysical Research Letters, 51, e2023GL106784. https://doi.org/10.1029/2023GL106784
5. Kirsch, B., Hohenegger, C., & Ament, F. (2024). Morphology and growth of convective cold pools observed by a dense station network in Germany. Quarterly Journal of the Royal Meteorological Society, 150(759), 857–876. https://doi.org/10.1002/qj.4626
6. Kirsch, B., Hohenegger, C., Klocke, D., Senke, R., Offermann, M., & Ament, F. (2022). Sub-mesoscale observations of convective cold pools with a dense station network in Hamburg, Germany. Earth System Science Data, 14(8), 3531–3548. https://doi.org/10.5194/essd-14-3531-2022
7. Pathak, J., Cohen, Y., Garg, P., Harrington, P., Brenowitz, N., Durran, D., Mardani, M., Vahdat, A., Xu, S., Kashinath, K., & Pritchard, M. (2025). Kilometer-scale convection allowing model emulation using generative diffusion modeling. Science Advances. https://doi.org/10.1126/sciadv.adv0423
8. Rotunno, R., Klemp, J. B., & Weisman, M. L. (1988). A theory for strong, long-lived squall lines. Journal of the Atmospheric Sciences, 45(3), 463–485. https://doi.org/10.1175/1520-0469(1988)045<0463:ATFSLL>2.0.CO;2
9. Schumacher, R. S., & Rasmussen, K. L. (2020). The formation, character and changing nature of mesoscale convective systems. Nature Reviews Earth & Environment. https://doi.org/10.1038/s43017-020-0057-7
10. Weisman, M. L., & Klemp, J. B. (1982). The dependence of numerically simulated convective storms on vertical wind shear and buoyancy. Monthly Weather Review, 110(6), 504–520. https://doi.org/10.1175/1520-0493(1982)110<0504:TDONSC>2.0.CO;2
11. Weisman, M. L., & Rotunno, R. (2004). "A theory for strong long-lived squall lines" revisited. Journal of the Atmospheric Sciences, 61(4), 361–382. https://doi.org/10.1175/1520-0469(2004)061<0361:ATFSLS>2.0.CO;2

Read into the library after publication:

• Sub-mesoscale observations of convective cold pools with a dense station network in Hamburg, Germany 10.5194/essd-14-3531-2022
• Morphology and growth of convective cold pools observed by a dense station network in Germany 10.1002/qj.4626
• How Variable Are Cold Pools? 10.1029/2023GL106784
• Observations of a Squall Line and Its Near Environment Using High-Frequency Rawinsonde Launches during VORTEX2 10.1175/2010MWR3359.1
• Views on Applying RKW Theory: An Illustration Using the 8 May 2009 Derecho-Producing Convective System 10.1175/MWR-D-11-00026.1
• A Theory for Strong, Long-Lived Squall Lines 10.1175/1520-0469(1988)045<0463:ATFSLL>2.0.CO;2
• "A Theory for Strong Long-Lived Squall Lines" Revisited 10.1175/1520-0469(2004)061<0361:ATFSLS>2.0.CO;2

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library already holds the structural case against ERA5 as the sole verification reference: the assimilation lookahead and the precipitation placeholder in Rasp et al. (2024), the station-versus-reanalysis discrepancies in Jin et al. (2024), the operational-condition drops in Li et al. (2026) and the reference-swap flip in Kasteleyn et al. (2026). What the three sources new to the library add is a demonstration on the same generation of models that the verdict on extremes changes with the grader. Olivetti and Messori (2024) show, on ERA5 alone, that GraphCast wins the extreme-day RMSE contest against IFS HRES across cold, hot and windy extremes while underestimating the warmest points by about 2 K over North America, with the shortfall growing with lead time; the two results coexist because RMSE rewards hedged forecasts even in the tails.

Pasche et al. (2025) supply the case the library did not hold: HRES scored against its own zero-lead forecast and the ML models against ERA5, by explicit design so that HRES has no error at lead time zero, with the result that HRES has smaller error than every ML model up to 6.5 days at the peak of the June 2021 Pacific Northwest heatwave, the reverse of the ordering in the 2022 baseline summer, while the ML models forecast the February 2021 North American winter storm better. The same paper records that surface relative humidity differs substantially between ERA5 and the HRES analysis, so that the heat index distributions computed from the two truths differ, and recommends direct comparison against station observations. Ben Bouallègue et al. (2024) add the design in which both models share the operational IFS analysis as initial condition and are verified against operational analysis and European SYNOP stations: Pangu-Weather remains competitive, beating IFS on summer heat between the 75th and 90th station percentiles, while neither model captures the lowest winter temperatures and Pangu-Weather underestimates tropical-cyclone intensity.

• On the same ERA5 exam, GraphCast beats IFS HRES on extreme-day RMSE while underestimating the warmest points by about 2 K, because RMSE rewards hedged forecasts even in the tails (Olivetti & Messori, 2024).
• With HRES graded against its own analysis and the ML models against ERA5, HRES has the smaller error up to 6.5 days at the peak of the 2021 Pacific Northwest heatwave, reversing the baseline-summer ordering (Pasche et al., 2025).
• The two model-based truths disagree substantially on surface humidity for the 2023 South Asian heatwave, so a ranking between models graded on different keys is partly a ranking of the keys (Pasche et al., 2025).
• With a shared operational initial condition and station verification over Europe, Pangu-Weather stays competitive for summer heat but neither model captures the lowest winter temperatures (Ben Bouallègue et al., 2024).

Where they differ from the library

The library's synthesis on the reanalysis-reference question already separates the reference-swap sensitivity from the initialization sensitivity and records Jin et al.'s (2024) finding that overall accuracy is not very sensitive to the observation reference. The new sources sharpen rather than contradict that reading. Ben Bouallègue et al. (2024) find the station verdict on 2 m temperature to be similar to the analysis verdict for the headline metrics, which extends Jin's insensitivity finding to a second, independent station network and model pair; the sensitivity in both studies is confined to the tails and to specific variables such as cyclone intensity.

Biegert et al. (2026) is held in the library as finding higher potential extreme skill for AIWP models under threshold-weighted scoring, and Pasche et al. (2025) appear to say the opposite for a single event. The episode's sources resolve this as a difference of conditioning rather than of models: Pasche condition on an event having occurred, Olivetti and Messori condition on observed extreme grid points, and Biegert score all days with a weighting, so the forecaster's dilemma that Biegert invoke is the mechanism that separates the verdicts. Pasche et al. also note the selection-bias risk of choosing events by real-world impact, which the library's extremes card does not record.

• The insensitivity of mean skill to the reference is now shown on two independent station networks and model pairs (Jin et al., 2024; Ben Bouallègue et al., 2024), so the reference question is a tails question.
• The opposite verdicts of Biegert et al. (2026) and Pasche et al. (2025) on AI models and extremes come from conditioning on all days versus on events that occurred, the forecaster's dilemma, and are consistent with each other.

How they bear on the open questions

On the open question of whether a station plus operational-analysis dual reference is sufficient, the new sources leave it open in substance. Pasche et al. (2025) show that the two model-based truths disagree on surface humidity for a humid heatwave and offer station comparison as the remedy, but do not perform it; Ben Bouallègue et al. (2024) perform it for 2 m temperature over Europe only. Observation-sparse regions, the library's first open subquestion, are untouched, and no source verifies precipitation extremes against stations at global scale, so the reference for the variable Rasp et al. (2024) call a placeholder remains ERA5.

On the second open subquestion, whether Jin's insensitivity finding bounds how much reference choice matters for mean skill, Ben Bouallègue et al. (2024) supply a second independent instance of the same bound for mean 2 m temperature error, and Jin et al.'s (2024) own heatwave case shows Aurora's advantage over the ECMWF model nearly vanishing at the daily maximum within two days, with both models sharing ERA5's bias pattern. The bound therefore holds for mean skill and fails at the tails, which is a substantive partial answer rather than a closure. The extremes-blind-protocol card's question about which sampling protocol makes extreme skill measurable is advanced by Biegert's all-day weighted scoring and by Pasche's event-centred design, but the two designs give opposite rankings on the same models, and no source has yet shown them agreeing on a held-out year, so that question stays open. All evaluated models date from 2023 to 2025; nothing here bears on the newer generation.

• Whether a station plus operational-analysis reference suffices stays open: humidity disagreement between the two model-based truths is documented but not resolved by station comparison, and sparse regions and global precipitation extremes are untested.
• Jin's insensitivity bound holds for mean skill and fails at the tails, where Aurora's advantage nearly vanishes at the daily maximum within two days (Jin et al., 2024).
• Event-centred and all-day weighted protocols rank the same models oppositely, so which sampling protocol makes extreme skill measurable remains unanswered.

References

1. Ben Bouallègue, Z., Clare, M. C. A., Magnusson, L., Gascón, E., Maier-Gerber, M., Janoušek, M., Rodwell, M., Pinault, F., Dramsch, J. S., Lang, S. T. K., Raoult, B., Rabier, F., Chevallier, M., Sandu, I., Dueben, P., Chantry, M., & Pappenberger, F. (2024). The rise of data-driven weather forecasting: A first statistical assessment of machine learning-based weather forecasts in an operational-like context. Bulletin of the American Meteorological Society, 105(6), E864–E883. https://doi.org/10.1175/BAMS-D-23-0162.1
2. Biegert, T., Allen, S., Alber, A., & Lerch, S. (2026). Towards fair comparisons of AI- and physics-based weather models for extreme events via the weighted potential CRPS. arXiv. https://doi.org/10.48550/arXiv.2606.21170
3. Jin, W., Weyn, J., Zhao, P., Xiang, S., Bian, J., Fang, Z., Dong, H., Sun, H., Thambiratnam, K., & Zhang, Q. (2024). WeatherReal: A benchmark based on in-situ observations for evaluating weather models. arXiv. https://doi.org/10.48550/arXiv.2409.09371
4. Kasteleyn, E., Maier, T., Lauer, A., Eyring, V., Gentine, P., & Lucic, A. (2026). PhysMetrics.Weather: An evaluation framework for physical consistency in ML weather models. arXiv. https://doi.org/10.48550/arXiv.2606.10642
5. Li, R., Wen, Z., Han, T., Chen, H., Ling, F., Zhang, W., Guo, S., & Bai, L. (2026). RealBench: Benchmarking data-driven numerical weather forecasting under operational conditions and extreme event challenges. arXiv. https://doi.org/10.48550/arXiv.2605.24945
6. Olivetti, L., & Messori, G. (2024). Do data-driven models beat numerical models in forecasting weather extremes? A comparison of IFS HRES, Pangu-Weather, and GraphCast. Geoscientific Model Development, 17(21), 7915–7962. https://doi.org/10.5194/gmd-17-7915-2024
7. Pasche, O. C., Wider, J., Zhang, Z., Zscheischler, J., & Engelke, S. (2025). Validating deep learning weather forecast models on recent high-impact extreme events. Artificial Intelligence for the Earth Systems, 4(1). https://doi.org/10.1175/AIES-D-24-0033.1
8. Rasp, S., Hoyer, S., Merose, A., Langmore, I., Battaglia, P., Russell, T., Sanchez-Gonzalez, A., Yang, V., Carver, R., Agrawal, S., Chantry, M., Ben Bouallègue, Z., Dueben, P., Bromberg, C., Sisk, J., Barrington, L., Bell, A., & Sha, F. (2024). WeatherBench 2: A benchmark for the next generation of data-driven global weather models. Journal of Advances in Modeling Earth Systems, 16(6), e2023MS004019. https://doi.org/10.1029/2023MS004019

Read into the library after publication:

• Do data-driven models beat numerical models in forecasting weather extremes? A comparison of IFS HRES, Pangu-Weather, and GraphCast 10.5194/gmd-17-7915-2024
• Validating Deep Learning Weather Forecast Models on Recent High-Impact Extreme Events 10.1175/AIES-D-24-0033.1
• The Rise of Data-Driven Weather Forecasting: A First Statistical Assessment of Machine Learning-Based Weather Forecasts in an Operational-Like Context 10.1175/BAMS-D-23-0162.1

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library records the aggregation mechanism as an author-stated challenge in two reviews (Runge et al., 2019; Camps-Valls et al., 2023) and as a failure mode that several applied studies sit inside without testing, but it holds no source that derives the mechanism. The episode adds the derivations. Gong et al. (2015) show that regular subsampling by a factor k of a first-order vector autoregression yields a series that is still a vector autoregression but with transition matrix equal to the k-th power of the original, and they give a two-variable example in which the k = 2 power is diagonal so that a genuine causal arrow is absent from the subsampled data. Gong et al. (2017) show that averaging k consecutive observations yields a vector autoregressive moving-average process, that a plain autoregressive fit again recovers only the k-th power, and that as k grows without bound the aggregated series converges to a linear instantaneous causal model; the original matrix remains identifiable when the noise is non-Gaussian, and the authors advise estimating only the no-self-loop part when k exceeds about six.

Barnett and Seth (2017) add the quantitative shape that the library lacked. For a continuous-time system with an explicit causal delay, Granger causality in the subsampled process decays exponentially once the sampling interval exceeds the delay, while below the delay detectability shows sweet spots and black spots, with a globally optimal interval well below the delay and cases in which coarser sampling improves detection. The authors conclude that sampling rates for causal analysis should be informed by domain-specific time scales. This is the first source in the library's orbit that makes the dependence of detectability on cadence non-monotone, which bears directly on how the missing two-cadence experiment should be designed.

Krich et al. (2020) supply an Earth-science statement of the trade-off that the library did not hold: increasing temporal resolution can resolve contemporaneous links and raise detection power through more samples, but it increases dimensionality if the maximal lag is adapted, splits causal information across more lags so that each is less detectable, and exposes more observational noise. Their half-hourly eddy-covariance analysis, restricted to midday samples with lags 0 to 8, still finds contemporaneous dependence predominant, which establishes that some contemporaneity in biosphere-atmosphere networks is physical rather than an archival artefact even at thirty-minute cadence.

• Subsampling a lagged linear system by a factor k leaves a lagged system whose matrix is the k-th power of the original, and that power can be diagonal where the original was not (Gong et al., 2015).
• Averaging k consecutive observations of the same system yields a moving-average process whose limit as k grows is a purely instantaneous model, so lagged relations become same-time relations (Gong et al., 2017).
• For a continuous-time system with a causal delay, detectability of the link decays exponentially once the sampling interval exceeds the delay and is non-monotone below it, with an optimum well below the delay (Barnett & Seth, 2017).
• Half-hourly biosphere-atmosphere data analysed at midday still show predominantly contemporaneous dependence, so part of the contemporaneity in flux-tower causal networks is physical (Krich et al., 2020).

Where they differ from the library

The library's synthesis treats contemporaneity at coarse cadence as consistent with aggregation but untested, and the episode's sources sharpen rather than contradict that reading. Barnett and Seth (2017) qualify any expectation that finer sampling monotonically restores lag structure: below the causal delay, detectability oscillates and can worsen, so a two-cadence comparison could show links that are harder to detect at the finer cadence for reasons unrelated to aggregation. Krich et al. (2020) qualify it from the other side: their finding of predominant contemporaneous dependence at half-hourly resolution means that a share of the contemporaneity in the daily FLUXNET networks reported by Camps-Valls et al. (2023) would survive any feasible increase in resolution. Both qualifications concern the interpretation of the two-cadence experiment, not whether it is needed.

Gong et al. (2017) also qualify the practical reach of aggregation-aware recovery. Identifiability of the fine-scale matrix from aggregated data rests on non-Gaussian noise and, for large k, on an instantaneous-model approximation that can recover only the matrix without its self-loop entries. For the cadences in the library's applied studies, with aggregation factors in the tens to hundreds relative to sub-daily processes, this means an aggregation-aware method could at best bound the fine-scale graph rather than reconstruct it, which is the second of the gap's open subquestions.

• Finer sampling does not monotonically restore lag structure: below the causal delay there are sampling intervals at which detection worsens (Barnett & Seth, 2017).
• Aggregation-aware recovery of the fine-scale matrix depends on non-Gaussian noise and, at large aggregation factors, recovers at best the matrix without its self-loops (Gong et al., 2017).

How they bear on the open questions

The library's open question asks how much of the reported contemporaneity in daily or monthly causal analyses is an aggregation artefact and whether causal structure can be recovered from aggregated series. The episode's sources settle the mechanism and its asymptotics (Gong et al., 2015, 2017), quantify detectability as a function of cadence for one delay model (Barnett & Seth, 2017), and state the trade-off in Earth-science terms (Krich et al., 2020). None of them re-analyses an Earth-system dataset at two cadences. The question therefore remains open, and it remains open in substance: the missing item is an experiment on a real system with hourly and daily versions of the same fields, together with the methodological question of whether aggregation-aware estimators can do more than bound the fine-scale graph. Debeire et al. (2025) moved from three-day to daily means relative to Nowack et al. (2020) but changed ensemble, period and node count simultaneously, so it still does not count as a cadence comparison.

For land-atmosphere coupling, the episode connects the same mechanism to the studies episode three built on. Sun et al. (2025) compute information flow on daily fields for a mechanism they place within the diurnal cycle and call for higher-frequency output; Tuttle and Salvucci (2016) test next-day Granger causality at a daily step and bound their result to that timestep. Both sit inside the gap as untested cases, and the library's separate open question on sub-daily archival is the data-side prerequisite for the experiment the episode proposes. For the causal-fingerprint work in the library (Nowack et al., 2020; Karmouche et al., 2023; Falkena et al., 2025), Barnett and Seth's result implies that lag-tolerant network scores computed at a single cadence compare graphs whose lag structure is itself a function of that cadence, which is a condition the fingerprint protocols have not examined.

• The mechanism behind the library's open question is now derived and quantified, and the question stays open because no Earth-system dataset has been analysed at two cadences with method, variables and period held fixed.
• The experiment that would close it is one system with hourly and daily versions of the same fields, counting the lag-zero links that become lagged and those that stay, with a non-monotone outcome expected (Barnett & Seth, 2017; Krich et al., 2020).
• The soil-moisture studies of episode three (Sun et al., 2025; Tuttle & Salvucci, 2016) and the causal-fingerprint protocols in the library (Nowack et al., 2020; Karmouche et al., 2023; Falkena et al., 2025) all sit inside this untested condition.

References

1. Barnett, L., & Seth, A. K. (2017). Detectability of Granger causality for subsampled continuous-time neurophysiological processes. Journal of Neuroscience Methods, 275, 93–121. https://doi.org/10.1016/j.jneumeth.2016.10.016
2. Camps-Valls, G., Gerhardus, A., Ninad, U., Varando, G., Martius, G., Balaguer-Ballester, E., Vinuesa, R., Diaz, E., Zanna, L., & Runge, J. (2023). Discovering causal relations and equations from data. Physics Reports, 1044, 1–68. https://doi.org/10.1016/j.physrep.2023.10.005
3. Debeire, K., Bock, L., Nowack, P., Runge, J., & Eyring, V. (2025). Constraining uncertainty in projected precipitation over land with causal discovery. Earth System Dynamics, 16, 607–630. https://doi.org/10.5194/esd-16-607-2025
4. Falkena, S. K. J., Dijkstra, H. A., & von der Heydt, A. S. (2025). Causal mechanisms of subpolar gyre variability in CMIP6 models. Earth System Dynamics, 16, 1833–1844. https://doi.org/10.5194/esd-16-1833-2025
5. Gong, M., Zhang, K., Schölkopf, B., Glymour, C., & Tao, D. (2017). Causal discovery from temporally aggregated time series. In Proceedings of the 33rd Conference on Uncertainty in Artificial Intelligence (UAI 2017). http://auai.org/uai2017/proceedings/papers/269.pdf
6. Gong, M., Zhang, K., Schölkopf, B., Tao, D., & Geiger, P. (2015). Discovering temporal causal relations from subsampled data. In Proceedings of the 32nd International Conference on Machine Learning, PMLR 37, 1898–1906. http://proceedings.mlr.press/v37/gongb15.pdf
7. Karmouche, S., Galytska, E., Runge, J., Meehl, G. A., Phillips, A. S., Weigel, K., & Eyring, V. (2023). Regime-oriented causal model evaluation of Atlantic–Pacific teleconnections in CMIP6. Earth System Dynamics, 14, 309–344. https://doi.org/10.5194/esd-14-309-2023
8. Krich, C., Runge, J., Miralles, D. G., Migliavacca, M., Perez-Priego, O., El-Madany, T., Carrara, A., & Mahecha, M. D. (2020). Estimating causal networks in biosphere–atmosphere interaction with the PCMCI approach. Biogeosciences, 17, 1033–1061. https://doi.org/10.5194/bg-17-1033-2020
9. Nowack, P., Runge, J., Eyring, V., & Haigh, J. D. (2020). Causal networks for climate model evaluation and constrained projections. Nature Communications, 11, 1415. https://doi.org/10.1038/s41467-020-15195-y
10. Runge, J., Bathiany, S., Bollt, E., Camps-Valls, G., Coumou, D., Deyle, E., Glymour, C., Kretschmer, M., Mahecha, M. D., Muñoz-Marí, J., van Nes, E. H., Peters, J., Quax, R., Reichstein, M., Scheffer, M., Schölkopf, B., Spirtes, P., Sugihara, G., Sun, J., Zhang, K., & Zscheischler, J. (2019). Inferring causation from time series in Earth system sciences. Nature Communications, 10, 2553. https://doi.org/10.1038/s41467-019-10105-3
11. Sun, J., Yang, K., He, X., Wang, G., Wang, Y., Yu, Y., & Lu, H. (2025). Causal pathways underlying global soil moisture-precipitation coupling. Nature Communications, 16, 8935. https://doi.org/10.1038/s41467-025-63999-7
12. Tuttle, S., & Salvucci, G. (2016). Empirical evidence of contrasting soil moisture–precipitation feedbacks across the United States. Science, 352(6287), 825–828. https://doi.org/10.1126/science.aaa7185

Read into the library after publication:

• Causal Discovery from Temporally Aggregated Time Series http://auai.org/uai2017/proceedings/papers/269.pdf
• Discovering Temporal Causal Relations from Subsampled Data http://proceedings.mlr.press/v37/gongb15.pdf
• Detectability of Granger causality for subsampled continuous-time neurophysiological processes 10.1016/j.jneumeth.2016.10.016
• Estimating causal networks in biosphere–atmosphere interaction with the PCMCI approach 10.5194/bg-17-1033-2020
• Functional convergence of biosphere–atmosphere interactions in response to meteorological conditions 10.5194/bg-18-2379-2021

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library already holds the MCS tracking method intercomparison (Feng et al., 2025) and uses it as the first addressing-type evidence that object-based conclusions are sensitive to identification parameters; what it does not hold is the earlier and smaller regional intercomparison by Prein et al. (2024), which runs the same per-tracker model-observation comparison on a regional domain. It states a distinction that the library's MCSMIP card does not record, although MCSMIP's own abstract makes it too: absolute MCS statistics (frequency, initiation frequency, size, duration, contribution to precipitation) vary substantially across six trackers, with annual counts over northwest South America ranging from 493 to 919 and the La Plata rainfall fraction from about 60% to over 80%, whereas model-minus-observation differences in life-cycle characteristics, movement speed and rain volume are consistent across trackers, and all six agree that the 4 km simulation captures MCS characteristics across South American climate zones (Prein et al., 2024). Prein et al. (2024) also name the dominant source of spread, the segmentation of cloud systems in space and time and the treatment of merges and splits, and provide a per-tracker characterisation table that the library's cards for PyFLEXTRKR (Feng et al., 2023) and tobac (Sokolowsky et al., 2024) do not.

The episode also adds two intercomparisons from outside the registered domains that establish the same pattern on other phenomena. Neu et al. (2013) ran fifteen extratropical cyclone detection and tracking methods on ERA-Interim for 1989 to 2009 and found Northern Hemisphere winter totals ranging from about 6,000 to 21,000, track-to-track matching rates of roughly 50 to 70%, and no better agreement between two methods derived from the same original algorithm than between unrelated ones, while counts of deep cyclones, interannual variability and the sign of regional trends were robust. Rutz et al. (2019) ran more than twenty atmospheric river methods on MERRA-2 for 1980 to 2017 and found that absolute frequency varied widely with the restrictiveness of criteria while normalised latitudinal distributions agreed closely and the most restrictive methods' rivers formed an approximate subset of the least restrictive methods' rivers. Neither project is in the library, and together they let the claim about tracker sensitivity be stated for feature tracking as a class rather than for MCS trackers alone.

• Under one shared four-criterion MCS definition, six trackers on the same three years over South America still gave annual system counts from 493 to 919 and La Plata rainfall fractions from about 60% to over 80% (Prein et al., 2024).
• Under every one of those six trackers the model-minus-observation differences in life cycle, speed and rain volume were consistent, and all six agreed the 4 km simulation captures MCS characteristics (Prein et al., 2024).
• Counts are fragile and what each project tested beyond the count is steadier for fifteen cyclone trackers on ERA-Interim, where deep cyclones and trend signs held (Neu et al., 2013), and for more than twenty atmospheric river methods on MERRA-2, where normalized latitudinal shapes and the strongest event held (Rutz et al., 2019); with the two convective studies that makes count sensitivity a property of feature tracking as a class.

Where they differ from the library

The library's card for Feng et al. (2025) once inferred that most model-observation bias figures in that paper were computed through one tracker and were therefore tracker-conditional; the card withdrew that inference on re-reading, and the full text bears the withdrawal out: the frequency, precipitation, diurnal-cycle, life-cycle and median-property comparisons (Figures 6–11 and 13) are shown for every tracker, and only the rain-rate contribution and precipitable-water analyses (Figures 12 and 14) use PyFLEXTRKR alone, with the other trackers reported as qualitatively consistent. The paper's own conclusion matches Prein et al. (2024): a factor of 2–3 tracker spread in observed MCS frequency and rainfall contribution, but model-observation differences largely consistent across trackers. The exceptions are MPAS and ICON, whose extensive cold cloud shields with many small embedded rain cores differ strongly from observed morphology; for them the sign of the MCS precipitation bias differs among trackers. Because the two studies share several authors and five trackers, their agreement is one community's finding reported twice rather than two independent confirmations.

The library's radar-tracking and forecast-verification cards handle the missing truth in different ways: Johnson et al. (1998) scored SCIT against a hand-verified set of storm cells, Davis et al. (2006) describe object matching as mainly a subjective judgment, and Gilleland et al. (2009) tested verification methods on geometric and perturbed cases with known errors, the kind of known-error test bed that could in principle be built for trackers. Neu et al. (2013) and Rutz et al. (2019) frame it differently: the methods were each built to answer a different scientific question, their disagreements arise from the interplay of many settings rather than from one identifiable design axis, and the authors decline to rank them because every one is 'right' for its own purpose. On that reading, a common benchmark would measure agreement, not correctness, and the library's open subquestion about an algorithm-independent cell definition is confirmed as the crux rather than answered.

• The library's card once inferred that MCSMIP's model biases were tracker-conditional and has withdrawn it; the full paper shows them for every tracker and finds them largely consistent, except for the two models whose storms look least like observed ones (Feng et al., 2025).
• Neu et al. (2013) and Rutz et al. (2019) treat tracker disagreement as the legitimate product of different scientific questions rather than as an error to be verified away, so a common benchmark would measure agreement, not correctness.

How they bear on the open questions

The library holds three open questions on this topic: whether object identification parameters change verification conclusions (partially addressed since Feng et al., 2025), whether any algorithm-independent definition of a cell makes tracker verification reproducible, and whether forecast-observation object matching can be assessed without match ground truth. Prein et al. (2024) and Feng et al. (2025) bear on the first by showing that the conclusions which flip are the count-based ones and the conclusions which hold are the differenced, shape-based ones, which sharpens the question rather than closing it. On the second and third, the two convective intercomparisons and Neu et al. (2013) say in terms that no reference data set exists, Rutz et al. (2019) decline to rank methods built to answer different questions, and Neu et al. (2013) explicitly propose a best-track data set for extreme cyclones as future work, which leaves the question open in substance: the missing item is a definition, and the improvements each project proposes for itself, finer temporal cadence in Feng et al. (2025), sensitivity studies of individual settings in Neu et al. (2013), an online comparison tool in Rutz et al. (2019), are technical and do not supply one. The library's separate open question about tracking cadence missing short-lived cells (Li et al., 2021) is likewise untouched: every study here ran at hourly or coarser cadence.

Across domains, the pattern connects to the ml-weather-forecasting domain's benchmark and verification cards, where object-based scores are used to evaluate nowcasts and km-scale AI models: if MCS counts and rain fractions carry a factor-of-two tracker dependence in observations alone, object-based evaluation of a forecast system inherits that dependence unless the same tracker is applied to both sides and the reported metric is a difference or a composite shape, which is the practice Prein et al. (2024) recommend. It also connects to the precipitation domain through the attribution question itself: how much rainfall is organised convection, and how much is atmospheric-river transport, are both shown here to be answers conditioned on an identification method, which qualifies any climatological figure the library later records for those fractions.

• The library's question of whether identification choices flip verification conclusions is sharpened: count-based conclusions flip, differenced shape-based ones hold, but this rests on two studies from one overlapping group of authors (Prein et al., 2024; Feng et al., 2025).
• The library's questions about an algorithm-independent object definition and a matching standard remain open in substance, because the convective and cyclone intercomparisons state that no reference exists, the river project ranks nothing, and each proposes only technical refinements.
• Object-based evaluation of forecast systems in the ml-weather-forecasting domain inherits the same tracker dependence unless one tracker is applied to both sides and differences, not counts, are reported.

References

1. Davis, C., Brown, B., & Bullock, R. (2006). Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Monthly Weather Review, 134. https://doi.org/10.1175/MWR3145.1
2. Feng, Z., Hardin, J., Barnes, H. C., Li, J., Leung, L. R., Varble, A., & Zhang, Z. (2023). PyFLEXTRKR: A flexible feature tracking Python software for convective cloud analysis. Geoscientific Model Development, 16, 2753–2776. https://doi.org/10.5194/gmd-16-2753-2023
3. Feng, Z., Prein, A. F., Kukulies, J., Fiolleau, T., Jones, W. K., Maybee, B., Moon, Z. L., Núñez Ocasio, K. M., Molina, M. J., Albright, M. G., Robledo, V., Song, J., Song, F., Leung, L. R., Varble, A. C., Klein, C., Roca, R., & Mejia, J. F. (2025). Mesoscale Convective Systems Tracking Method Intercomparison (MCSMIP): Application to DYAMOND global km-scale simulations. Journal of Geophysical Research: Atmospheres, 130(8), e2024JD042204. https://doi.org/10.1029/2024JD042204
4. Gilleland, E., Ahijevych, D., Brown, B. G., Casati, B., & Ebert, E. E. (2009). Intercomparison of spatial forecast verification methods. Weather and Forecasting, 24. https://doi.org/10.1175/2009WAF2222269.1
5. Johnson, J. T., MacKeen, P. L., Witt, A., Mitchell, E. D., Stumpf, G. J., Eilts, M. D., & Thomas, K. W. (1998). The Storm Cell Identification and Tracking algorithm: An enhanced WSR-88D algorithm. Weather and Forecasting, 13. https://doi.org/10.1175/1520-0434(1998)013<0263:TSCIAT>2.0.CO;2
6. Li, J., Feng, Z., Qian, Y., & Leung, L. R. (2021). A high-resolution unified observational data product of mesoscale convective systems and isolated deep convection in the United States for 2004–2017. Earth System Science Data, 13. https://doi.org/10.5194/essd-13-827-2021
7. Neu, U., Akperov, M. G., Bellenbaum, N., Benestad, R., Blender, R., Caballero, R., Cocozza, A., Dacre, H. F., Feng, Y., Fraedrich, K., Grieger, J., Gulev, S., Hanley, J., Hewson, T., Inatsu, M., Keay, K., Kew, S. F., Kindem, I., Leckebusch, G. C., … Wernli, H. (2013). IMILAST: A community effort to intercompare extratropical cyclone detection and tracking algorithms. Bulletin of the American Meteorological Society, 94(4), 529–547. https://doi.org/10.1175/BAMS-D-11-00154.1
8. Prein, A. F., Feng, Z., Fiolleau, T., Moon, Z. L., Núñez Ocasio, K. M., Kukulies, J., Roca, R., Varble, A. C., Rehbein, A., Liu, C., Ikeda, K., Mu, Y., & Rasmussen, R. M. (2024). Km-scale simulations of mesoscale convective systems over South America—A feature tracker intercomparison. Journal of Geophysical Research: Atmospheres, 129(8), e2023JD040254. https://doi.org/10.1029/2023JD040254
9. Rutz, J. J., Shields, C. A., Lora, J. M., Payne, A. E., Guan, B., Ullrich, P., O'Brien, T., Leung, L. R., Ralph, F. M., Wehner, M., Brands, S., Collow, A., Goldenson, N., Gorodetskaya, I., Griffith, H., Kashinath, K., Kawzenuk, B., Krishnan, H., Kurlin, V., … Viale, M. (2019). The Atmospheric River Tracking Method Intercomparison Project (ARTMIP): Quantifying uncertainties in atmospheric river climatology. Journal of Geophysical Research: Atmospheres, 124(24), 13777–13802. https://doi.org/10.1029/2019JD030936
10. Sokolowsky, G. A., Freeman, S. W., Jones, W. K., Kukulies, J., Senf, F., Marinescu, P. J., Heikenfeld, M., Brunner, K. N., Bruning, E. C., Collis, S. M., Jackson, R. C., Leung, G. R., Pfeifer, N., Raut, B. A., Saleeby, S. M., Stier, P., & van den Heever, S. C. (2024). tobac v1.5: Introducing fast 3D tracking, splits and mergers, and other enhancements for identifying and analysing meteorological phenomena. Geoscientific Model Development, 17, 5309–5330. https://doi.org/10.5194/gmd-17-5309-2024

Read into the library after publication:

• Km-scale simulations of mesoscale convective systems over South America—A feature tracker intercomparison 10.1029/2023JD040254

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The library holds this question through one measurement and two conceptual sources: Nguyen et al. (2026) report that process components correlate with numeric correctness at 0.19 to 0.36 on TerraBench, while the survey of Ren et al. (2025) and the position paper of Munir et al. (2026) argue, without data, that end-to-end scoring hides step-level failure. The episode adds three benchmarks. ThinkGeo (Shabbir et al., 2025) comprises 486 remote-sensing tasks with 1,778 verified reasoning steps and 14 tools, the main results covering the 436 optical tasks and eleven models. GPT-4o selects the right tool in 67.7 percent of steps and supplies correct arguments in 34.8 percent, and its final answers are correct in 9.8 percent of tasks, or 20.4 percent under an image-grounded scoring. On the 50 synthetic-aperture-radar tasks GPT-4o reaches 77.9 percent tool accuracy with answers below 6 percent, and the authors conclude that tool competence alone is not sufficient. The paper reports Pearson correlations of 0.33 to 0.44 between its four step metrics and answer accuracy without stating the unit of analysis. Recomputing from its Table 3 reproduces the tool-selection and argument values to three decimals (0.435 and 0.422) as correlations across the eleven model rows; this is our calculation and not a statement in the paper. With eleven points, a coefficient of 0.435 has a 95 percent interval of roughly -0.2 to 0.8 by the Fisher transformation. ThinkGeo also takes its step metrics from the step-by-step mode of GTA (Wang et al., 2024), in which the model is given the preceding reference steps and predicts the next, whereas answers come from end-to-end runs, so the two scores do not describe the same execution.

The second author cluster is the group behind Earth-Agent. Earth-Bench (Feng et al., 2025) has 248 tasks, 104 tools and thirteen language-model backbones; its PDF could not be obtained and its numbers were read through an automated extraction of the arXiv HTML version. Under autonomous planning DeepSeek-V3.1 has the highest coverage of required tools (78.3 percent) with 51.4 percent accuracy, GPT-5 has 68.7 percent coverage with 66.0 percent accuracy, and LLaMA-4 has 16.5 percent coverage with 44.9 percent accuracy, above GPT-4o (65.7 percent coverage, 43.7 percent accuracy). Supplying the steps explicitly raised trajectory metrics for nearly all models, and the authors write that this does not necessarily lead to higher final accuracy; GPT-5 fell from 66.0 to 62.4 percent. Earth-Agent-Pro (Lv et al., 2026), read in full, explains the LLaMA-4 row: Earth-Bench used multiple-choice questions, which the authors say can hide numeric, date and unit errors. They rebuild the 248 task cores as 744 open-ended questions, scored by a language-model judge. In that setting Llama-3.1-8B covers 75.3 percent of required tools and is judged correct on 13.7 percent of questions; the group's fine-tuned 9B model leads every trajectory metric (tool coverage 91.3, parameter accuracy 32.8) and is judged correct on 50.0 percent, against 66.1 percent for GPT-5 with lower trajectory scores. The authors state that workflow fidelity and final-task accuracy remain distinct. An oracle-plan run that fixes the reference tool order gives GPT-5 69.0 percent against 66.1 percent under its own plan, and the base Qwen3.5-9B 46.4 against 38.3 percent; the oracle run also disables repair mechanisms, so the comparison is indicative.

A rereading of TerraBench adds two observations that the library's card does not carry. First, the run-level correlations sit beside a strong model-level association: across the seventeen rows of Table 2 of Nguyen et al. (2026), tool accuracy and numeric score correlate at about 0.86 by our calculation. Second, argument accuracy is the weakest run-level predictor of the numeric score (0.19) although the paper's failure decomposition names wrong argument values, at 69.5 to 96.8 percent of failed items, as one of the two dominant failure modes. The argument metric averages key and value matching over aligned tool pairs, and the mismatch suggests it does not register the argument errors that decide the number. The paper's logistic regression (AUC 0.803, McFadden pseudo R squared 0.160) is fitted on pooled model-item observations, and no within-model relation is reported.

• ThinkGeo's reported correlations of 0.33 to 0.44 between step metrics and answer accuracy are reproduced from its results table as correlations across its eleven models, so they are model-level statistics on eleven points and cannot be distinguished from zero (Shabbir et al., 2025; our recomputation).
• ThinkGeo scores steps in a mode where the model is given the reference steps so far and scores answers from free runs, so its process and answer scores describe different executions.
• On the multiple-choice Earth-Bench a model covering 17 percent of required tools scored 45 percent, and in the same group's open-ended rebuild a model covering 75 percent of tools was judged correct 14 percent of the time, so the answer format shapes the apparent decoupling as much as the process metric does (Feng et al., 2025; Lv et al., 2026).
• Fixing the tool order to the reference in Earth-Agent-Pro moved GPT-5 from 66.1 to 69.0 percent judged accuracy, so for that backbone tool selection and ordering were not where most errors arose (Lv et al., 2026).
• Across TerraBench's seventeen model averages tool accuracy and numeric score correlate at about 0.86 by our calculation, against the 0.19 to 0.36 that Nguyen et al. (2026) report at run level.

Where they differ from the library

The library records that process metrics correlate only moderately with correctness and that trace-alignment leaderboards systematically overstate capability. The sources narrow both statements. The moderate correlation is a run-level result from one benchmark. At the level of model averages in the same benchmark the association is strong, so a process leaderboard in TerraBench ranks models roughly as an answer leaderboard would, with exceptions: Claude Haiku 4.5 is second on process and GPT-5.5 second on answers, and Gemma 4 26B edges GPT-5.4 on process with about a third of its hit rate (Nguyen et al., 2026). The overstatement therefore concerns the level of capability, 59 on process against 23 percent inside tolerance for the best model, and the certification of single answers, more than rank. ThinkGeo's 0.33 to 0.44 cannot be read as a replication of TerraBench's 0.19 to 0.36: by our recomputation it is a model-level statistic on eleven points, from a benchmark in which process and answers are scored in different modes (Shabbir et al., 2025). What does recur across the two author clusters is qualitative. Rankings invert at the top, DeepSeek-V3.1 against GPT-5 in Earth-Bench as Claude Haiku 4.5 against GPT-5.5 in TerraBench, and interventions that improve trajectories, explicit step guidance in Feng et al. (2025) and an oracle plan in Lv et al. (2026), move answers little for strong models.

The library's statement that a right answer for a wrong reason is indistinguishable from a right answer rests on the conceptual argument of Munir et al. (2026) that a correct output can rely on misaligned coordinate systems, inconsistent resolution or wrong unit conversions. The one empirical case of high accuracy with poor process that the new sources supply, LLaMA-4 on Earth-Bench, has a different cause: the multiple-choice format gave credit that did not require the workflow (Feng et al., 2025; Lv et al., 2026). It is evidence about the validity of the answer score, not about invalid intermediate operations cancelling out. No source measures how often a correct answer rests on an invalid intermediate state, so that half of the library's statement remains an argument.

Two further refinements follow. The library's synthesis says that no implemented verifier has been shown to predict final correctness. Earth-Agent-Pro does implement step-level verification, a gate combining deterministic checks with a language-model checker, and an answer controller that checks units, numerical scale, aggregation and coverage before answering (Lv et al., 2026). The paper neither ablates the gate nor reports how often it accepted a wrong result, and GPT-5 is judged wrong on about a third of questions with it in place. The statement should read that a verifier has been built and none has been evaluated as a predictor. Second, the library files trace alignment and state validity under one heading. All four benchmarks score process as agreement with a single reference trace, in TerraBench a canonical trace produced by an annotating agent and approved by annotators. Munir et al. (2026) propose something else, a validity score for each state transition whichever tool produced it. Low alignment does not establish an invalid process, because more than one route can be valid, so part of the measured decoupling belongs to the metric and not to the agents.

• The library's finding that process metrics correlate only moderately with correctness holds for individual runs in TerraBench; across model averages in the same benchmark the association is strong, so process leaderboards overstate the level of capability and cannot certify an answer, while ranking models roughly.
• The TerraBench and ThinkGeo correlations are computed at different levels and are not one finding replicated.
• What recurs across the two author clusters is that interventions on process, explicit step guidance in Earth-Bench and an oracle tool plan in Earth-Bench-Pro, improved trajectories more than answers for strong models (Feng et al., 2025; Lv et al., 2026).
• Earth-Agent-Pro implements step-level gates and a final answer check without evaluating them, so the library's 'no implemented verifier' should read 'no verifier evaluated as a predictor of correctness' (Lv et al., 2026).
• All four benchmarks score process as agreement with one reference trace, a different construct from the state-validity audit of Munir et al. (2026), and low agreement does not establish an invalid process.

How they bear on the open questions

On the library's open question, whether the correctness of an agent's answer can be certified from its execution trace, the verdict is that it remains open in substance. What the literature now settles is limited: alignment with a reference trace does not certify individual answers in the one benchmark that measured this at run level (Nguyen et al., 2026), and perfecting the tool sequence adds about three points for a strong backbone in the one framework that tried it (Lv et al., 2026). What it leaves is the core of the question. No source tests which intermediate-state checks, among coordinate system, grid alignment, time window, units, provenance and physical range, predict final correctness, or what they cost. No failure labeller has been validated against expert judgement; the one in TerraBench is not described. The relation between process and outcome inside a single model is unreported, and the oracle-plan result suggests it depends on capability, the plan being worth about eight points to a 9B model and three to GPT-5. These are an untested mechanism and unprobed conditions, not refinements of scale.

The sources bear on two neighbouring questions in the same domain. The library notes that nothing outside TerraBench tests whether numeric grounding, and not tool selection, is the bottleneck. Earth-Agent-Pro supplies that from a second author cluster: for the closed models and the fine-tuned 9B model, tool coverage of 88 to 91 percent sits beside parameter accuracy of 27 to 33 percent, GPT-5 reaches only 39.6 percent parameter accuracy even under the oracle plan, and the authors conclude that selecting relevant tools is easier than instantiating a faithful executable workflow (Lv et al., 2026). The ordering therefore holds in two benchmarks with different tolerance rules, annotator-set bands in one and a language-model judge in the other. The library's subquestions, whether the miss rate survives an independent audit of the tolerance bands and whether forcing simulator use helps, are untouched. On benchmark coverage, Earth-Bench-Pro adds runtime data acquisition and open-ended answers, and neither it nor ThinkGeo treats forecasting, nowcasting or hydrology as first-class tasks, so that deficit is unchanged for the atmosphere.

Across domains, the weather-agent benchmark the library holds scores answers only. Zephyrus (Varambally et al., 2025) reports that replacing its forecasting tool with ground truth raised correctness on medium tasks and did not improve the hard ones, which the authors read as a reasoning bottleneck. That is a second case, from an unrelated group and a different kind of task, in which perfecting an upstream component left the hardest answers unchanged, and it points the same way as the oracle-plan result. It also marks a gap in the library's own holdings: no weather or convection agent benchmark it holds reports a process score at all, so the process-outcome relation is unmeasured for the atmosphere. The answer-format finding also connects to a question the library holds in severe convection, that verification against imperfect labels cannot credit a model that outperforms them; Flora et al. (2021), for instance, write that false alarms arising from missing storm reports likely make their verification underestimate the skill of their machine-learning guidance. An answer key is a label too, and a multiple-choice format or an unaudited tolerance band is noise in it, which limits what any comparison of process and outcome can show.

• Whether an answer can be certified from its execution trace remains open in substance: no source tests which intermediate-state checks predict correctness or at what cost, and no failure labeller has been validated against experts.
• The relation between process and outcome appears to depend on capability, an oracle plan adding about three points for GPT-5 and eight for a base 9B model in Lv et al. (2026), and no benchmark reports the relation inside a single model.
• Earth-Agent-Pro gives the library's numeric-grounding question a second author cluster: tool coverage near 90 percent beside parameter accuracy near 30 percent, the ordering Nguyen et al. (2026) found under a different tolerance rule.
• None of the four benchmarks treats forecasting, nowcasting or hydrology as a first-class task, and the weather-agent benchmark the library holds scores answers only (Varambally et al., 2025), so the process-outcome relation is unmeasured for the atmosphere.

References

1. Feng, P., Lv, Z., Ye, J., Wang, X., Huo, X., Yu, J., Xu, W., Zhang, W., Bai, L., He, C., & Li, W. (2025). Earth-Agent: Unlocking the full landscape of Earth observation with agents. arXiv. https://doi.org/10.48550/arXiv.2509.23141 (Conference paper at ICLR 2026)
2. Flora, M. L., Potvin, C. K., Skinner, P. S., Handler, S., & McGovern, A. (2021). Using machine learning to generate storm-scale probabilistic guidance of severe weather hazards in the Warn-on-Forecast System. Monthly Weather Review. https://doi.org/10.1175/mwr-d-20-0194.1
3. Lv, Z., Dang, C., Feng, Y., Gong, Y., Wang, X., Ye, J., He, C., & Li, W. (2026). Earth-Agent-Pro: Towards real-world full-chain Earth observation with agents. arXiv. https://arxiv.org/abs/2609.12533
4. Munir, M. A., Sheikh, M. U., Shabbir, A., Khan, M. H., Khan, F., Zhu, X. X., Demir, B., & Khan, S. (2026). Agentic AI for remote sensing: Technical challenges and research directions. arXiv. https://arxiv.org/abs/2604.24919
5. Nguyen, D. T., Nguyen, T., Maani, F. A., Le, H. M., Sheikh, M. U., Saeed, N., Khan, M. H., & Khan, S. (2026). TerraBench: Can agents reason over heterogeneous Earth-system data? arXiv. https://doi.org/10.48550/arXiv.2606.13148
6. Ren, S., Xie, C., Jian, P., Ren, Z., Leng, C., & Zhang, J. (2025). Towards scientific intelligence: A survey of LLM-based scientific agents. arXiv. https://doi.org/10.48550/arXiv.2503.24047
7. Shabbir, A., Munir, M. A., Dudhane, A., Sheikh, M. U., Khan, M. H., Fraccaro, P., Moreno, J. B., Khan, F. S., & Khan, S. (2025). ThinkGeo: Evaluating tool-augmented agents for remote sensing tasks. arXiv. https://arxiv.org/abs/2505.23752
8. Varambally, S., Fisher, M., Thakker, J., Chen, Y., Xia, Z., Jafari, Y., Niu, R., Jain, M., Manivannan, V. V., Novack, Z., Han, L., Eranky, S., Rühling Cachay, S., Berg-Kirkpatrick, T., Watson-Parris, D., Ma, Y.-A., & Yu, R. (2025). Zephyrus: An agentic framework for weather science. arXiv. https://doi.org/10.48550/arXiv.2510.04017
9. Wang, J., Ma, Z., Li, Y., Zhang, S., Chen, C., Chen, K., & Le, X. (2024). GTA: A benchmark for general tool agents. Advances in Neural Information Processing Systems, 37, 75749–75790. https://arxiv.org/abs/2407.08713

Read into the library after publication:

• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks arXiv:2505.23752
• Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents arXiv:2509.23141
• Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents arXiv:2609.12533

Against the library

How this episode sits against the listener's own paper library. Coverage: partial.

What the sources add

The library holds this question at second hand. It has the Perspective that proposed evaluating physical models by their reconstructed causal dependencies (Runge, Bathiany, et al., 2019), a review that restates the first execution of the idea in a few paragraphs (Camps-Valls et al., 2023), and the method paper for the algorithm (Runge, Nowack, et al., 2019). The episode supplies the primary study. Nowack et al. (2020) reduced daily sea-level pressure to fifty Varimax-rotated components per season, with loadings estimated from the NCEP-NCAR reanalysis and applied unchanged to every model, averaged the series to three-day means, and ran PCMCI with lags to thirty days at a significance level of 10^-4, obtaining 400 to 500 links out of 24,500 possible. Networks were compared by an F1 score over the existence of links, modified to require the same sign and to tolerate a lag difference of two steps. In pre-industrial runs of twenty CMIP5 models, segments of one model were more alike than different models; HadGEM2-ES, HadGEM2-CC, ACCESS1-0 and ACCESS1-3 were recovered as one family, as were the IPSL, MPI and GFDL groups and two of three MIROC models, while MIROC5 was less similar to MIROC-ESM than the multi-model average, and surrogate networks from noise scored very low. In historical runs for 1948 to 2017 the scores against the reanalysis formed a continuum between about 0.56 and 0.66, with nearly the same ordering against ERA-Interim, and correlated with Taylor skill scores for land precipitation at about 0.7 globally and over North America, 0.6 over East Asia and 0.5 over Africa and South Asia, with no significant relation over Australia or Indonesia. The review's restatement omits three things that the primary text states: the regional exceptions, the dependence of regional correlations on the number of links retained, and the authors' reading that at strict thresholds the networks are dominated by links shorter than one week and so largely represent storm-track dynamics, with simpler storm-track metrics correlating with precipitation skill more weakly.

On projections, Nowack et al. (2020) fitted a Gaussian-process regression to the twenty models and found an approximately parabolic relation between the score and the change in land precipitation for 2050 to 2100 relative to 1860 to 1910 under RCP8.5, the best-matching models projecting 0.0 to 0.1 mm per day, with no relation for the precipitation skill score itself; they stated that the relation is constrained to the CMIP5 ensemble and should be verified in CMIP6. Debeire et al. (2025) did so with 23 CMIP6 models and 154 members for 1979 to 2014, daily data, sixty components, lags of one to twenty days, a significance level of 10^-5, and both ERA5 and NCEP-NCAR as references. Model rankings against the two references have a rank correlation of 0.91; models sharing the HadGEM lineage (ACCESS, UKESM, HadGEM3, KACE) again cluster; and a second-degree polynomial relation between score and projected change is significant at the five percent level in five of six combinations of scenario and reference (p = 0.06 in the sixth), while linear fits are not. A weighting scheme combining closeness to the reference with independence from other models leaves the weighted mean statistically unchanged and narrows the 5 to 95 percent range of 2081 to 2100 land-precipitation change by 10 to 16 percent and the 17 to 83 percent range by 16 to 41 percent with ERA5 as reference, the upper bounds moving down and the lower bounds showing no consistent change. The performance parameter was calibrated by a perfect-model test in which the usual requirement that 80 percent of pseudo-truths fall within the 10 to 90 percent band was lowered to 70 percent in order to favour performance over independence.

Two further sources bear on how such scores should be read. Karmouche et al. (2023), from the same research circle, applied PCMCI+ to four yearly indices (Atlantic multidecadal and Pacific decadal variability and the Pacific-North American and Pacific-South American patterns) for 1900 to 2014 in twelve CMIP6 large ensembles, by regime. They report that high pattern correlations do not ensure similar causal networks, UKESM1-0-LL being the example; that two members of IPSL-CM6A-LR gave the best match to the reanalysis network in one regime while three other members of the same model showed no similarity; and that the highest-scoring simulations came from the two largest ensembles, which they attribute in part to the number of realisations needed to sample internal variability resembling the observed one. Falkena et al. (2025), an Utrecht group with no authors in common with the fingerprint studies, took the opposite route: a hypothesised five-variable feedback loop of the North Atlantic subpolar gyre tested link by link with PCMCI+ and causal-effect estimation in the pre-industrial control runs of 32 CMIP6 models. The link from gyre strength to surface salinity is absent in 22 of the 32, the models that show a significant feedback from subsurface temperature through density to gyre strength do not agree on its sign, and only CESM2 has every link significant. They state that where common drivers are omitted, causal sufficiency is violated and the presence of links cannot be identified robustly, and they restrict the analysis to a hypothesis they judge causally sufficient.

• Nowack et al. (2020) show, for twenty CMIP5 models and sea-level pressure alone, that a lagged dependency network recovers shared model ancestry and that its edge-level match to the NCEP-NCAR reanalysis correlates at about 0.7 with global land-precipitation skill, with no relation over Australia or Indonesia.
• In the same study the late-century change in land precipitation under RCP8.5 is a concave function of that match, the best-matching models projecting 0.0 to 0.1 mm per day, and the authors confine the relation to the CMIP5 ensemble.
• Debeire et al. (2025) recover the concave relation in 23 CMIP6 models under three scenarios and two reanalyses, and a weighting built on it narrows the 5 to 95 percent range of end-of-century land-precipitation change by 10 to 16 percent while leaving the mean unchanged.
• Karmouche et al. (2023) find, for four yearly indices in twelve CMIP6 large ensembles, that members of one model range from the best match to no match with the reanalysis network, so on short records the score depends on which realisation is drawn.
• Falkena et al. (2025) test a five-variable subpolar-gyre feedback loop link by link in 32 CMIP6 control runs and find one link absent in 22 models and every link significant only in CESM2, a statement of where models fail that a network-level score does not give.

Where they differ from the library

The library's current reading is that fingerprints discriminate models beyond descriptive statistics and that the step from fingerprint similarity to projection reliability is supported by one association and otherwise untested. The primary sources narrow both halves. The association now exists twice, in CMIP5 and in CMIP6, with the same functional form, but the two studies share three authors and no replication from outside the group was found. More importantly, the form of the association qualifies the premise as Runge, Bathiany, et al. (2019) wrote it, that models causally similar to observations will yield more reliable projections. What Nowack et al. (2020) and Debeire et al. (2025) find is not a monotonic link between similarity and a projected quantity but a concave one in which low-scoring models lie at both extremes. Such a relation constrains spread and not the central estimate, which is what Debeire et al. (2025) report: an unchanged mean and a narrower range whose size depends on a stated calibration choice. Reliability itself is not observed in either study. The perfect-model test examines whether the weights are calibrated on the assumption that reality behaves like a member of the ensemble, and errors shared by all models are outside its reach.

The wording the library inherited from Camps-Valls et al. (2023), that more realistic fingerprints 'affect' projected changes in land precipitation and that the networks constitute characteristic causal fingerprints, is stronger than the primary texts. Nowack et al. (2020) describe an emergent relationship across twenty models and call the F1 score a widely used but necessarily imperfect metric for an imbalanced comparison. Debeire et al. (2025) go further and disclaim the causal reading: causal sufficiency is not met because the analysis is restricted to sea-level pressure, and they consider this less critical because the aim is to derive a metric of the data and not to determine the causal relevance of each link. Read with that statement, a fingerprint is a lag-resolved conditional-dependence signature. Falkena et al. (2025) differ on grounds of purpose and not of method: interpreting individual links requires sufficiency, scoring whole networks for likeness does not, and the two positions are compatible. The consequence is that a fingerprint score can indicate that a model's dynamics are unlike the reference and cannot indicate which mechanism is at fault. A further observation is ours and not the authors': Runge, Bathiany, et al. (2019) note that the absence of a link is the more robust inference because it does not require causal sufficiency, whereas an F1 score is built from true positives, false positives and false negatives and therefore gives no credit for agreement on absences, which at link densities of a few percent make up most of each network.

The two fingerprint studies also differ from each other in a way the library's summary cannot show. Nowack et al. (2020) used three-day means and did not use lag-zero links, whose direction cannot be resolved; Debeire et al. (2025) used daily data with a minimum lag of one day and found 99.9 percent of dependencies within ten days, the most significant ones predominantly between neighbouring components. The score ranges differ between the studies, and the component sets, reference periods and significance levels differ as well, so the CMIP5 and CMIP6 fingerprints are related constructions and not one measurement repeated. The sensitivity tests of Debeire et al. (2025) varied the significance level, the number of components and the maximum lag, with rank correlations of 0.95 to 0.98 against the baseline ranking, but not the averaging window. Karmouche et al. (2023) add a condition under which scores become unstable, namely short yearly records split by regime, where they report that finite-sample errors produce both false and missing links and that a physical explanation cannot be attached to every detected link.

• The library's record that the projection premise rests on one association should now read: one concave association, reproduced once in the successor ensemble by an overlapping author group (Nowack et al., 2020; Debeire et al., 2025).
• That association is not the monotonic link implied by the premise of Runge, Bathiany, et al. (2019) but a constraint on spread: in Debeire et al. (2025) the weighted mean does not move and only the upper bound comes down consistently.
• The size of the narrowing depends on a stated calibration choice, the perfect-model coverage criterion having been lowered from 80 to 70 percent (Debeire et al., 2025).
• Debeire et al. (2025) state that causal sufficiency is not met and that the network serves as a metric of the data, so the causal fingerprint described by Camps-Valls et al. (2023) is a conditional-dependence signature and not a set of identified mechanisms.
• The CMIP5 and CMIP6 fingerprints are different constructions, three-day means without same-step links against daily data with a one-day minimum lag, and no study has computed both on one ensemble.

How they bear on the open questions

On the library's open question of how physical models should be evaluated by their causal structure, the verdict is that it remains open in substance, and the arrival of the primary papers changes the evidence base and not the status. Four parts of the question can be separated. The comparison statistic is still an edge-level count of shared links; Debeire et al. (2025) list measures that discriminate better between networks or that use causal effects as future work, and none has been tested. The premise that similarity implies reliable projections has a concave cross-model relation in two ensembles from overlapping authors behind it, no replication by an independent group, no test with one model per family, and no observable against which reliability could be checked. The handling of deterministic relations among model output variables, which Runge, Bathiany, et al. (2019) flagged, has not been encountered because both studies use a single variable. Transfer to convection-permitting or learned weather models has not been attempted in any source found for this episode. None of these is a technical refinement: they are an untested premise, an unresolved difference over what a fingerprint is, and unprobed conditions.

The sources bear on two neighbouring open questions in the same domain. On time aggregation, the library records that averaging can make lagged links appear contemporaneous and that their direction is then not identifiable from time order. The fingerprint studies are an unused test of how much this matters: the three-day networks of Nowack et al. (2020) set aside same-step links, and the daily networks of Debeire et al. (2025) place nearly all links within the range that such averaging compresses. Computing both on one ensemble would show whether rankings and the projection relation survive the change of cadence; until that is done the question is untouched. On the extraction of causal variables, both studies derive Varimax components from the reanalysis and project every model onto them. Nowack et al. (2020) report that results are insensitive to which components are retained, but the design cannot separate a model with wrong dynamics between correctly placed centres of action from one with realistic dynamics between displaced centres. The library's statement that the causal method cannot validate the upstream choice of variables applies here without change. The use of a reanalysis as the reference also sits within the library's question about the lack of causal ground truth: agreement of rankings between NCEP-NCAR and ERA5 shows insensitivity to the choice of product, not independence from the observations and model physics that both products share.

Across domains, two connections follow. First, mechanism-first causal evaluation now has author-disjoint instances: Sun et al. (2025), already in the library, scored sixteen CMIP6 models against information-flow patterns of soil moisture-precipitation coupling and found that most fail to reproduce them, and Falkena et al. (2025) did the equivalent for a subpolar-gyre feedback loop. Fingerprint-style evaluation, by contrast, rests on one group (Nowack et al., 2020; Karmouche et al., 2023; Debeire et al., 2025). The two approaches answer different questions, likeness of the whole against presence of a named mechanism, and the library's gap is better served by holding both. Second, for machine-learning weather forecasting the fingerprint is attractive because it needs only model output, and it would test temporal coherence that frame-wise scores do not. The library already records that emulators verified against the analyses they were trained on cannot reveal inherited bias, StormCast against HRRR analyses and Aardvark against ERA5 being the recorded cases (Pathak et al., 2024; Allen et al., 2025). A fingerprint scored against the training reanalysis would fall under the same circularity, so the protocol would need held-out periods or independent observations before its scores could be compared with those of physics-based models.

• How to evaluate models by their causal structure stays open in substance: the comparison statistic is still an edge-level count, the projection premise has no independent replication, and deterministic model diagnostics and kilometre-scale or learned models have not been tried.
• The two fingerprint studies are an unused test of the library's time-aggregation question, since Debeire et al. (2025) find 99.9 percent of daily links within ten days, the range that the three-day averaging of Nowack et al. (2020) compresses.
• Both fingerprint studies project every model onto patterns derived from the reanalysis, so the score cannot separate wrong dynamics between correctly placed regions from realistic dynamics between displaced ones, which leaves the variable-extraction question where Runge, Bathiany, et al. (2019) put it.
• Mechanism-first causal evaluation has author-disjoint instances, Sun et al. (2025) on soil moisture and precipitation and Falkena et al. (2025) on the subpolar gyre, whereas fingerprint evaluation rests on one group.
• A fingerprint of an emulator scored against its training reanalysis would be in-sample, the same verification circularity the library records for StormCast and Aardvark (Pathak et al., 2024; Allen et al., 2025).

References

1. Allen, A., Markou, S., Tebbutt, W., Requeima, J., Bruinsma, W. P., Andersson, T. R., Herzog, M., Lane, N. D., Chantry, M., Hosking, J. S., & Turner, R. E. (2025). End-to-end data-driven weather prediction. Nature. https://doi.org/10.1038/s41586-025-08897-0
2. Camps-Valls, G., Gerhardus, A., Ninad, U., Varando, G., Martius, G., Balaguer-Ballester, E., Vinuesa, R., Diaz, E., Zanna, L., & Runge, J. (2023). Discovering causal relations and equations from data. Physics Reports, 1044, 1–68. https://doi.org/10.1016/j.physrep.2023.10.005
3. Debeire, K., Bock, L., Nowack, P., Runge, J., & Eyring, V. (2025). Constraining uncertainty in projected precipitation over land with causal discovery. Earth System Dynamics, 16, 607–630. https://doi.org/10.5194/esd-16-607-2025
4. Falkena, S. K. J., Dijkstra, H. A., & von der Heydt, A. S. (2025). Causal mechanisms of subpolar gyre variability in CMIP6 models (Version 3). arXiv. https://arxiv.org/abs/2408.16541 (Published in Earth System Dynamics, 16, 1833–1844. https://doi.org/10.5194/esd-16-1833-2025)
5. Karmouche, S., Galytska, E., Runge, J., Meehl, G. A., Phillips, A. S., Weigel, K., & Eyring, V. (2023). Regime-oriented causal model evaluation of Atlantic–Pacific teleconnections in CMIP6. Earth System Dynamics, 14, 309–344. https://doi.org/10.5194/esd-14-309-2023
6. Nowack, P., Runge, J., Eyring, V., & Haigh, J. D. (2020). Causal networks for climate model evaluation and constrained projections. Nature Communications, 11, 1415. https://doi.org/10.1038/s41467-020-15195-y
7. Pathak, J., Cohen, Y., Garg, P., Harrington, P., Brenowitz, N., Durran, D., Mardani, M., Vahdat, A., Xu, S., Kashinath, K., & Pritchard, M. (2024). Kilometer-scale convection allowing model emulation using generative diffusion modeling. arXiv. https://doi.org/10.48550/arXiv.2408.10958
8. Runge, J., Bathiany, S., Bollt, E., Camps-Valls, G., Coumou, D., Deyle, E., Glymour, C., Kretschmer, M., Mahecha, M. D., Muñoz-Marí, J., van Nes, E. H., Peters, J., Quax, R., Reichstein, M., Scheffer, M., Schölkopf, B., Spirtes, P., Sugihara, G., Sun, J., . . . Zscheischler, J. (2019). Inferring causation from time series in Earth system sciences. Nature Communications, 10, 2553. https://doi.org/10.1038/s41467-019-10105-3
9. Runge, J., Nowack, P., Kretschmer, M., Flaxman, S., & Sejdinovic, D. (2019). Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances. https://doi.org/10.1126/sciadv.aau4996
10. Sun, J., Yang, K., He, X., Wang, G., Wang, Y., Yu, Y., & Lu, H. (2025). Causal pathways underlying global soil moisture–precipitation coupling. Nature Communications, 16, 8935. https://doi.org/10.1038/s41467-025-63999-7

Read into the library after publication:

• Causal networks for climate model evaluation and constrained projections 10.1038/s41467-020-15195-y
• Constraining uncertainty in projected precipitation over land with causal discovery 10.5194/esd-16-607-2025
• Regime-oriented causal model evaluation of Atlantic-Pacific teleconnections in CMIP6 10.5194/esd-14-309-2023
• Causal mechanisms of subpolar gyre variability in CMIP6 models 10.5194/esd-16-1833-2025

Against the library

How this episode sits against the listener's own paper library. Coverage: partial.

What the sources add

The library holds the land-atmosphere domain from the model-diagnostic side and lacks the observational canon on the sign of the soil moisture-precipitation feedback, which the episode's sources supply. Taylor et al. (2012) established from satellite soil moisture and rainfall over six continents that afternoon rain falls preferentially over soils drier than their surroundings at the 50 to 100 km scale, with no evidence of a positive spatial feedback at that scale and a dominant positive feedback in six global weather and climate models. Taylor et al. (2011) had already shown in the Sahel that the probability of convective initiation doubles over strong soil-moisture gradients at 10 to 40 km scales, with 37 percent of initiations over the steepest quartile of gradients. Guillod et al. (2015) reconciled the spatial and temporal results within one framework: afternoon rain occurs on wet, heterogeneous days and lands on the drier patch, so correlations of opposite sign coexist in the same region and dataset, and the positive temporal coupling does not necessarily imply causation. Tuttle and Salvucci (2016) added a Granger-causal estimate over the contiguous United States in which soil moisture anomalies significantly alter rainfall probability over 38 percent of the area, by a median factor of 13 percent, positive in the west and negative in the east. None of these is in the library, whose only observational statement of the feedback is the positive loop described in a figure caption of Rahmati et al. (2024), and whose only dry-patch initiation result is the single reliability-flagged preprint of Engelbrecht et al. (2021).

The sources also supply the model-side explanation that the library does not carry. Hohenegger et al. (2009) found over the Alps in July 2006 that the same model sustains a strong positive feedback at 25 km with parameterised convection and a predominantly negative one at 2.2 km with explicit convection. Taylor et al. (2013) isolated the cause: convection-permitting runs at 4 and 12 km reproduce the observed Sahelian relation between soil moisture and triggering, switching the convection scheme on at the same 12 km flips the sign from negative to positive, and nine of ten regional climate models at 50 km share the positive sign. Moon et al. (2019) showed nine CMIP5 models failing to reproduce the observed spatial preference while agreeing better on the temporal and heterogeneity diagnostics. Lee and Hohenegger (2024) then revisited the feedback in a global coupled simulation at 5 km grid spacing and found it weaker and more negative than in coarse-resolution climate models, with the stated implication that coarse models may overestimate future changes in droughts and heatwaves, and Lee et al. (2026) found that two global storm-resolving models disagree with each other, ICON placing afternoon rain over drier-than-surrounding soil while IFS shows only a temporal preference for drier days and MPI-ESM keeps the preference for wetter surroundings. The library holds no convection-permitting or storm-resolving coupling study, and Findell et al. (2024) name comparison with global storm-resolving efforts as future work.

Three sources postdate the library's July 2026 snapshot of the domain or lie outside it. Sun et al. (2025) applied Liang-Kleeman information flow to satellite surface soil moisture for 2003 to 2020 and found strong coupling over about 16 percent of global land in eight hotspots, separating a sensible-heat pathway that deepens the boundary layer from an evapotranspiration pathway that lowers the lifting condensation level and dominates in the African hotspots, with most CMIP6 models reproducing little of the pattern. Maybee et al. (2025) ran 78 Unified Model experiments initialised from scale-filtered Sahelian soil moisture and found that homogenising all sub-synoptic variability reduces peak mesoscale convective system counts by 23 percent, while retaining small-scale variability preserves primary initiation rates. Klein and Taylor (2020) showed from satellite observations that dry soils at scales of 200 km and above intensify mature mesoscale convective systems in the Sahel long after initiation, with convective cores favoured downstream of dry patches.

• Taylor and colleagues (2012) put afternoon rain over soil drier than its surroundings across six continents at scales of 50 to 100 km, an observational statement of the sign that the library held from no source of its own.
• Guillod and colleagues (2015) reconcile the spatial and the temporal findings in a single framework: rain falls on wet, heterogeneous days and lands on the drier patch within them.
• Taylor and colleagues (2013) locate the model error in the scheme rather than the resolution: at a fixed 12 km grid in the Sahel, switching the convection scheme on flips the feedback from negative to positive.
• Lee and Hohenegger (2024) at 5 km globally and Lee and colleagues (2026) across ICON, IFS and MPI-ESM both weaken the coupling and push it negative, and they disagree with each other about which preference survives.
• Taylor and colleagues (2011) and Klein and Taylor (2020) place the effect at different scales and stages: gradients of 10 to 40 km trigger new storms, dry patches of 200 km and above intensify mature ones.

Where they differ from the library

Read against the library, the sources qualify its framing of the feedback rather than any specific measurement. The positive loop that Rahmati et al. (2024) describe, in which wet soil sustains above-average evaporation and triggers further precipitation, is the temporal and regional-scale answer, and Taylor et al. (2012) and Guillod et al. (2015) show that at the patch scale the sign is negative and that the positive temporal association is exposed to persistence of the weather itself. Yin and Porporato (2024) are qualified on grounds of model class: their finding that the feedback is positive over the central Great Plains and negative over India rests on one EC-Earth3 experiment with parameterised convection, the class that Taylor et al. (2013) and Moon et al. (2019) show biased toward a positive sign, so the positive Great Plains sign cannot be separated from that bias, whereas the negative sign in India runs against it and is the more robust of the two directions. Their central result, that the 2004 coupling index is dominated by soil-moisture variance, is untouched by this and is supported by Sun et al. (2025), who find coupling strength depending on soil moisture variability in observations as well.

Gao et al. (2025) are narrowed on the question being asked. Their finding that morning soil moisture explains up to 40 percent of summertime afternoon-precipitation variance over the Great Plains, and that first-order effects boost rainfall by up to 8 mm under wet conditions on rainy days, concerns the amount of rain on days when it rains, at 16 to 80 km grid cells in the CONUS404 reanalysis. That is Guillod et al.'s temporal question, not the spatial one, and the same reanalysis, being a 4 km convection-permitting WRF product, cannot be assumed to share the parameterisation bias, so the positive amount effect and the negative location preference are not in conflict. The authors' own statement that HDMR pathways must be validated by process-based modelling applies with more force once Tuttle and Salvucci (2016) and Guillod et al. (2015) are read alongside it, because both show that a positive same-day association survives confounding by precipitation persistence unless it is explicitly removed.

Engelbrecht et al. (2021) are strengthened rather than contradicted. Their modelled result that convective initiation over South Africa favours dry to moderate soils and the peripheries of wet patches, where surface convergence and sensible heat flux are elevated, is the same mechanism Taylor et al. (2011) observed and Taylor et al. (2013) reproduced at 4 km, and their 4.4 km explicit-convection configuration is in the class those studies show to capture the observed sign. The library's caution about that preprint, recorded because it is uncited and unpublished, should be read as a caution about the specific centroid-error and timing numbers, not about the direction of the soil moisture-initiation relation. Findell et al. (2024) are extended rather than qualified: their finding that the choice of averaging window changes diagnosed coupling strength by about 30 percent is a temporal instance of a sensitivity that the episode's sources show in space as well, since Taylor et al. (2012) at 50 to 100 km, Taylor et al. (2011) at 10 to 40 km and Klein and Taylor (2020) at 200 km and above find effects of different sign and kind depending on the scale of comparison.

• The positive loop of Rahmati and colleagues (2024) survives as the temporal, regional-scale answer, while the negative sign of Taylor and colleagues (2012) and Guillod and colleagues (2015) belongs to the within-day comparison between neighbouring patches.
• Taylor and colleagues (2013) and Moon and colleagues (2019) both show parameterised convection biased toward a positive sign, so the positive Great Plains result of Yin and Porporato (2024) cannot be separated from that bias while their negative Indian result runs against it.
• The finding of Yin and Porporato (2024) that the 2004 coupling index is dominated by soil-moisture variance is untouched by that bias and is supported in observations by Sun and colleagues (2025).
• Engelbrecht and colleagues (2021), which the library flagged as an uncited preprint, is strengthened in the direction it reports even where its centroid and timing numbers stay uncertain.
• Findell and colleagues (2024) move diagnosed coupling strength by about 30 percent with the averaging window alone, so a coupling number quoted without both its window and its comparison scale cannot be set against another.

How they bear on the open questions

The sources bear directly on the library's open question about sub-daily archival of model output for coupling diagnostics. Lee and Hohenegger (2024) and Lee et al. (2026) are the storm-resolving comparisons that Findell et al. (2024) flagged as future work, and they show that resolving convection changes both the sign and the strength of the diagnosed feedback, which raises the cost of archives that cannot resolve the afternoon. On the sub-question of whether three-hourly output would resolve the 30 percent disagreement, the observational designs of Taylor et al. (2012) and Guillod et al. (2015) need only a morning soil-moisture state and an afternoon precipitation occurrence, so three-hourly archival would allow those diagnostics to be computed from model output for the first time; but the disagreement between ICON and IFS in Lee et al. (2026) shows that output frequency is not the only barrier, since two models archived identically still give different answers, and the mesoscale breeze mechanism of Taylor et al. (2011) additionally needs spatial resolution of 10 to 40 km patches that no archived grid-cell mean can carry.

In the severe-convection domain the sources supply a candidate origin for boundary-layer convergence lines, which the library records as an open question. Taylor et al. (2011) and Maybee et al. (2025) show that soil-moisture gradients at tens of kilometres generate surface convergence that doubles initiation probability and accounts for a quarter of peak storm-system counts, which is a land-surface source for the convergence lines that the radar-based initiation nowcasting of Wilson and Mueller (1993) detects and tracks without attributing. Klein and Taylor (2020) connect dry patches to the intensification and propagation of mature mesoscale convective systems through convergence, instability and low-level shear, which touches the library's questions about storm propagation and about the sensitivity of initiation to shear and stability. Crook (1996) showed that initiation switches on or off with 1 degree Celsius or 1 g per kg in the boundary layer, and the soil-moisture partitioning of surface energy is one of the processes that sets those values, so the land-atmosphere results give the sensitivity result a physical driver at the scale of days. Engelbrecht et al. (2021) remain the only card connecting the two domains at storm scale.

In the time-series domain the sources are three applications of the library's causal formalisms to one physical question, and they illustrate its open question of distinguishing direct drivers from shared causes and persistence. Tuttle and Salvucci (2016) apply Granger causality after removing daily persistence and seasonal and interannual variability, which is precisely the conditioning that Runge et al. (2019) describe as necessary and that bivariate Granger analyses omit. Sun et al. (2025) choose Liang-Kleeman information flow over convergent cross-mapping because the latter misreads strong unidirectional coupling as bidirectional, and the library's reading of Liang (2016) as a one-sided nil-causality test with an open converse case applies to their inferred pathways. Gao et al. (2025) decompose variance rather than test causality and say so. Guillod et al. (2015) supply the concrete confounder, precipitation persistence producing a positive same-day association, that Schreiber (2000) describes abstractly as two signals responding to a common trigger. The sign question in land-atmosphere coupling is therefore a worked example of the library's causal-versus-correlational gap, with the added feature that the confounder is known and can be conditioned out.

• The comparison against global storm-resolving models that Findell and colleagues flagged as future work has now been run, and it widens the question rather than settling it: explicit convection changes the sign of the diagnosed feedback, not only its strength.
• Three-hourly archival would make the spatial and temporal diagnostics computable from model output for the first time, but ICON and IFS disagree in Lee and colleagues (2026) while archived identically, so output frequency is not the only barrier.
• Taylor and colleagues (2011) supply a land-surface origin for the convergence lines that the radar nowcasting of Wilson and Mueller (1993) detects and tracks without attributing.
• Klein and Taylor (2020) carry the same gradients past initiation into the intensification and propagation of mature systems, which touches the library's questions on storm propagation and on sensitivity to shear and stability.
• The sign question is an unusually tractable instance of the library's causal-versus-correlational gap, because Guillod and colleagues (2015) name the confounder, precipitation persistence, that Tuttle and Salvucci (2016) then condition out.

References

1. Crook, N. A. (1996). Sensitivity of moist convection forced by boundary layer processes to low-level thermodynamic fields. Monthly Weather Review, 124, 1767–1785. https://doi.org/10.1175/1520-0493(1996)124<1767:SOMCFB>2.0.CO;2
2. Engelbrecht, E. H., Landman, W. A., & Landman, S. (2021). The impact of high-resolution soil moisture states on short-term numerical weather prediction of convective initiation over South Africa. arXiv. https://arxiv.org/abs/2102.08523
3. Findell, K. L., & Eltahir, E. A. B. (2003). Atmospheric controls on soil moisture–boundary layer interactions. Part I: Framework development. Journal of Hydrometeorology, 4(3), 552–569. https://doi.org/10.1175/1525-7541(2003)004<0552:ACOSML>2.0.CO;2
4. Findell, K. L., Yin, Z., Seo, E., Dirmeyer, P. A., Arnold, N. P., Chaney, N., Fowler, M. D., Huang, M., Lawrence, D. M., Ma, P.-L., & Santanello, J. A., Jr. (2024). Accurate assessment of land–atmosphere coupling in climate models requires high-frequency data output. Geoscientific Model Development. https://doi.org/10.5194/gmd-17-1869-2024
5. Gao, Y., Li, R., Foufoula-Georgiou, E., & Vrugt, J. A. (2025). Functional data decomposition reveals unexpectedly strong soil moisture-precipitation coupling over the Great Plains. arXiv. https://arxiv.org/abs/2506.13939
6. Guillod, B. P., Orlowsky, B., Miralles, D. G., Teuling, A. J., & Seneviratne, S. I. (2015). Reconciling spatial and temporal soil moisture effects on afternoon rainfall. Nature Communications, 6, 6443. https://doi.org/10.1038/ncomms7443
7. Hohenegger, C., Brockhaus, P., Bretherton, C. S., & Schär, C. (2009). The soil moisture–precipitation feedback in simulations with explicit and parameterized convection. Journal of Climate, 22(19), 5003–5020. https://doi.org/10.1175/2009JCLI2604.1
8. Klein, C., & Taylor, C. M. (2020). Dry soils can intensify mesoscale convective systems. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2007998117
9. Koster, R. D., Dirmeyer, P. A., Guo, Z., Bonan, G., Chan, E., Cox, P., Gordon, C. T., Kanae, S., Kowalczyk, E., Lawrence, D., Liu, P., Lu, C.-H., Malyshev, S., McAvaney, B., Mitchell, K., Mocko, D., Oki, T., Oleson, K., Pitman, A., Sud, Y. C., Taylor, C. M., Verseghy, D., Vasic, R., Xue, Y., & Yamada, T. (2004). Regions of strong coupling between soil moisture and precipitation. Science, 305(5687), 1138–1140. https://doi.org/10.1126/science.1100217
10. Lee, J., & Hohenegger, C. (2024). Weaker land–atmosphere coupling in global storm-resolving simulation. Proceedings of the National Academy of Sciences, 121(12), e2314265121. https://doi.org/10.1073/pnas.2314265121
11. Lee, J., Warnau, S., Lim, K.-S. S., Benedict, I., & van Heerwaarden, C. (2026). Spatial and temporal preferences for afternoon precipitation over morning soil moisture in global storm-resolving simulations. Geoscience Letters, 13, 13. https://doi.org/10.1186/s40562-026-00461-6
12. Liang, X. S. (2016). Information flow and causality as rigorous notions ab initio. Physical Review E, 94, 052201. https://doi.org/10.1103/PhysRevE.94.052201
13. Maybee, B., Klein, C., Taylor, C. M., Burns, H., & Marsham, J. H. (2025). Homogeneous soil moisture fields suppress Sahelian MCS frequency. arXiv. https://arxiv.org/abs/2509.12118
14. Moon, H., Guillod, B. P., Gudmundsson, L., & Seneviratne, S. I. (2019). Soil moisture effects on afternoon precipitation occurrence in current climate models. Geophysical Research Letters. https://doi.org/10.1029/2018GL080879
15. Rahmati, M., Amelung, W., Brogi, C., Dari, J., Flammini, A., Bogena, H., Brocca, L., Chen, H., Groh, J., Koster, R. D., McColl, K. A., Montzka, C., Moradi, S., Rahi, A., Sharghi S., F., & Vereecken, H. (2024). Soil moisture memory: State-of-the-art and the way forward. Reviews of Geophysics. https://doi.org/10.1029/2023RG000828
16. Runge, J., Nowack, P., Kretschmer, M., Flaxman, S., & Sejdinovic, D. (2019). Detecting and quantifying causal associations in large nonlinear time series datasets. Science Advances. https://doi.org/10.1126/sciadv.aau4996
17. Schreiber, T. (2000). Measuring information transfer. Physical Review Letters, 85, 461–464. https://doi.org/10.1103/PhysRevLett.85.461
18. Sun, J., Yang, K., He, X., Wang, G., Wang, Y., Yu, Y., & Lu, H. (2025). Causal pathways underlying global soil moisture–precipitation coupling. Nature Communications. https://doi.org/10.1038/s41467-025-63999-7
19. Taylor, C. M., Birch, C. E., Parker, D. J., Dixon, N., Guichard, F., Nikulin, G., & Lister, G. M. S. (2013). Modeling soil moisture-precipitation feedback in the Sahel: Importance of spatial scale versus convective parameterization. Geophysical Research Letters, 40. https://doi.org/10.1002/2013GL058511
20. Taylor, C. M., de Jeu, R. A. M., Guichard, F., Harris, P. P., & Dorigo, W. A. (2012). Afternoon rain more likely over drier soils. Nature, 489, 423–426. https://doi.org/10.1038/nature11377
21. Taylor, C. M., Gounou, A., Guichard, F., Harris, P. P., Ellis, R. J., Couvreux, F., & De Kauwe, M. (2011). Frequency of Sahelian storm initiation enhanced over mesoscale soil-moisture patterns. Nature Geoscience, 4, 430–433. https://doi.org/10.1038/ngeo1173
22. Tuttle, S., & Salvucci, G. (2016). Empirical evidence of contrasting soil moisture–precipitation feedbacks across the United States. Science, 352(6287), 825–828. https://doi.org/10.1126/science.aaa7185
23. Wilson, J. W., & Mueller, C. K. (1993). Nowcasts of thunderstorm initiation and evolution. Weather and Forecasting. https://doi.org/10.1175/1520-0434(1993)008<0113:NOTIAE>2.0.CO;2
24. Yin, J., & Porporato, A. (2024). Coupling hotspots: Distinguishing between positive and negative land-atmosphere interaction. arXiv. https://arxiv.org/abs/2412.03049

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

Both of the episode's new sources are now library holdings, read in on 15 September 2026, and both entered at low weight as preprints under the rule that frozen evidence cannot close or reverse an open question. Antonio (2026) is attached to four of them: it supports the library's statements of the deterministic-blur problem, of what a probabilistic verification standard would have to be, and of how much credit a score should give a near miss, and it partially addresses the question of whether the newer spatial scores can themselves be hedged. On that last one the library records it as the first analytic answer to the question it was minted from, restricted to the neighbourhood family, with no demonstration on real forecasts and no object-based or field-deformation score analysed, and leaves the question open. Park et al. (2026) is attached to seven, among them the deterministic-blur problem, the attribution of ensemble under-dispersion, heavy-rain skill at long lead, heavy-rain verification and single-region generalisation, and moves none of them either. What the episode contributed to the library is therefore two papers that corroborate without settling, which is what its own coverage assessment implied.

It contributed no framing, which is the correction a second reading makes to the first. The episode's organising claim, that the blur is the error-minimising response to inherent uncertainty rather than a defect to engineer away, is the library's own statement of the question it was written against, and the mechanism behind it is held from three directions already: the conditional-mean collapse under a multimodal distribution (Prudden et al., 2020), its unavoidability for any statistical model minimising mean squared error on a highly unpredictable variable (Foresti et al., 2019), and the same-architecture loss ablation in which the mean-squared-error model washes out detail while style and adversarial losses keep radar-like texture (Veillette et al., 2020). Two details the episode claimed to have sharpened by reading the primary texts, the critical success index advantage over PySTEPS holding for all three deep-learning systems including the 32 km U-Net in Ravuri et al. (2021), and the importance-sampled extreme subsets behind the posterior first-choice rates in Zhang et al. (2023), are both recorded on the cards for those papers. The honest statement of what this episode added, now that its own sources can be read in full, is two preprints and a set of corrections to its own reading.

• Antonio (2026) and Park and colleagues (2026) are now library cards, entered as low-weight preprints that by the library's own rule cannot close or reverse a question.
• Between them Antonio (2026) and Park and colleagues (2026) touch the library's questions on deterministic blur, spatial-score hedging, near-miss credit, ensemble under-dispersion, heavy-rain skill at long lead and single-region generalisation, and move none of them.
• The framing the episode was built on, blur as the error-minimising response to uncertainty, is the library's own statement of the question rather than something the episode supplied.
• The two details it claimed to have sharpened, the 32 km effective resolution of the U-Net in Ravuri and colleagues (2021) and the importance-sampled extreme subsets behind the percentages in Zhang and colleagues (2023), are already on those papers' cards.
• Read a second time, what the episode added is two preprints and a set of corrections to its own reading.

Where they differ from the library

Reading both new sources in full corrects the episode where it worked from abstracts. On the verification side the attribution is the wrong way round: it is the fractions skill score, not the Brier divergence skill score, that can be improved by over-predicting the mean neighbourhood frequency, which is the specific reason Antonio (2026) advocates percentile thresholds, and the Brier divergence skill score is not improved by over-prediction at all. On the smoothing side the episode's ranking was right and its magnitude was not stated: both scores reward under-predicting the neighbourhood standard deviation, the fractions skill score less so, and the gap is about tenfold at a neighbourhood correlation of 0.5, with the fractions skill score maximised at no smoothing at all in the small-neighbourhood limit; percentile thresholds remove the frequency hedge only with reflective padding, and never remove the smoothing one. On the ensemble side, the 38 to 47 percent recovery of missed heavy-rain pixels belongs to two high-impact 2023 events on Korean radar, not to MeteoNet, which is a separately trained replication rather than a transfer of weights; over the full 2023 test period the recovery rate is lower, because the period is dominated by regimes in which the deterministic parent already detects almost nothing and there is little to recover; and 3.4 seconds is one member, with a thirty-member ensemble taking about a hundred seconds on one GPU when the members are drawn in sequence.

The episode's qualifications of what the library holds survive the second reading unchanged. The library's statement of the blur problem lists the skill advantage of the deep generative model over PySTEPS beside its spectral fidelity, which invites the reading that the categorical score credited sharpness; since the blurred U-Net also beat PySTEPS on that score, the advantage is evidence of placement skill rather than of sharpness, which is what Ravuri et al. (2021) conclude when they write that standard verification metrics and expert judgements are not mutually indicative of value. The RainNet headline that the network significantly outperforms its benchmarks at all lead times on mean absolute error and the critical success index (Ayzel et al., 2020) has to be read together with the same paper's finding that it entirely fails to predict exceedance of 15 mm per hour beyond the first minutes, and with Antonio (2026): the scores on which a smoothed model wins are the scores now shown to reward smoothing, so the headline is not evidence of value at the intensities for which warnings are issued. The uncertainty quantification that Leinonen et al. (2023) describe as accurate is narrowed by Bonte et al. (2026), whose fifty-member ensembles over twenty Belgian events were slightly under-dispersive above 5 km and could not localise the ensemble-mean error, surrogate ensembles with matched distributions and spectra but randomised spatial phase scoring almost identically.

The episode's own claims are qualified in turn, and one of the items it left open is now closed. Its closing statement that nobody has shown skill below about 10 km beyond about an hour, and that there may be none to show, is in the library's terms a measurement limit rather than a demonstrated absence: Germann and Zawadzki (2002) state that nothing can be said about the lifetime of scales below 10 km wavelength, and Surcel et al. (2015) that nothing about predictability below 12 km can be inferred because of uncertainty in the observed fields. The independent test of NowcastNet (Das et al., 2024) is a real win on categorical scores, but NowcastNet was initialised from and verified against the same radar mosaic while HRRR was an independent model, and 62 to 97 percent of its predicted rain pixels were false alarms. The open item was whether the thirty members of Park et al. (2026) carry any spread information, which the abstract did not say. The full paper does: the Brier skill score is positive from ten members and reliability improves as members are added, while the rank histogram is close to uniform at ten minutes and develops a right-hand excess reaching 2.3 times the uniform rate by sixty minutes, far more strongly on the heavy-rain subset, so the ensemble is under-dispersive where it matters most. The authors leave the cause among the residual formulation, the frozen backbone and the sampler, and the library records the result as a third author-disjoint case of a noise-only ensemble coming out under-dispersive without settling the attribution.

• Antonio (2026) shows it is the fractions skill score, not the Brier divergence skill score, that can be improved by over-predicting mean frequency, which is the whole reason for using percentile thresholds.
• Antonio (2026) also shows both neighbourhood scores rewarding under-predicted variance, the fractions skill score by about a tenth as much at a neighbourhood correlation of 0.5, and percentile thresholds never remove that second hedge.
• The 38 to 47 percent heavy-rain recovery of Park and colleagues (2026) comes from two Korean events in 2023, not from the French dataset, and their full-period figure is lower.
• The episode's readings of Ravuri and colleagues (2021), Ayzel and colleagues (2020) and Leinonen and colleagues (2023) all survive: placement skill rather than sharpness, a headline that fails above 15 mm per hour, and uncertainty accurate in aggregate but not in location.
• Park and colleagues (2026) report a rank histogram reaching 2.3 times the uniform rate by sixty minutes and worse on the heavy-rain subset, which closes the one item the episode left open, and closes it against the ensemble.

How they bear on the open questions

The sources bear most directly on the library's question of whether deterministic pixel-loss nowcasting can ever be sharp and accurate at long lead, and on its two open sub-questions. The first, whether modelling the predictive distribution recovers sharpness without sacrificing categorical skill, now has ensemble-level evidence in Park et al. (2026) beside the single-sample gains the library records for residual diffusion (Yu et al., 2024) and for cascaded generation conditioned on a blurred deterministic forecast (Gong et al., 2024); but Antonio (2026) shows that the yardstick for skill retention is itself tilted toward smooth fields, so part of any reported tie between a sharp and a blurred forecast on neighbourhood scores is an artefact of the score. The library's own handling of that paper is the sharper lesson: it is recorded as the first analytic answer to the hedging question and still leaves it open, because the result is derived rather than demonstrated on real forecasts and covers only two scores of one family.

The second sub-question, how much of the blur is loss-induced and how much irreducible, is answered in part by the spectral decomposition of Ayzel et al. (2020), which attributes the loss of detail below 16 km at five minutes to spatial smoothing learned as an efficient way to factor in spatial uncertainty, and the growth of that loss to 32 km at sixty minutes to an artefact similar to numerical diffusion produced by feeding the network its own output. The library's separate question about rollout cautions against reading the split as a full answer: the direct, lead-time-conditioned models of Sønderby et al. (2020) and Espeholt et al. (2022) perform no recursion and still blur with lead time, so removing the rollout removes only one component. Seed (2003) had already built the decay of each scale at its own measured lifetime into a forecast by construction and noted that adding a noise term turns the same model into a generator of conditional simulations; Roberts and Lean (2008) prescribe smoothing to the smallest useful scale and treating smaller scales as stochastic noise. The generative turn the episode describes is, in the library's reading, the re-addition of a noise term written down twelve years before the first convolutional LSTM (Shi et al., 2015), and the failed conditional GAN two years after it (Shi et al., 2017) marks how long that step took.

On the probabilistic side the episode's closing claim, that a crowd of samples knows how large its error is but not where it is, is the distinction the library now uses to file both of its new sources. Park et al. (2026) answers a magnitude question and is recorded against the under-dispersion, heavy-rain and generalisation questions without moving any; the placement question belongs to Bonte et al. (2026), whose surrogate-ensemble test, matching value distributions and spectra while randomising spatial phase, is the instrument that would show whether thirty members carry dynamical information rather than statistical spread, and it has not been run on them. The cross-domain neighbours are unchanged: Lim and Zohren (2021) note that the recursive structure of iterative multi-horizon forecasting accumulates error over the horizon and that direct sequence-to-sequence prediction alleviates it, and Benidis et al. (2022) add that probabilistic forecasts matter even when only a point forecast is wanted. The nowcasting evidence that removing rollout does not remove blur is a two-dimensional result those one-dimensional surveys do not carry.

• Park and colleagues (2026) add ensemble-level evidence to the single-sample gains of Yu and colleagues (2024), and the question of whether a predictive distribution recovers sharpness without losing skill stays open, because their members are drawn around a frozen deterministic parent and come out under-dispersive.
• Antonio (2026) shows the yardstick itself is tilted toward smooth fields, so part of any reported tie between a sharp and a blurred forecast on neighbourhood scores is an artefact of the score.
• The proof in Antonio (2026) is analytic, covers two scores of one family and is demonstrated on no real forecast, which is why the library leaves the hedging question open.
• Ayzel and colleagues (2020) split the blur into learned smoothing and a recursion artefact, but Sønderby and colleagues (2020) and Espeholt and colleagues (2022) blur without recursion, so removing the rollout removes only one component.
• Magnitude and placement stay separate: Park and colleagues (2026) answer the first, and the surrogate-ensemble test of Bonte and colleagues (2026) that would answer the second has not been run on them.

References

1. Antonio, B. (2026). Investigating how the Fractions Skill Score and Brier Divergence Skill Score reflect forecast error. arXiv. https://arxiv.org/abs/2603.15247
2. Ayzel, G., Scheffer, T., & Heistermann, M. (2020). RainNet v1.0: A convolutional neural network for radar-based precipitation nowcasting. Geoscientific Model Development. https://doi.org/10.5194/gmd-13-2631-2020
3. Benidis, K., Rangapuram, S. S., Flunkert, V., Wang, Y., Maddix, D., Turkmen, C., Gasthaus, J., Bohlke-Schneider, M., Salinas, D., Stella, L., Aubet, F.-X., Callot, L., & Januschowski, T. (2022). Deep learning for time series forecasting: Tutorial and literature survey. ACM Computing Surveys. https://doi.org/10.1145/3533382
4. Bonte, M., De Cruz, L., Debal, F., & Vannitsem, S. (2026). Spread/error relationship and spatial error structure of precipitation ensemble nowcasting: Comparison of STEPS and generative AI. arXiv. https://doi.org/10.48550/arXiv.2601.19298
5. Das, P., Posch, A., Barber, N., Hicks, M., Vandal, T. J., Duffy, K., Singh, D., van Werkhoven, K., & Ganguly, A. R. (2024). Hybrid physics-AI outperforms numerical weather prediction for extreme precipitation nowcasting. npj Climate and Atmospheric Science. https://doi.org/10.1038/s41612-024-00834-8
6. Espeholt, L., Agrawal, S., Sønderby, C., Kumar, M., Heek, J., Bromberg, C., Gazen, C., Carver, R., Andrychowicz, M., Hickey, J., Bell, A., & Kalchbrenner, N. (2022). Deep learning for twelve hour precipitation forecasts. Nature Communications. https://doi.org/10.1038/s41467-022-32483-x
7. Foresti, L., Sideris, I. V., Nerini, D., Beusch, L., & Germann, U. (2019). Using a 10-year radar archive for nowcasting precipitation growth and decay: A probabilistic machine learning approach. Weather and Forecasting. https://doi.org/10.1175/WAF-D-18-0206.1
8. Germann, U., & Zawadzki, I. (2002). Scale-dependence of the predictability of precipitation from continental radar images. Part I: Description of the methodology. Monthly Weather Review. https://doi.org/10.1175/1520-0493(2002)130<2859:SDOTPO>2.0.CO;2
9. Gilleland, E., Ahijevych, D., Brown, B. G., Casati, B., & Ebert, E. E. (2009). Intercomparison of spatial forecast verification methods. Weather and Forecasting. https://doi.org/10.1175/2009WAF2222269.1
10. Gong, J., Bai, L., Ye, P., Xu, W., Liu, N., Dai, J., Yang, X., & Ouyang, W. (2024). CasCast: Skillful high-resolution precipitation nowcasting via cascaded modelling. Proceedings of the 41st International Conference on Machine Learning (ICML 2024). https://doi.org/10.48550/arXiv.2402.04290
11. Leinonen, J., Hamann, U., Nerini, D., Germann, U., & Franch, G. (2023). Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification. arXiv. https://doi.org/10.48550/arXiv.2304.12891
12. Lim, B., & Zohren, S. (2021). Time series forecasting with deep learning: A survey. Philosophical Transactions of the Royal Society A. https://doi.org/10.1098/rsta.2020.0209
13. Park, D., Song, C., Chang, T., Ham, Y.-G., & Hong, Y. (2026). Diffusion-based refinement for kilometer-scale probabilistic precipitation nowcasting. arXiv. https://arxiv.org/abs/2608.30205
14. Prudden, R., Adams, S., Kangin, D., Robinson, N., Ravuri, S., Mohamed, S., & Arribas, A. (2020). A review of radar-based nowcasting of precipitation and applicable machine learning techniques. arXiv. https://doi.org/10.48550/arXiv.2005.04988
15. Pulkkinen, S., Nerini, D., Pérez Hortal, A. A., Velasco-Forero, C., Seed, A., Germann, U., & Foresti, L. (2019). Pysteps: An open-source Python library for probabilistic precipitation nowcasting (v1.0). Geoscientific Model Development. https://doi.org/10.5194/gmd-12-4185-2019
16. Ravuri, S., Lenc, K., Willson, M., Kangin, D., Lam, R., Mirowski, P., Fitzsimons, M., Athanassiadou, M., Kashem, S., Madge, S., Prudden, R., Mandhane, A., Clark, A., Brock, A., Simonyan, K., Hadsell, R., Robinson, N., Clancy, E., Arribas, A., & Mohamed, S. (2021). Skilful precipitation nowcasting using deep generative models of radar. Nature. https://doi.org/10.1038/s41586-021-03854-z
17. Roberts, N. M., & Lean, H. W. (2008). Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Monthly Weather Review. https://doi.org/10.1175/2007MWR2123.1
18. Seed, A. W. (2003). A dynamic and spatial scaling approach to advection forecasting. Journal of Applied Meteorology. https://doi.org/10.1175/1520-0450(2003)042<0381:ADASSA>2.0.CO;2
19. Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2015). Convolutional LSTM network: A machine learning approach for precipitation nowcasting. Advances in Neural Information Processing Systems 28. https://doi.org/10.48550/arXiv.1506.04214
20. Shi, X., Gao, Z., Lausen, L., Wang, H., Yeung, D.-Y., Wong, W.-K., & Woo, W.-C. (2017). Deep learning for precipitation nowcasting: A benchmark and a new model. Advances in Neural Information Processing Systems 30. https://doi.org/10.48550/arXiv.1706.03458
21. Sønderby, C. K., Espeholt, L., Heek, J., Dehghani, M., Oliver, A., Salimans, T., Agrawal, S., Hickey, J., & Kalchbrenner, N. (2020). MetNet: A neural weather model for precipitation forecasting. arXiv. https://doi.org/10.48550/arXiv.2003.12140
22. Song, C., Chang, T. Y., & Hong, Y. (2026). Extreme weather nowcasting via local precipitation pattern prediction. The Fourteenth International Conference on Learning Representations (ICLR 2026). https://arxiv.org/abs/2602.05204
23. Surcel, M., Zawadzki, I., & Yau, M. K. (2015). A study on the scale dependence of the predictability of precipitation patterns. Journal of the Atmospheric Sciences. https://doi.org/10.1175/JAS-D-14-0071.1
24. Veillette, M. S., Samsi, S., & Mattioli, C. J. (2020). SEVIR: A storm event imagery dataset for deep learning applications in radar and satellite meteorology. Advances in Neural Information Processing Systems 33. https://proceedings.neurips.cc/paper/2020/hash/fa78a16157fed00d7a80515818432169-Abstract.html
25. Yu, D., Li, X., Ye, Y., Zhang, B., Luo, C., Dai, K., Wang, R., & Chen, X. (2024). DiffCast: A unified framework via residual diffusion for precipitation nowcasting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024). https://doi.org/10.1109/CVPR52733.2024.02622
26. Zhang, Y., Long, M., Chen, K., Xing, L., Jin, R., Jordan, M. I., & Wang, J. (2023). Skilful nowcasting of extreme precipitation with NowcastNet. Nature. https://doi.org/10.1038/s41586-023-06184-4

Read into the library after publication:

• Investigating how the Fractions Skill Score and Brier Divergence Skill Score reflect forecast error arXiv:2603.15247
• Diffusion-Based Refinement for Kilometer-Scale Probabilistic Precipitation Nowcasting arXiv:2608.30205

Against the library

How this episode sits against the listener's own paper library. Coverage: covered.

What the sources add

The episode's three sources were read into the library on 14 September 2026, and the first thing a second reading finds is what the library has since done with them. Doswell and Schultz (2006) is recorded as reframing the question of how composite parameters transfer across event selection, storm-motion definitions, analysis systems and operational constraints, at high weight but explicitly as a position paper carrying no new data, and as supporting three further questions: whether the null cases in severe-weather samples are complete, whether surface moisture flux convergence has ever been established as a forecast rather than a diagnostic variable, and whether outcome can be attributed to vertical structure, where its pair of soundings with equal convective available potential energy is held as an illustration rather than as evidence. Zhang et al. (2025) partially addresses the transfer question at medium weight, provisionally because the paper sits under the library's citation floor, and supports the questions of analysis-sounding fidelity away from radiosonde sites and of null-case completeness. Maddox and Crisp (1999) enters at low weight as a single 1948 case at 1948 data density and touches only the question of how far mesoscale detail augments tornado potential. None of the three closes a question, and the library's record of why is now more specific than the episode was.

What the episode adds beyond its papers is an operational fact and a framing, and both are still absent from the library on this reading. The Storm Prediction Center's conditional intensity groups, in effect since 3 March 2026, leave outlook probabilities unchanged and state expected intensity separately, conditional on a report occurring (Storm Prediction Center, 2026). No card records the change. It is the operational adoption of the distinction Doswell and Schultz (2006) drew between a diagnostic variable, which describes the atmosphere at one instant, and a forecast parameter, which must show a time-lagged correlation with the weather it is meant to predict, and it is the only place in the episode's material where that distinction has been acted on rather than argued for.

The framing is the episode's organising claim, that proximity-sounding designs answer the conditional question of storm type and are silent on the unconditional question of occurrence. It is still not stated as such anywhere in the library, although its ingredients are all held. Thompson et al. (2003) assessed the significant tornado parameter "within our dependent sample" and warned that "the actual frequency of significant tornadoes is likely lower than shown here". Rasmussen and Blanchard (1998) concluded that sounding information "can only delineate if conditions are generally favorable", after which the forecaster must turn to mesoscale features such as boundaries. Taszarek et al. (2020) supply the reanalysis population of favourable environments against which any occurrence base rate would have to be computed. The framing is therefore an assembly of library material rather than a new finding, which is the honest description of what a well-covered topic can add.

• Doswell and Schultz (2006), Maddox and Crisp (1999) and Zhang and colleagues (2025) are now library cards, attached to its questions on composite-parameter transfer, null-case completeness, analysis-sounding fidelity and vertical-structure attribution, and none of them closes one.
• The Storm Prediction Center's conditional intensity groups, operational since 3 March 2026, are recorded on no card in the library.
• That change is a forecast office acting on the distinction Doswell and Schultz (2006) argued for, between a variable that describes the atmosphere now and one that must correlate with the weather at a lead time.
• The episode's claim that proximity soundings answer storm type and not storm occurrence is stated nowhere in the library, although Thompson and colleagues (2003), Rasmussen and Blanchard (1998) and Taszarek and colleagues (2020) supply every piece of it.

Where they differ from the library

No source contradicts a library finding. Doswell and Schultz (2006) reclassify the within-sample discrimination results as diagnostic rather than forecast value, on the grounds that proximity evaluations pair index values with events already known to have occurred and so never test lagged correlation, discrimination of nonevents, or skill on a case set independent of the developmental one. On that reading the guideline that a significant tornado parameter of 1 separates significantly tornadic from nontornadic supercells (Thompson et al., 2003) and the ranking of the energy-helicity index as the best supercell discriminator (Rasmussen and Blanchard, 1998) are reframed rather than disputed, and the latter is restated, through a later re-evaluation of the same 1992 sample, as roughly a fifty percent chance of a significant tornado given a supercell, from about thirty cases per class. The library holds a partial reply: Thompson et al. (2012) applied effective-layer composite parameters to an independent sample of 22,901 events from 2003 to 2011 and found the significant tornado parameter rising monotonically with EF rating. That sample comes from the same author group and, as the authors note, its exclusion of weaker events "precludes a complete assessment of null cases". The standard Doswell and Schultz set is therefore asserted rather than met by anything the library holds, their own paper included.

Three holdings are narrowed on specific grounds. Zhang et al. (2025), on ERA5 proximity soundings for 273 Chinese supercells from 2002 to 2021, find that convective available potential energy, lifting condensation level, convective inhibition and low-level humidity do not separate significantly tornadic from non-tornadic storms, that the US-calibrated formulations of the significant tornado parameter fall to true skill statistics of 0.29 and 0.14 against values above 0.45 in the United States, and that the best discriminators are helicity and shear in the lowest 300 m; the mixed-layer cloud-base offset reported by Thompson et al. (2003) does not appear in China's humid environments, which makes cloud-base height regional rather than universal, and the refitted parameter reaches 0.51 as a maximum over a thousand thresholds on the whole sample with no held-out verification, so it inherits the dependent-sample limitation it was meant to escape. Coffer et al. (2025) find that substituting observed for estimated storm motion reverses the trend of shallower helicity layers having more skill, with tornadic supercells moving slower and slightly right of the estimate and nontornadic ones faster and well left, while advising that without prior knowledge of motion forecasters "may still be better off using SRH500"; Coniglio and Thompson (2024) locate the same reversal in only 240 profiles with 37 significant tornadoes. Maddox and Crisp (1999) contradict Miller's account that the two March 1948 settings were alike, finding similar surface patterns but markedly different conditions aloft, and conclude that a new base severe-weather plan and General Borum's presence motivated the forecast at least as much as pattern similarity did.

The library qualifies the episode in return, on the half of its argument that a first pass did not test. The episode holds that environment analysis is poor at saying whether a storm will form. That is true of the sounding and not of the boundary, and the evidence was already on the shelf. Wilson and Schreiber (1986) found that 79 percent of 418 storms initiating in their study area formed close to radar-observed boundary-layer convergence lines, rising to 95 percent for storms reaching 60 dBZ, and concluded that what often appears as random thunderstorm formation "is usually deterministic". Roberts and Rutledge (2003) added the cumulus growth above those features from satellite, with cloud tops crossing freezing and cooling faster than 8 degrees Celsius in fifteen minutes preceding the first radar echo by about fifteen minutes and a 35 dBZ storm by about thirty, and with the probability of detection of a fifteen-minute initiation nowcast rising from 0.25 to 0.85 over the radar-only baseline at little cost in false alarms. Neither card is new to the library; both were held before the episode was written, and neither was surfaced by its six original queries. The occurrence question is therefore not unanswerable but answerable from a different instrument, on a lead time of tens of minutes rather than hours, which sharpens the episode's argument about soundings rather than weakening it.

• Doswell and Schultz (2006) reclassify the within-sample results of Thompson and colleagues (2003) and Rasmussen and Blanchard (1998) as diagnostic rather than forecast evidence, because a proximity design never tests lagged correlation or discrimination of nonevents.
• The independent-sample reply the library holds, Thompson and colleagues (2012) on 22,901 events from 2003 to 2011, comes from the same author group and excludes weaker events, so the verification standard of Doswell and Schultz (2006) is still asserted rather than met.
• Zhang and colleagues (2025) find the US-calibrated tornado parameters falling to true skill statistics of 0.29 and 0.14 on 273 Chinese supercells, and the refit that reaches 0.51 was tuned on the whole sample with no held-out verification.
• Coffer and colleagues (2025) and Coniglio and Thompson (2024) both find that substituting observed for estimated storm motion reverses which helicity layer discriminates best, which makes the near-ground signal a diagnostic result rather than a forecast-time one.
• Wilson and Schreiber (1986) counted 79 percent of 418 storms initiating close to radar-observed convergence lines, so the episode's claim that the method is poor at whether a storm forms holds for the sounding and not for the boundary.

How they bear on the open questions

The sources bear most directly on the library's question of how composite parameters transfer across event selection, storm-motion definitions, analysis systems and operational constraints, and that question now carries the episode's own material as its statement of the standard: a verification dataset "completely independent of the developmental dataset", discrimination between events and nonevents, and contingency tables at 12, 6, 3 and 1 h before the event (Doswell and Schultz, 2006). Zhang et al. (2025) answer its regional-transfer facet negatively for China without meeting that standard, and the storm-motion results of Coniglio and Thompson (2024) and Coffer et al. (2025) fall under its event-selection, storm-motion and analysis-system facets and leave it open there as well. Nothing read into the library since the episode has moved it.

The initiation half rests on questions the library records as unresolved, and one of them has acquired a driver. How much of initiation uncertainty is observational and how much intrinsic remains open: Crook (1996) showed that temperature and moisture changes of 1 degree Celsius and 1 g per kg, within observational variability, switch initiation on or off, and Weckwerth and Parsons (2006) report that most targeted dryline missions of the 2002 campaign were null cases, but neither establishes a numerical limit. What is new since the episode is a physical origin for the boundaries themselves. Taylor et al. (2011) found from satellite observations over West Africa that storm initiation is twice as likely above strong soil-moisture gradients as over uniform soil, that 37 percent of initiations occurred over the steepest quartile of gradients at scales of 10 to 40 km, and that the sensitivity is strongest under large-scale conditions otherwise unfavourable, which implies that mesoscale surface patterns trigger storms that would not otherwise occur. Maybee et al. (2025) found in 78 model experiments that homogenising all sub-synoptic soil-moisture variability cuts peak mesoscale convective system counts by 23 percent. The convergence lines Wilson and Schreiber (1986) counted therefore have a candidate source that is observable in the morning, which is a different kind of answer to the occurrence question than any sounding parameter can give, and it currently rests on one region, one satellite analysis and one preprint.

Two neighbouring domains are touched, and one of them has become concrete since the episode. In data-driven weather forecasting, the critique of single-scalar composites reaches the Extreme Weather Bench of McGovern et al. (2026), which uses the Craven-Brooks composite as its sole convective-risk proxy. In time-series analysis, the lagged-correlation criterion that Doswell and Schultz (2006) propose now has a formal home in the library: it is Granger causality, which Runge et al. (2019) describe as testing whether omitting the past of one series worsens the prediction of the next step of another, and which they note is limited to lagged dependencies and degrades on sub-sampled series, a caveat that bears directly on contingency tables built at 12, 6, 3 and 1 h. The same holdings name the confounder that a proximity design cannot remove, through Reichenbach's principle that dependent variables are either causal to each other or driven by a common driver: a favourable sounding and a tornado may both be consequences of the synoptic pattern that produced them, which is why a lagged correlation is the minimum evidence Doswell and Schultz ask for and not sufficient evidence.

• Doswell and Schultz (2006) set the standard the library now uses to state its composite-parameter question, an independent verification set, discrimination of nonevents and contingency tables at 12, 6, 3 and 1 h, and nothing read into the library since has met it.
• Crook (1996) and Weckwerth and Parsons (2006) leave the initiation question where they found it, since neither separates observational uncertainty from intrinsic growth, and nothing read since does either.
• What is new is a driver rather than an answer: Taylor and colleagues (2011) find Sahelian initiation twice as likely over strong soil-moisture gradients, with 37 percent of it over the steepest quartile at scales of 10 to 40 km.
• Maybee and colleagues (2025) put 23 percent of peak Sahelian storm counts on that heterogeneity across 78 model experiments, which makes the driver observable in the morning and so far demonstrated in one region, by one satellite analysis and one preprint.
• Runge and colleagues (2019) give the episode's lagged-correlation criterion its formal name, Granger causality, together with its weakness on sub-sampled series and the common driver that a proximity design cannot condition away.

References

1. Chavas, D. R., & Dawson, D. T., II. (2021). An idealized physical model for the severe convective storm environmental sounding. Journal of the Atmospheric Sciences, 78(2), 653–670. https://doi.org/10.1175/JAS-D-20-0120.1
2. Coffer, B. E., Parker, M. D., Coniglio, M. C., & Homeyer, C. R. (2025). Supercell environments using GridRad-Severe and the HRRR: Addressing discrepancies between prior tornado datasets. Weather and Forecasting, 40, 1405–1428. https://doi.org/10.1175/WAF-D-24-0251.1
3. Coniglio, M. C., & Thompson, R. L. (2024). Impacts of sampling and storm-motion estimates on RUC/RAP-based discriminations of nontornadic and tornadic supercell environments. Weather and Forecasting, 39(10), 1417–1434. https://doi.org/10.1175/WAF-D-24-0027.1
4. Crook, N. A. (1996). Sensitivity of moist convection forced by boundary layer processes to low-level thermodynamic fields. Monthly Weather Review, 124, 1767–1785. https://doi.org/10.1175/1520-0493(1996)124<1767:SOMCFB>2.0.CO;2
5. Doswell, C. A., III, & Schultz, D. M. (2006). On the use of indices and parameters in forecasting severe storms. Electronic Journal of Severe Storms Meteorology, 1(3), 1–22. https://doi.org/10.55599/ejssm.v1i3.4
6. Maddox, R. A., & Crisp, C. A. (1999). The Tinker AFB tornadoes of March 1948. Weather and Forecasting, 14, 492–499. https://doi.org/10.1175/1520-0434(1999)014<0492:TTATOM>2.0.CO;2
7. Maybee, B., Klein, C., Taylor, C. M., Burns, H., & Marsham, J. H. (2025). Homogeneous soil moisture fields suppress Sahelian MCS frequency. arXiv. https://arxiv.org/abs/2509.12118
8. McGovern, A., Mandelbaum, T., Rothenberg, D., Loveday, N., Potvin, C., Flora, M., Magnusson, L., Gilleland, E., & Allen, J. (2026). Extreme Weather Bench: A framework and benchmark for evaluation of high-impact weather. arXiv. https://doi.org/10.48550/arXiv.2605.01126
9. Rasmussen, E. N., & Blanchard, D. O. (1998). A baseline climatology of sounding-derived supercell and tornado forecast parameters. Weather and Forecasting, 13, 1148–1164. https://doi.org/10.1175/1520-0434(1998)013<1148:ABCOSD>2.0.CO;2
10. Roberts, R. D., & Rutledge, S. (2003). Nowcasting storm initiation and growth using GOES-8 and WSR-88D data. Weather and Forecasting, 18. https://doi.org/10.1175/1520-0434(2003)018<0562:NSIAGU>2.0.CO;2
11. Runge, J., Bathiany, S., Bollt, E., Camps-Valls, G., Coumou, D., Deyle, E., Glymour, C., Kretschmer, M., Mahecha, M. D., Muñoz-Marí, J., van Nes, E. H., Peters, J., Quax, R., Reichstein, M., Scheffer, M., Schölkopf, B., Spirtes, P., Sugihara, G., Sun, J., Zhang, K., & Zscheischler, J. (2019). Inferring causation from time series in Earth system sciences. Nature Communications, 10. https://doi.org/10.1038/s41467-019-10105-3
12. Storm Prediction Center. (2026). Conditional intensity in SPC convective outlooks. National Oceanic and Atmospheric Administration, National Weather Service. https://www.spc.noaa.gov/exper/conditional-intensity-information/
13. Taszarek, M., Allen, J. T., Púčik, T., Hoogewind, K. A., & Brooks, H. E. (2020). Severe convective storms across Europe and the United States. Part II: ERA5 environments associated with lightning, large hail, severe wind, and tornadoes. Journal of Climate, 33, 10263–10286. https://doi.org/10.1175/JCLI-D-20-0346.1
14. Taylor, C. M., Gounou, A., Guichard, F., Harris, P. P., Ellis, R. J., Couvreux, F., & De Kauwe, M. (2011). Frequency of Sahelian storm initiation enhanced over mesoscale soil moisture patterns. Nature Geoscience, 4, 430–433. https://doi.org/10.1038/ngeo1173
15. Thompson, R. L., Edwards, R., Hart, J. A., Elmore, K. L., & Markowski, P. (2003). Close proximity soundings within supercell environments obtained from the Rapid Update Cycle. Weather and Forecasting, 18, 1243–1261. https://doi.org/10.1175/1520-0434(2003)018<1243:CPSWSE>2.0.CO;2
16. Thompson, R. L., Smith, B. T., Grams, J. S., Dean, A. R., & Broyles, C. (2012). Convective modes for significant severe thunderstorms in the contiguous United States. Part II: Supercell and QLCS tornado environments. Weather and Forecasting, 27, 1136–1154. https://doi.org/10.1175/WAF-D-11-00116.1
17. Weckwerth, T. M., & Parsons, D. B. (2006). A review of convection initiation and motivation for IHOP_2002. Monthly Weather Review, 134, 5–22. https://doi.org/10.1175/MWR3067.1
18. Wilson, J. W., & Schreiber, W. E. (1986). Initiation of convective storms at radar-observed boundary-layer convergence lines. Monthly Weather Review, 114. https://doi.org/10.1175/1520-0493(1986)114<2516:IOCSAR>2.0.CO;2
19. Zhang, R., Xue, M., & Yu, X. (2025). Environments of tornadic and non-tornadic supercells in China and optimized significant tornado parameter for China region. Quarterly Journal of the Royal Meteorological Society, 151(772), e5027. https://doi.org/10.1002/qj.5027

Read into the library after publication:

• On the Use of Indices and Parameters in Forecasting Severe Storms 10.55599/ejssm.v1i3.4
• The Tinker AFB Tornadoes of March 1948 10.1175/1520-0434(1999)014<0492:TTATOM>2.0.CO;2
• Environments of tornadic and non-tornadic supercells in China and optimized significant tornado parameter for China region 10.1002/qj.5027