Skip to content

Skilful nowcasting of extreme precipitation with NowcastNet ​

Status: completed

Authors: Yuchen Zhang, Mingsheng Long, Kaiyuan Chen, Lanxiang Xing, Ronghua Jin, Michael I. Jordan, Jianmin Wang

Venue / Year: Nature / 2023

Affiliations: Tsinghua University, China Meteorological Administration, UC Berkeley

Links: DOI | DGMR Code

Tags: [[precipitation-nowcasting]] [[physics-informed-ml]] [[extreme-weather]] [[deep-generative-models]] [[radar-forecasting]]

One-Sentence Summary ​

NowcastNet unifies physical-evolution schemes and conditional deep generative learning to produce skillful 3-hour precipitation nowcasts with sharp multiscale patterns, addressing the limitations of both numerical weather prediction (too coarse, slow) and pure data-driven methods (violate physical conservation laws).

Problem ​

Core Challenge ​

Precipitation nowcasting requires predicting high-resolution, long-lead-time precipitation patterns with local details for extreme events (>95th percentile). The challenge stems from:

  • Multiscale nature: Mesoscale patterns (20+ km) governed by physical laws vs. convective-scale features (1-2 km) with chaotic dynamics
  • Physical constraints: Must obey advective conservation laws and precipitation physics
  • Chaotic dynamics: Convective initiation is highly sensitive to initial conditions, limiting predictability

Motivation ​

Current methods fail in distinct ways:

  1. Numerical weather prediction (NWP): Too coarse (mesoscale resolution), update cycles in hours, cannot capture convective-scale features
  2. Advection-based methods (pySTEPS/DARTS): Respect conservation laws but limited to linear regimes, ~1h lead time, location errors accumulate
  3. Pure deep learning (DGMR/PredRNN): End-to-end training but violate physical laws, suffer from blur/dissipation, unnatural motion for extremes

The paper argues that embedding physical knowledge into data-driven models is necessary for skillful extreme precipitation nowcasting.

Method ​

Core Innovation ​

NowcastNet introduces a physics-conditional deep generative model that combines:

  1. Evolution network (ϕ): Deterministic, physics-informed - captures mesoscale advective patterns
  2. Generative network (θ): Stochastic, data-driven - generates convective-scale details

Key insight: Scale disentanglement via conditioning mechanism prevents error propagation between scales.

Architecture ​

ComponentFunctionKey Design
Evolution EncoderContext from past radarShared U-Net backbone
Motion DecoderPredicts motion fields v₁:ₜNonlinear, end-to-end learnable
Intensity DecoderPredicts residuals s₁:ₜCaptures growth/decay
Evolution OperatorIterative advection + additionDifferentiable, stop-gradient between steps
Nowcast EncoderLatent representationSame structure as evolution encoder
Nowcast DecoderFinal prediction conditioned on evolutionSpatially adaptive normalization (SPADE)
Temporal DiscriminatorRealism check3D conv with multiple temporal kernels

Key Equations ​

Continuity equation (modified for precipitation):

∂x∂t+(v⋅∇)x=s\frac{\partial x}{\partial t} + (v \cdot \nabla)x = s

Evolution operator:

xt′=Advect(xt−1′′,vt),xt′′=xt′+stx_t' = \text{Advect}(x_{t-1}'', v_t), \quad x_t'' = x_t' + s_t

Physics-conditional generation:

P(x^1:T∣x−T0:0;θ,ϕ)=∫P(x^1:T∣x−T0:0,ϕ(x−T0:0),z;θ)P(z)dzP(\hat{x}_{1:T} | x_{-T_0:0}; \theta, \phi) = \int P(\hat{x}_{1:T} | x_{-T_0:0}, \phi(x_{-T_0:0}), z; \theta) P(z) dz

Training objectives:

  • Evolution: Jevolution=Jaccum+λJmotionJ_{evolution} = J_{accum} + \lambda J_{motion} (accumulation loss + motion regularization)
  • Generative: Jgenerative=βJadv+γJpoolJ_{generative} = \beta J_{adv} + \gamma J_{pool} (adversarial + pool regularization)

Architecture Deep Dive ​

Evolution Network Detail ​

The evolution network is built on a two-path U-Net architecture with the following structure:

Input: Past radar fields x_{-T₀:0} (9 frames, 256×256)
  ↓
Evolution Encoder (U-Net backbone)
  ├─ Motion Decoder → Predicts motion fields v₁:ₜ (20 frames)
  └─ Intensity Decoder → Predicts residuals s₁:ₜ (20 frames)

Key design choices:

  1. Spectral Normalization: Applied to every convolution layer (Miyato et al. 2018) for training stability
  2. Skip connections: Temporal concatenation - input/output fields concatenated along channel dimension
  3. Full temporal dependency: Encoder sees all past frames; decoders predict all future frames simultaneously (not autoregressive)

Evolution Operator Implementation:

python
# Backward semi-Lagrangian scheme
for t in range(T):
    # Step 1: Advect using motion field
    x'_t = backward_advection(x''_{t-1}, v_t)  # nearest interpolation
    
    # Step 2: Add intensity residual  
    x''_t = x'_t + s_t
    
    # Step 3: Stop gradient (critical!)
    x''_t = stop_gradient(x''_t)

Gradient handling trick:

  • Forward pass uses nearest interpolation (preserves sharpness)
  • Backward pass uses bilinear interpolation (differentiable)
  • This "gradient trick" enables end-to-end training while maintaining field sharpness

Accumulation Loss (for motion field optimization):

J_accum = Σ_{t=1}^T [ L_wdis(x_t, x'_t) + L_wdis(x_t, x''_t) ]

where L_wdis(x, x') = ||(x - x') ⊙ w(x)||₁
      w(x) = min(24, 1 + x)  # weights for heavy rain

Motion Regularization (based on continuity equation):

J_motion = Σ_{t=1}^T [ ||∇v_{t,1} ⊙ w(x_t)||₂² + ||∇v_{t,2} ⊙ w(x_t)||₂² ]

Gradient computed via Sobel filter:
∂₁v ≈ [[1,0,-1], [2,0,-2], [1,0,-1]] ⊛ v

Physics-Conditioning Mechanism (SPADE-based) ​

The key innovation enabling scale separation:

Nowcast Decoder Layer:
  Input: Features from nowcast encoder + latent z
  ↓
  Instance Normalization (parameter-free)
  ↓
  Concat with Evolution Network output (resized via avg pooling)
  ↓
  2-layer Conv → New mean & variance
  ↓
  Denormalized output

Why this works:

  • Evolution network provides mesoscale "guidance" (~20km scale)
  • SPADE modulates activations spatially, allowing convective-scale details to be injected
  • Prevents error propagation: evolution errors don't cascade into generation

Generative Network Detail ​

Structure:

Input: Past observations x_{-T₀:0} + Evolution predictions x''₁:ₜ
  ↓
Nowcast Encoder (same as evolution encoder)
  ↓
Nowcast Decoder (conditioned on evolution)
  ← Latent z ~ N(0,I) transformed by Noise Projector
  ↓
Output: Final predictions x̂₁:ₜ (1-2km scale)

Basic Blocks:

  • D-Block: Downsampling block with spectral norm + leaky ReLU
  • S-Block: Standard residual block
  • Spatial Norm: SPADE module for physics-conditioning

Temporal Discriminator:

Input: Real or predicted sequence x₁:ₜ
  ↓
Multi-scale 3D conv (kernel sizes: 4, 8, ..., T)
  ↓
Concatenate multi-scale features
  ↓
Score (real vs fake)

Ensemble Generation:

  • Sample k=4 latent vectors z₁,...,z₄
  • Generate k predictions x̂^{z₁}₁:ₜ, ..., x̂^{z₄}₁:ₜ
  • Pool regularization: Compare spatially-pooled mean to observations

Training Pipeline ​

Stage 1: Train Evolution Network

  • Input: T₀=9 past frames → Output: T=20 future frames
  • Optimizer: Adam, lr=1e-3, batch=16
  • Iterations: 3×10⁵ (lr decay to 1e-4 at 2×10⁵)
  • Loss weight: λ = 1×10⁻²

Stage 2: Train Generative Network

  • Evolution network frozen
  • Optimizer: Adam, lr=3e-5 (nowcast enc/dec/disc)
  • Iterations: 5×10⁵
  • Loss weights: β=6 (adversarial), γ=20 (pool), k=4 (ensemble)

Stage 3: Transfer Learning (if needed)

  • Pre-train on USA → Fine-tune on China
  • Evolution lr: 1/10 of generative lr (to preserve physical knowledge)
  • Iterations: 2×10⁵

Comparison with DGMR Architecture ​

AspectDGMRNowcastNet
CorePure generative (Latent AF + GAN)Physics-conditional (Evolution + GAN)
Scale handlingSingle scale, learns implicit physicsExplicit scale separation (20km + 1-2km)
Physical constraintsImplicit via adversarial trainingExplicit via evolution network
ArchitectureLatent AF + conditioning stackU-Net evolution + SPADE conditioning
TrainingEnd-to-end jointStaged (physics first, then refinement)
Lead time90 min180 min
ExtremesStruggles (blur/dissipation)Handles well (physical guidance)

Key architectural insight: DGMR relies on the generator to implicitly learn physics; NowcastNet explicitly encodes physical constraints (continuity equation) in a separate network, using the generative network only for detail refinement.

Design Decisions & Trade-offs ​

1. Why U-Net for both networks? ​

  • Pros: Skip connections preserve spatial details; proven for image-to-image tasks
  • Cons: Quadratic memory cost with resolution; limited long-range dependencies
  • Alternative: Could use Vision Transformer for better global context, but at higher compute cost

2. Why stop-gradient in evolution operator? ​

  • Problem: Backprop through advection operator causes numerical instability
  • Solution: Detach gradient between time steps, treat each step as independent for gradients
  • Trade-off: Sacrifices some temporal gradient flow for training stability

3. Why nearest + bilinear interpolation? ​

  • Nearest: Preserves sharp precipitation boundaries (critical for extremes)
  • Bilinear: Differentiable, enables gradient flow
  • The trick: Forward with nearest, backward with bilinear - best of both worlds

4. Why SPADE vs other conditioning? ​

  • SPADE (Park et al. 2019): Spatially adaptive normalization modulates per-location statistics
  • Alternative (FiLM): Feature-wise linear modulation - less spatial flexibility
  • Alternative (Concat): Simple concatenation - weaker conditioning
  • Choice: SPADE allows spatially varying guidance (evolution network output varies spatially)

5. Two-stage training vs end-to-end? ​

  • Two-stage: Train evolution first (physics), then generative (details)
  • Pros: Stable training, clear separation of concerns, transfer learning easier
  • Cons: Suboptimal - end-to-end might find better joint optimum
  • Evidence: Authors tried end-to-end but found it unstable (implied by design)

6. Spectral Normalization everywhere? ​

  • Why: Prevents mode collapse, stabilizes GAN training
  • Where: Every conv layer in both networks
  • Trade-off: Slightly restricts model capacity vs batch normalization

Implementation Challenges ​

  1. Memory: 256×256×20 frames with full U-Net requires significant GPU memory

    • Solution: Gradient checkpointing mentioned in paper
  2. Numerical stability: Advection with learned motion fields can diverge

    • Solution: Stop-gradient, motion regularization, weighted distance
  3. Training time: 5×10⁵ iterations at batch 16 ≈ weeks on high-end GPUs

    • Mitigation: Pre-training on large USA dataset, transfer to smaller regions
  4. Hyperparameter sensitivity: Multiple interacting hyperparameters (λ, β, γ, k)

    • Solution: Grid search with CSI+PSD criterion (mentioned in Methods)

Datasets ​

DatasetSourceTrainingTestSpatialTemporal
USAMRMS (NOAA)2016-202020213,500×7,000 @ 0.01°10 min
ChinaCMASep 2019-Mar 2021Apr-Jun 20213,584×3,584 @ 0.01°10 min

Sampling: Importance sampling for training (bias toward heavy precipitation); two test sets (importance-sampled for extremes, uniform for overall).

Key Results ​

Meteorologist Evaluation (n=62, China experts):

EvaluationUSA EventsChina Events
Posterior75.8% first choice67.2% first choice
Prior71.9% first choice64.4% first choice

Statistical significance: P < 10⁻⁴ vs competitors (pySTEPS, DGMR, PredRNN)

Quantitative Metrics (CSI neighbourhood at threshold ≥16 mm/h):

ModelT+1hT+2hT+3h
PredRNN~0.35~0.25~0.15
pySTEPS~0.45~0.30~0.20
DGMR~0.50~0.35~0.25
NowcastNet~0.55~0.45~0.35

PSD (Power Spectral Density): NowcastNet maintains consistent spectral characteristics across all wavelengths (1-1024 km) at 3h lead time, while competitors lose high-frequency components.

Case Study Highlights ​

Case 1 (Dec 11, 2021, USA): Convective fine line with tornado outbreak

  • pySTEPS: Large location error, loses line shape
  • DGMR: Unnatural dissipation, distorted shapes
  • NowcastNet: Accurate movement, preserves envelope, sharp multiscale patterns

Case 2 (May 14, 2021, China): Three distinct convective cells with red warnings

  • PredRNN/DGMR: Fast dissipation by T+2h
  • pySTEPS: Direction only, no shape prediction
  • NowcastNet: Predicts evolution of all three cells at T+3h

Claims → Evidence Analysis ​

ClaimTypeEvidenceStrengthRisk
NowcastNet outperforms SOTA on extreme precipitationEmpiricalFig 4: CSI, PSD metrics; 62 meteorologist evalStrongMetrics chosen align with strengths; no MAE/RMSE comparison
Physics-conditional design enables 3h lead timeEmpirical + TheoreticalCase studies (Fig 2-3), multiscale separationMedium-StrongNo ablation showing pure physics-only or pure ML-only baselines
Evolution network respects conservation lawsTheoreticalBased on continuity equation (Eq 2)MediumLearned motion fields may not strictly conserve mass; only partial physical constraints
Scale disentanglement prevents error propagationConceptualArchitecture design, conditioning mechanismMediumNo direct ablation measuring cross-scale error propagation
Transfer learning works across regionsEmpiricalPre-train USA, fine-tune ChinaMediumChina dataset smaller; may not generalize to radically different climates

Critical Assessment of Claims ​

  1. "First to achieve skillful 3h nowcasting": Supported by meteorologist evaluation and CSI improvements, but "skillful" is defined relative to competitors, not absolute operational thresholds.

  2. Physical plausibility: Case studies show visually plausible patterns, but no quantitative measure of physical consistency (e.g., mass conservation checks).

  3. Extreme precipitation focus: Importance sampling biases test set toward extremes; uniform sampling results less impressive (Supplementary Figs 10-11).

Limitations ​

Explicitly Acknowledged ​

  1. Momentum conservation not included - Future work mentioned
  2. Satellite data not exploited - Only radar, could add satellite
  3. Limited to 2D fields - No vertical atmospheric structure

Additional Concerns ​

  1. No code availability at publication - "Code Ocean" link provided but not GitHub
  2. Small meteorologist sample - 62 evaluators, all from China (potential regional bias)
  3. Missing baselines - No comparison to newer methods (2022-2023)
  4. Hyperparameter sensitivity - Limited discussion of tuning effort
  5. Computational cost - Inference "seconds" but no exact timing or hardware specs
  6. Calibration - No reliability diagrams or probabilistic forecast verification

Experimental Design Issues ​

  • Baseline implementation: Authors reproduced DGMR, not original DeepMind evaluation
  • Transfer learning: China dataset much smaller, may inflate relative performance
  • Threshold selection: CSI at 16/32/64 mm/h - why these specific values?

Contribution Assessment ​

Innovation Type ​

  • [x] Methodological: Novel physics-conditional architecture with scale separation
  • [x] Empirical: First to show 3h skill for extremes on large-scale radar datasets
  • [ ] Conceptual: Problem framing similar to physics-informed ML literature
  • [ ] Theoretical: Limited new theoretical insights
  • [ ] Engineering: System integration well-executed but not the main contribution

Incremental Value vs Competitors ​

Aspectvs DGMRvs pySTEPS
Core differenceAdds explicit physics (evolution network)Adds nonlinear learning + generative refinement
Lead time3h vs 90min3h vs ~1h
ExtremesBetter for advective/convectiveMuch better
Trade-offMore complex architectureStill requires ML training

Innovation Score: 7.5/10 ​

Justification: The scale separation and physics-conditioning mechanism is a genuine architectural innovation that addresses a real problem (multiscale precipitation forecasting). However:

  • Conceptually similar to other physics-informed ML approaches
  • Empirical validation strong but limited to two regions
  • No fundamental theoretical breakthrough
  • Strong competitor (DGMR) from DeepMind already existed

Critical Assessment ​

Strengths ​

  1. Clear problem identification - Articulates limitations of both NWP and pure ML methods
  2. Novel architecture - Physics-conditional design with scale separation
  3. Strong empirical validation - Two large datasets, expert evaluation
  4. Practical impact potential - Addresses real operational need in meteorology
  5. Well-motivated design choices - Each component tied to specific limitation

Weaknesses & Risks ​

  1. Limited ablation studies - No systematic ablation of key components (evolution network only, generative network only)
  2. Regional generalization - Only USA and China; tropical cyclones, monsoons untested
  3. Operational readiness - "Seconds" inference time vague; real-world deployment challenges unclear
  4. Comparison fairness - DGMR reproduced by authors, may not match original exactly
  5. Probabilistic skill unclear - Ensemble spread properties not evaluated

Reviewer Feedback (Hypothetical) ​

ReviewerScoreKey Points
ML Expert7/10Nice architecture, but physics-conditioning not new; need more ablations
Meteorologist8/10Addresses real need, impressive case studies, but operational validation needed
Theory Reviewer6/10Limited theoretical analysis; convergence? uncertainty quantification?

Recommendation ​

Accept with Minor Revisions (equivalent to Nature's "Accept")

  • Core contribution solid and well-supported
  • Novelty sufficient for Nature (major journal)
  • Impact significant for operational meteorology
  • Minor concerns about code availability, broader evaluation

Mainstream Approaches (as of 2023) ​

  1. Advection-based: pySTEPS (operational), DARTS - fast, physical but limited to ~1h
  2. Deep generative: DGMR (DeepMind) - strong for low intensity, struggles with extremes
  3. RNN-based: PredRNN - deployed in China, blurry predictions
  4. NWP-based: Traditional numerical methods - too coarse for nowcasting

Concurrent/Competing Work ​

  • 2021: DGMR (DeepMind + Met Office) - direct predecessor
  • 2022: MetNet (Google) - different approach using neural weather models
  • 2023: GraphCast (DeepMind) - global weather, not nowcasting specifically

Position Statement ​

NowcastNet occupies the "physics-informed ML" niche for precipitation nowcasting - stronger physical grounding than pure ML, more learning capability than pure physics.

Research Direction Assessment ​

Follow-up Potential ​

  • High value: Adding momentum/thermodynamic constraints, satellite fusion, 3D structure
  • Method extension: Physics-conditioning applicable to other multiscale prediction tasks
  • Operational deployment: Integration into existing forecasting pipelines

Personal Decision ​

  • [x] 值得参考: 思想有用,但不会基于这个直接做后续 - The physics-conditioning idea is transferable but domain-specific

Research Quality Score ​

创新性 (Novelty):        ★★★★☆ (4/5)
  - Architecture innovation is solid
  - Not paradigm-shifting but significant incremental

严谨性 (Rigor):         ★★★★☆ (4/5)
  - Strong empirical validation
  - Missing some ablations and theoretical analysis

影响力 (Impact):        ★★★★★ (5/5)
  - Nature publication ensures visibility
  - Direct operational relevance
  - Addresses pressing societal need (extreme weather)

Summary ​

这是一篇在降水临近预报领域通过物理条件生成模型实现了3小时极端降水可预报的论文,相比DGMR的优势是显式物理约束和多尺度分离,劣势是复杂度和可解释性。在Nature的标准下,这是一篇高质量的论文。


Reusable Ideas ​

  1. Scale disentanglement via conditioning - Applicable to any multiscale prediction
  2. Physics-informed evolution + data-driven refinement - Template for other physical systems
  3. Stop-gradient for underdetermined systems - Numerical stability technique
  4. Expert evaluation protocol - Framework for validating operational models

Notes ​

Static research notes built with VitePress and KaTeX.