Theme
From Drops to Grid: Noise-Aware Spatio-Temporal Neural Process for Rainfall Estimation
One-Sentence Summary
DropsToGrid is a translation-equivariant Neural Process that fuses noisy, sparse personal weather station (PWS) time series with radar spatial context via a SetConv + U-Net + attention bottleneck architecture, producing calibrated probabilistic rainfall grids that outperform operational and deep learning baselines on real-world Danish and European datasets.
Problem
Core Challenge
Generating high-resolution rainfall maps from sparse, noisy surface observations. Traditional rain gauge networks (SYNOP) are too sparse ($\sim<$3 km) rainfall variability. Radar provides dense coverage but measures reflectivity, not surface rainfall, and requires error-prone empirical Z-R conversion. Privately-owned weather stations (PWS) offer denser crowd-sourced coverage but suffer from variable hardware quality, noise, and spatially irregular, dynamically changing networks.
Motivation
The core technical challenge is a spatial densification + multi-modal fusion problem under noise: model must infer dense gridded rainfall from irregular, sparse, noisy PWS time series while leveraging radar for spatial context. Radar and stations measure complementary quantities (atmospheric moisture vs. surface rainfall) with complex, non-local, context-dependent mappings. Existing deep learning densification methods (mostly NP-based) focus on smoother variables (temperature) or synthetic noise-free settings, leaving rainfall densification largely unexplored under real-world conditions.
The motivation is solidly engineering-driven with clear practical value: operational rainfall products are biased and low-resolution, and PWS networks are growing rapidly. The problem framing is well-motivated by real limitations in current meteorological practice.
Previous Work Gaps
- Classical interpolation (IDW, Kriging, GPs): degrade with irregular networks, intermittent rainfall, poor scalability
- NP-based methods (ConvCNP, SwinTNP): tested on temperature or coarse daily rainfall, not hourly rainfall with real noise
- Radar nowcasting (Earthformer, CasCast, DiffCast): ignore surface station data, inherit radar biases
- End-to-end weather models (Aardvark, FuXi-DA): exclude rainfall or limit to coarse accumulations
Method
Core Innovation
Two technical contributions:
- Translation-equivariant spatio-temporal architecture combining SetConv encoding, U-Net spatial extraction, and an attention bottleneck for temporal + cross-modal + spatial fusion
- Zero-Inflated Gamma (ZIG) loss explicitly modeling rainfall's zero-heavy, right-skewed distribution
Architecture
Overall pipeline: Sparse PWS sequence + gridded radar SetConv encoders U-Net encoder Temporal + Cross-modal + TE-Fusion bottleneck U-Net decoder ZIG distribution parameters
| Component | Function | Key Design |
|---|---|---|
| SetConv Encoder (x2) | Project sparse inputs to dense latent grid | Depthwise conv with non-negative weights; density channel normalization; independent encoders for PWS and radar |
| Modality-specific MLPs | Embed dense representations | Pointwise FFN per modality |
| 2D U-Net (depth 3) | Multi-scale spatial feature extraction | Processes each (timestep, source) independently; 32 channels; kernel size 3 |
| Temporal Pixelwise Transformer | Aggregate T timesteps per pixel | Learned global query ; learned positional embeddings for temporal order |
| Gated Cross-Attention | Fuse station (query) with radar (key/value) | TE-Transformer; learned spatial gate |
| TE-Fusion Transformer | Merge all features with translation equivariance | Relative displacement ; MLP-based pairwise attention weights |
| Probabilistic Decoder | Output ZIG parameters | U-Net decoder with skip connections; MLP to 3 channels |
Data flow dimensions (typical):
- Input: PWS + Radar
- After SetConv + MLP: per modality
- U-Net bottleneck: where
- After temporal attention: (T collapsed)
- After TE-fusion:
- Decoder output: (three ZIG parameters)
Key Equations
SetConv with density normalization:
where , , and is depthwise convolution with non-negative weights.
Zero-Inflated Gamma distribution:
ZIG loss:
where excludes input station cells to prevent memorization.
TE-Transformer relative displacement:
ZIG mean and variance (deterministic indicator):
where , , .
Architecture Design Deep Dive
SetConv Encoder: Each observation contributes a Gaussian-shaped bump with learnable width. The density channel prevents signal inflation in dense station regions. Two independent encoders for PWS and radar prevent cross-modal interference in aggregation. This is a direct extension of ConvCNP's SetConv, adapted for gridded rainfall inputs.
Temporal Pixelwise Transformer: Operates independently per pixel (no spatial attention at this stage). A single learned query aggregates all T timesteps via dot-product attention. This is computationally efficient ( per pixel, not ) but limits temporal modeling to per-pixel patterns. Learned positional embeddings encode timestep order.
Gated Cross-Attention: Station features (queries) attend to radar features (keys/values). A spatial gate modulates radar contribution, acting as a learned filter. This design assumes station observations are the primary signal and radar provides supplementary spatial context -- reasonable given that rain gauges are the ground truth.
TE-Fusion Transformer: Final fusion uses relative position encoding via normalized coordinate differences. Attention weights depend on both feature similarity and geometric displacement through a lightweight MLP. This ensures translation equivariance: shifting the input field shifts the output equivalently. Critical for geographic generalization.
Algorithm Flow
Training: Randomly mask 30-50% of PWS pixels per patch forward pass ZIG loss on non-input station cells AdamW with cosine-annealed LR (), weight decay , EMA decay 0.999 up to 50K steps, batch 32.
Inference: Full PWS + radar input forward pass output per grid cell rain indicator mean , uncertainty .
Training vs. inference difference: During training, input stations are excluded from the loss target to prevent memorization. During inference, all available stations are used as input.
Evidence
Key Results
| Method | CSI (PWS) | FSS (PWS) | CSI (SYNOP) | FSS (SYNOP) | CRPS (SYNOP) |
|---|---|---|---|---|---|
| OPERA | 0.298 | 0.526 | 0.323 | 0.551 | - |
| RainViewer | 0.292 | 0.546 | 0.304 | 0.525 | - |
| IMERG | 0.202 | 0.449 | 0.194 | 0.425 | - |
| ERA5 | 0.204 | 0.390 | 0.203 | 0.363 | - |
| Climate (DMI) | 0.437 | 0.740 | - | - | - |
| MM_ConvCNP | 0.484 | 0.769 | 0.507 | 0.760 | 0.025 |
| MM_SwinTNP | 0.407 | 0.676 | 0.424 | 0.666 | 0.030 |
| OOTG_ConvCNP | 0.486 | 0.748 | 0.522 | 0.754 | 0.023 |
| OOTG_SwinTNP | 0.433 | 0.711 | 0.457 | 0.700 | 0.028 |
| DropsToGrid | 0.532 | 0.819 | 0.551 | 0.795 | 0.023 |
Key findings:
- 78% CSI improvement over best operational baseline (Climate) on PWS holdout
- 6% CSI improvement over best DL baseline (OOTG_ConvCNP) on SYNOP stations
- CRPS tied with OOTG_ConvCNP (0.023) -- probabilistic advantage is marginal
- With only 5% of stations, still outperforms operational estimators by $>$24% CSI
- Cross-region evaluation: Denmark-trained model generalizes to all of Europe (2025, unseen)
Claims to Evidence Mapping
| Claim | Type | Evidence | Strength | Risk |
|---|---|---|---|---|
| Outperforms operational baselines | Empirical | Table 1, 4: CSI/FSS/FBI across PWS and SYNOP | Strong | Only on Denmark data (2024) |
| Outperforms DL baselines | Empirical | Table 3: all metrics | Strong | Only ConvCNP/SwinTNP as DL baselines |
| Well-calibrated uncertainty | Empirical | Fig. 4, CRPS scores | Moderate | No reliability diagram or PIT histogram reported |
| Works with few stations (5%) | Empirical | Appendix 13 station analysis | Strong | Single threshold tested, no continuous curve |
| Cross-regional generalization | Empirical | Appendix 14 Europe-wide eval | Strong | Only Denmark Europe, no reverse direction |
| Translation equivariance critical | Empirical | Ablation (Sec. 4, Appendix 12) | Strong | 4% CSI drop without it |
| ZIG loss better than Gaussian | Not directly tested | - | Weak | No ablation comparing ZIG vs. Gaussian NLL |
| Fusion bottleneck critical | Empirical | Ablation: 8% CSI drop without it | Strong | Only compared vs. standard conv bottleneck |
Limitations
- Single dataset scope: Primary evaluation only on Denmark 2024. Cross-region test (Appendix 14) adds breadth but Denmark-trained model was not compared against region-specific baselines.
- No ZIG loss ablation: The paper motivates ZIG heavily but does not compare against Gaussian NLL or other distributional losses. The advantage of ZIG over simpler alternatives is unquantified.
- CRPS is not clearly superior: DropsToGrid's CRPS (0.023) ties with OOTG_ConvCNP (0.023). The probabilistic improvement is negligible for this metric.
- Limited DL baselines: Only ConvCNP and SwinTNP variants. No comparison with recent transformer-based weather models or diffusion-based methods.
- No computational cost analysis: Training time (3h on H100) is reported but no inference time comparison, FLOPs, or memory analysis.
- Code not available: GitHub repo exists but only has README ("code coming soon"). Reproducibility cannot be verified.
- Single temporal resolution: Only 1-hour rainfall accumulations. No analysis of sub-hourly or multi-hourly windows.
- Deterministic rain indicator: The paper uses for mean/variance, discarding probabilistic information. The more general mixture variance formula is mentioned in the Appendix but not used.
Critical Assessment
Strengths
- Well-matched problem and method: Neural Processes are naturally suited for variable-size, irregular observation sets with uncertainty. Rainfall densification from PWS is an ideal application.
- Principled loss design: ZIG distribution explicitly models zero-inflation and right-skew of rainfall, which is physically motivated and theoretically grounded.
- Translation equivariance: Enables geographic generalization without retraining -- critical for real-world deployment across regions.
- Comprehensive evaluation: Operational baselines (OPERA, RainViewer, IMERG, ERA5, Climate) + DL baselines (ConvCNP, SwinTNP) + ablation + station density analysis + cross-region test + uncertainty visualization.
- Compact model: 192K parameters, 3 hours training on H100. Highly practical.
- Real-world data: Uses actual crowd-sourced PWS data with real noise, not synthetic benchmarks.
Weaknesses and Risks
Important Issues:
- No ZIG vs. Gaussian ablation: The core novelty claim (noise-aware ZIG loss) is not directly validated against simpler alternatives. This is a significant omission given that ZIG is presented as a key contribution.
- CRPS not clearly superior: The tied CRPS with OOTG_ConvCNP undermines the "better uncertainty" narrative. CRPS measures both sharpness and calibration; the tie suggests the probabilistic improvement may come from the architecture, not the ZIG loss.
- Pairwise CSI table (Table 2) is misleading: The paper claims "higher similarity to other products suggests greater robustness." But without ground truth, pairwise CSI between products is not a meaningful quality metric. This table adds noise rather than insight.
Minor Issues:
- The paper claims "first Neural Process-based approach for densifying rainfall from PWS data" but this is a narrow claim -- NPs have been used for climate downscaling (ConvCNP for temperature). The novelty is in the rainfall-specific adaptations (ZIG, radar fusion).
- The 5% station result ("outperforms by 24%") is presented without error bars or continuous station density curve.
- No discussion of failure modes: when does DropsToGrid fail? Heavy convection? Orographic rainfall? Frontal systems?
Reviewer Feedback
| Aspect | Assessment |
|---|---|
| Novelty | Moderate: extends ConvCNP with temporal attention, cross-modal fusion, and ZIG loss for rainfall. Each component is known; the combination is new for this problem. |
| Clarity | Good: well-structured, clear figures, detailed appendix. |
| Significance | Good: addresses a real practical problem with real-world data. |
| Correctness | Appears correct: standard NP framework, well-known loss, proper evaluation setup. |
| Reproducibility | Poor: code not yet available. |
Innovation Score: 6/10
Solid incremental contribution within the NP + weather application space. The combination of temporal attention, cross-modal gated fusion, TE-Transformer, and ZIG loss is well-motivated and empirically validated. However, no single component is individually novel -- the value is in the system-level integration for a specific (important) application.
Reusable Ideas
- Zero-Inflated Gamma loss for rainfall tasks: Any rainfall prediction/regression problem with zero-heavy distributions can adopt this loss. Simple to implement, well-motivated.
- SetConv with density normalization for sparse irregular observations: Useful pattern for any task involving sparse sensor data projected onto grids.
- Translation-equivariant cross-modal fusion via relative displacement attention: General technique for fusing heterogeneous spatial data sources where geographic generalization matters.
- Excluding input cells from loss target: Simple trick to prevent memorization of noisy inputs. Applicable to any denoising/densification task.
- Gated cross-attention for modality fusion: Learned spatial gate to filter unreliable modality contributions.
Related Work
| Paper | Main Contribution | vs. DropsToGrid |
|---|---|---|
| ConvCNP (Gordon et al., 2020) | Translation-equivariant CNP with SetConv | DropsToGrid extends with temporal attention, cross-modal fusion, ZIG loss |
| Gridded TNP (Ashman et al., 2025) | Swin Transformer in NP latent space for spatiotemporal data | DropsToGrid outperforms on rainfall; SwinTNP tested on smoother variables |
| ConvCNP for climate downscaling (Vaughan et al., 2022) | CNP for temperature downscaling | DropsToGrid tackles harder rainfall problem with noise and zero-inflation |
| MetNet-3 (Andrychowicz et al., 2023) | Sparse stations for weather forecasting | Focuses on temperature/wind, not rainfall; different task formulation |
| Earthformer (Gao et al., 2024) | Space-time transformer for weather | Radar-only nowcasting, no station fusion |
| CasCast (Gong et al., 2024) | Cascaded diffusion for precipitation nowcasting | Dense radar-to-radar, not station densification |
| RainSeer (Chen et al., 2025) | Physics-informed radar-rainfall reconstruction | Different problem: 3D radar to 2D rainfall, not PWS densification |
Research Direction and Outlook
Problem Value
High practical significance. PWS networks are growing globally (millions of stations). Operational meteorology increasingly relies on crowd-sourced data. The problem of integrating noisy, sparse observations with remote sensing is general and recurring.
Method Extensibility
The architecture can extend to: (1) additional input modalities (satellite, lightning), (2) multi-temporal prediction targets (not just current time), (3) other zero-inflated environmental variables (snow, soil moisture). The NP framework naturally handles variable input configurations.
Research Space
Next steps: (1) longer temporal windows for nowcasting, (2) adaptive station selection/attention weights for quality control, (3) integration with NWP models for data assimilation, (4) evaluation on more diverse climatic regions (tropics, arid zones).
Personal Decision
Worth referencing for anyone working on rainfall estimation from sparse observations. The ZIG loss and SetConv + density normalization patterns are directly reusable. The full system is too domain-specific for direct transfer to other problems, but the component-level ideas are valuable.
Scientific Taste Score
Novelty (创新性): ★★★☆☆ (3/5)
- Each component (SetConv, temporal attention, cross-modal fusion, ZIG) is known
- Novel combination for rainfall densification specifically
- "First NP for rainfall from PWS" is a narrow first claim
Rigor (严谨性): ★★★★☆ (4/5)
- Comprehensive evaluation with operational + DL baselines
- Proper holdout strategy (20% pixel holdout + independent SYNOP)
- Cross-region Europe evaluation
- Missing: ZIG ablation, reliability diagrams, computational cost analysis
Impact (影响力): ★★★★☆ (4/5)
- Direct practical value for operational meteorology
- PWS networks are growing; this addresses a real need
- Compact model (192K params) is deployment-friendly
- Cross-region generalization demonstratedOne-Sentence Verdict
DropsToGrid is a solid engineering contribution that successfully adapts Neural Processes to real-world rainfall densification, with comprehensive empirical validation, but lacks a decisive ablation to validate its core ZIG loss claim and offers only incremental architectural novelty over ConvCNP.