Population under active NWS alerts
Reading the National Weather Service alert feed…
People under alerts, by state
Top alert types by people affected
Alert lifecycle right now
Most-exposed counties
Every county under an active alert, its worst alert, and all alerts currently covering it.
Snapshot from the streaming pipeline
Gold tables written by the Kafka and Spark Structured Streaming job, exported from Delta Lake.
CAP messages landed in bronze, by type
Alert lifecycle in silver
Browser vs pipeline: verified identical
The live tab runs a JavaScript port of the Spark transforms. Both were run on the same 342-alert NWS snapshot and produced identical results: 285 active and 57 pending alerts, 118 counties, 45,666,969 people under alert, 18,047,155 under Severe or Extreme, and zero per-county differences.
How it's built
Download source code (.zip)Java producer (Maven, JUnit), PySpark streaming job, Docker Compose stack, Streamlit dashboard and tests
Architecture
Engineering decisions
- Exactly-once bronze. Each micro-batch appends with Delta
txnAppId/txnVersion, so a retried batch is a no-op. - Insert-only MERGE on the immutable CAP message id. Replaying Kafka from offset 0 rebuilds an identical lakehouse, and this was verified.
- Lifecycle is recomputed, not patched. Updates supersede and Cancels end earlier messages, so a late Cancel can never leave a phantom active alert.
- No double counting. A county is counted once, at its worst active severity, with the full alert list kept alongside.
A real-data catch
On the first live run, the largest "Extreme" alert in the country was a statewide Child Abduction Emergency (AMBER Alert), about 30 million people in California. NWS relays these in the same feed, under CAP category Rescue.
Exposure now counts weather alerts (category = Met) only, and the change has a regression test. Use the toggle on the Live tab to see the difference.