Real-time flood monitoring system using zone-based hydrological modeling to track river conditions across multiple data sources. Combines Rust for reliable data curation with Python for statistical analysis and regression testing.
Status: Zone-based architecture implemented | Historical data ingestion complete | Analysis infrastructure operational
- Vision
- Project Structure
- Data Sources
- Reference Implementation
- Zone-Based Architecture
- Getting Started
- Configuration
- Documentation
Riverviews is a generalized flood monitoring system designed to work with any river or waterway. The system consists of:
- Rust Monitoring Daemon - Reliable, server-side data curation and simple alerting
- Python Analysis Scripts - Complex statistical analysis, regression, and ML modeling
The daemon ingests data from multiple sources (USGS gauges, USACE CWMS, ASOS weather stations, NWS events), validates and curates it in PostgreSQL, and provides zone-based monitoring through an HTTP API. Python scripts perform complex analysis on the curated data including historical flood characterization and regression testing.
riverviews/
├── flomon_service/ # Rust monitoring daemon
│ ├── src/
│ │ ├── bin/
│ │ │ ├── historical_ingest.rs # USGS historical backfill
│ │ │ ├── ingest_cwms_historical.rs # CWMS historical backfill
│ │ │ ├── ingest_peak_flows.rs # NWS peak flow ingestion
│ │ │ ├── analyze_flood_events.rs # Event analysis (deprecated)
│ │ │ └── detect_backwater.rs # Backwater detection
│ │ ├── alert/ # Threshold and staleness monitoring
│ │ ├── analysis/ # Zone-based groupings
│ │ ├── ingest/ # Multi-source API clients
│ │ ├── model.rs # Core data structures
│ │ ├── stations.rs # Station and zone registry
│ │ ├── lib.rs # Shared library
│ │ └── main.rs # HTTP API server
│ ├── scripts/
│ │ ├── generate_flood_zone_snapshots.py # Zone regression testing
│ │ └── README.md # Scripts documentation
│ ├── docs/ # Architecture documentation
│ ├── sql/ # Database migrations (001-006)
│ ├── zones.toml # Zone definitions
│ └── Cargo.toml
├── floml/ # Python analysis package
│ ├── floml/ # Core library (regression, correlation, db)
│ ├── scripts/ # Visualization and analysis tools
│ │ ├── zone_dashboard.py # Live monitoring dashboard
│ │ ├── visualize_zones.py # Zone detail viewer
│ │ ├── demo_correlation.py # Correlation analysis
│ │ └── README.md # Tool documentation
│ └── notebooks/ # Jupyter analysis notebooks
└── illinois_river_flood_warning.wiki/ # Technical documentation
Source: U.S. Geological Survey National Water Information System (NWIS)
| API | Coverage | Resolution | Use Case |
|---|---|---|---|
| Instantaneous Values (IV) | Last 120 days | 15 minutes | Real-time monitoring, recent flood detail |
| Daily Values (DV) | 1939-present | Daily means | Historical analysis, long-term trends, model training |
Measurements:
- Discharge (parameter 00060): Streamflow in cubic feet per second
- Gage height (parameter 00065): River stage in feet
Status: ✅ Fully implemented - API clients, database storage, 87 years of historical data loaded
Source: National Weather Service Advanced Hydrologic Prediction Service (AHPS)
Data Types:
- Historical flood events with crest times and peak stages
- Official flood stage thresholds (action/minor/moderate/major)
- Annual peak flow records
Use Case: Historical flood analysis, regression testing, threshold validation
Status: ✅ Implemented - 118 historical floods ingested (1993-2026), thresholds database-driven
Source: Iowa Environmental Mesonet (IEM) ASOS/AWOS network
Data Types:
- Hourly precipitation observations (1-hour accumulation)
- Temperature, dewpoint, humidity
- Wind speed, direction, and gusts
- Atmospheric pressure and visibility
- Sky conditions and present weather codes
Use Case: Precipitation is the primary driver of tributary flooding - monitor basin rainfall for flood prediction. Temperature and pressure support severe weather detection.
Stations:
- KPIA (Peoria) - Primary local precipitation (15-min polling)
- KBMI (Bloomington) - Mackinaw River basin (60-min)
- KSPI (Springfield) - Sangamon River basin (60-min)
- KGBG (Galesburg) - Spoon River basin (60-min)
- KORD (O'Hare) - Des Plaines River basin (6-hr)
- KPWK (Wheeling) - Des Plaines River tributary (6-hr)
Status: ✅ Fully operational - API client implemented, schema deployed (sql/006_iem_asos.sql), 6/6 stations verified, 16 integration tests passing
Source: U.S. Army Corps of Engineers CWMS Data API
Data Types:
- Mississippi River stage readings (backwater source monitoring)
- Illinois River lock/dam pool levels and tailwater stages
- Lock operations and gate positions
Use Case: Backwater flood detection - when high Mississippi River levels block Illinois River drainage, causing bottom-up flooding
Locations: Grafton, Alton, Hannibal (Mississippi); LaGrange, Peoria, Starved Rock (Illinois locks)
Status: ✅ Implemented - Historical CWMS data ingestion, backwater event detection and severity classification
Source: NOAA National Digital Forecast Database (NDFD) + Multi-Radar Multi-Sensor (MRMS)
Data Types:
- Precipitation forecasts (predicted rainfall, 1-7 day outlook)
- Radar-estimated observed rainfall
Use Case: Forecast future discharge based on predicted precipitation in upstream basins
Status: 📋 Schema designed, not implemented
Source: USDA NRCS SNOTEL + NOAA Climate Prediction Center
Data Types:
- Point observations from field stations (volumetric water content)
- Basin-averaged saturation index
Use Case: Saturated ground amplifies runoff - high soil moisture increases flood risk from precipitation
Status: 📋 Schema designed, not implemented
Geographic Focus: Peoria, Illinois and Upper Peoria Lake
Why Illinois River?
- Developer's local area (stakeholder knowledge)
- Major tributary system (Mississippi River basin)
- Managed waterway with many sensors under FOIA
- Documented flood history (1993, 2013, 2019)
Zone-Based Monitoring: The system organizes sensors into 7 hydrological zones from the Mississippi River (backwater source) through the Illinois River basin to the Chicago area. Each zone has defined lead times (0-120 hours) for flood prediction at the Peoria property zone. See Zone-Based Architecture below.
Riverviews uses a zone-based hydrological model rather than individual site monitoring. Sensors are organized into 7 geographic zones representing the flood propagation path to Peoria, IL:
| Zone | Name | Lead Time | Primary Sensors | Role |
|---|---|---|---|---|
| 0 | Mississippi River | 12h-5 days | Grafton, Alton, Hannibal | Backwater source detection |
| 1 | Lower Illinois | 6-24 hours | LaGrange, Havana, Spoon River | Backwater interface |
| 2 | Upper Peoria Lake | 0-6 hours | Peoria pool, Kingston Mines | Property zone (primary) |
| 3 | Local Tributaries | 6-18 hours | Mackinaw, Spoon, KBMI | Tributary flood monitoring |
| 4 | Mid Illinois | 18-48 hours | Henry, Starved Rock | Upstream flood propagation |
| 5 | Upper Illinois | 36-72 hours | Dresden, Kankakee, Des Plaines | Confluence monitoring |
| 6 | Chicago CAWS | 3-5 days | Lockport, CSSC, KORD weather | Lake Michigan drainage |
Flood Type Classification:
- Top-down: Upstream zones (4-6) elevated, flows downstream to property
- Bottom-up: Mississippi backwater (Zone 0) blocks Illinois drainage
- Local tributary: Zone 3 precipitation causing tributary flooding
- Compound: Multiple zones active simultaneously (worst case)
API Endpoints:
GET /zones- List all zones with metadataGET /zone/{id}- All sensors in a zone with current readingsGET /status- Overall basin flood status across all zonesGET /backwater- Backwater flood risk analysis
See flomon_service/zones.toml for complete zone definitions and illinois_river_flood_warning.wiki/ZONE_ENDPOINT_MIGRATION.md for API documentation.
- Rust 1.70+ (Edition 2024)
- PostgreSQL 14+
- USGS NWIS API access (no key required, rate-limited)
See flomon_service/docs/DATABASE_SETUP.md for complete instructions.
Quick start:
# Create database and user
sudo -u postgres psql <<SQL
CREATE DATABASE flopro_db;
CREATE USER flopro_admin WITH PASSWORD 'your_password';
GRANT ALL PRIVILEGES ON DATABASE flopro_db TO flopro_admin;
SQL
# Apply migrations
for f in flomon_service/sql/*.sql; do
sudo -u postgres psql -d flopro_db -f "$f"
done
# Grant permissions
sudo -u postgres psql -d flopro_db -f flomon_service/scripts/grant_permissions.sql
# Configure connection
echo "DATABASE_URL=postgresql://flopro_admin:your_password@localhost/flopro_db" > flomon_service/.env
# Validate setup
./flomon_service/scripts/validate_db_setup.shTest configuration against live APIs:
cd flomon_service
cargo run --release -- verifyVerifies all configured data sources are operational:
- USGS: Tests real-time API access for all configured gauges
- CWMS: Queries USACE catalog for lock/dam timeseries availability
- ASOS: Fetches recent observations from all weather stations
Generates verification report showing which stations are working and data availability. See docs/DATA_SOURCE_VERIFICATION.md for details.
USGS gauge data (1939-present):
cd flomon_service
cargo run --bin historical_ingest- Phase 1: Daily values from Oct 1939 to ~125 days ago
- Phase 2: 15-minute values for last 120 days
- ~692,000 readings loaded in 4-5 minutes
- State tracked in
historical_ingest_state.json
CWMS backwater monitoring:
cargo run --bin ingest_cwms_historical- Mississippi River stages (Grafton, Alton, Hannibal)
- Illinois River lock/dam data (LaGrange, Peoria, Starved Rock)
NWS peak flow events:
cargo run --bin ingest_peak_flows- 118 historical flood events (1993-2026)
- Annual peak discharge records
cd flomon_service
cargo run --release -- --endpoint 8080Available endpoints:
http://localhost:8080/zones- List all monitoring zoneshttp://localhost:8080/zone/2- Peoria property zone statushttp://localhost:8080/status- Basin-wide flood statushttp://localhost:8080/backwater- Backwater risk analysishttp://localhost:8080/health- Service health check
Sensor zones are defined in flomon_service/zones.toml. Each zone includes:
- Geographic extent and hydrological role
- Lead time to Peoria property
- Sensor list with roles (direct, boundary, precip, proxy)
- Alert conditions
Example zone entry:
[[zone]]
id = 2
name = "Upper Peoria Lake — Property Zone (Primary)"
lead_time_hours_min = 0
lead_time_hours_max = 6
primary_alert_condition = "Peoria pool > 447.5 ft / Kingston Mines stage > 14 ft"
[[zone.sensor]]
site_code = "05567500-PEORIA-POOL"
role = "direct"
relevance = "MOST IMPORTANT SINGLE READING for property elevation"Data source configurations:
- usgs_stations.toml - USGS gauge stations with flood thresholds
- iem_asos.toml - ASOS weather stations with basin assignments
- usace_iem.toml - USACE CWMS locations (lock/dam pools)
Legacy site-based configuration; zones.toml is now primary for monitoring organization.
Multi-schema architecture for data source separation:
usgs_raw.* -- USGS gauge readings and site metadata
nws.* -- NWS flood thresholds, forecasts, alerts (planned)
noaa.* -- Precipitation observations and forecasts (planned)
usace.* -- Lock/dam operations and releases (planned)
soil.* -- Soil moisture and saturation (planned)
Applied migrations:
001_initial_schema.sql- USGS gauge readings and site metadata002_monitoring_metadata.sql- Staleness tracking and health monitoring003_flood_metadata.sql- NWS flood thresholds and historical events004_usace_cwms.sql- CWMS locations, timeseries, backwater event detection005_flood_analysis.sql- Flood analysis tables and zone views006_iem_asos.sql- ASOS weather observations and precipitation summaries
Key tables:
usgs_raw.gauge_readings- Time-series discharge and stage (87 years × 15min resolution)nws.flood_events- Historical floods with crest times and peak stages (118 events)usace.cwms_timeseries- Mississippi/Illinois lock data for backwater detectionusace.backwater_events- Detected backwater floods with severity classificationasos_observations- Weather station observations (temperature, precipitation, wind, pressure)asos_precip_summary- Precipitation aggregations for flood risk thresholdsflood_analysis.zone_snapshots- Zone status at historical flood crests
See flomon_service/docs/SCHEMA_EXTENSIBILITY.md for schema design patterns.
- floml/README.md - Python analysis package (regression, correlation, ML)
- floml/scripts/README.md - Visualization tools and live monitoring
- floml/QUICKSTART.md - Get started with analysis
- flomon_service/docs/README.md - Full docs index for Rust daemon
- Database setup and configuration
- Data source integration (USGS, CWMS, ASOS)
- Architecture and design patterns
- Operational procedures
- flomon_service/scripts/README_ZONE_SNAPSHOTS.md - Zone snapshot regression
- flomon_service/scripts/generate_flood_zone_snapshots.py - Snapshot generation
Technical notes and design decisions: illinois_river_flood_warning.wiki/
Run all tests:
cd flomon_service
cargo testIntegration test suites:
tests/asos_integration.rs- ASOS weather station data collection and storage (16 tests)tests/data_source_verification.rs- Live API verification for USGS/CWMS/ASOS (4 tests)tests/daemon_lifecycle.rs- Daemon startup and monitoring behavior (13 tests)tests/peak_flow_integration.rs- Historical flood analysis integration (8 tests)
Test coverage:
- 78 unit tests (library code)
- 41 integration tests (end-to-end workflows)
- All ASOS stations verified operational (6/6)
- 5/8 USGS stations verified (3 decommissioned)
This is a personal flood monitoring project for portfolio demonstration. For official flood warnings and emergency information, always consult the National Weather Service and local emergency management authorities.