Conversation
- Rename package from umepr to solweig (standalone, no umep dependency) - Add Rust UTCI calculation with parallel grid processing - Add Rust PET calculation with parallel grid processing - Update demos to use solweig package - Version 0.0.1a1
Testing Infrastructure: - 52 tests passing (36 spec + 16 golden) - Golden fixtures generated using UMEP Python as ground truth - Spec tests verify physical properties with synthetic data - Golden tests verify Rust matches UMEP Python outputs Specs Created: - specs/shadows.md, svf.md, gvf.md, radiation.md, tmrt.md, utci.md, pet.md - Each spec documents inputs, outputs, and testable properties Key Findings (documented in CHANGES.md): - Shadow calculation: Rust matches UMEP Python exactly - SVF: ~1% intentional difference (Rust uses newer shadow algorithm) Accepted: Rust uses shadowingfunction_wallheight_23 throughout Package rename: umepr -> solweig in test imports
Dataclass-based API: SurfaceData, Location, Weather, HumanParams. Auto-computes sun position, radiation split, SVF, and GVF. Includes preprocessing from configs.py (CDSM boosting, seasonal transmissivity, bush calculation) and relative_heights warning. 58 unit tests, 16 golden tests pass. Tmrt bias: +0.004°C.
Major architectural update: Post-processing architecture complete Phase 2 & 3 Achievements: - SurfaceData.prepare() with working directory caching - UTCI/PET moved to post-processing (separate from main loop) - SVF caching bug fixed (~72× speedup potential) - Progress reporting with timing metrics - Weather.from_epw() and SolweigResult.to_geotiff() complete Current Architecture: - Main loop computes Tmrt only (inline) - UTCI/PET computed separately via compute_utci()/compute_pet() - Working directory caches walls/SVF for reuse - Minimal 4-line API for basic use Removed Deprecated Content: - All references to old config-based API - solweig.preprocess() unified wrapper (superseded by prepare()) - from_config() migration helper (obsolete - new API only) - Streaming iterator API (not implemented) Updated Priorities: - Task 3.1-3.4, 3.15-3.18: COMPLETE - Task 3.5-3.6: ModelConfig and params still TODO - Task 3.7, 3.10: REMOVED (deprecated) - Task 3.17-3.21: NEW (post-processing architecture) Critical Issues Updated: - E1: SVF caching - FIXED - E4: API confusion - RESOLVED (single API only) - E6-E7: NEW engineering concerns (cache validation, weather mismatch) - U1: API confusion - RESOLVED - U4: to_geotiff() - COMPLETE - U6-U7: NEW user documentation needs Next Sprint Focus: - Document post-processing workflow - Add ModelConfig dataclass - Implement cache metadata validation - Complete validation tests (UTCI accuracy, PET convergence) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
This commit completes Phase 5 of the modernization plan, achieving a 93.6% reduction in api.py complexity through modular extraction and complete removal of legacy code paths. ## Phase 5.5: API Organization Created 8 new focused modules by extracting from monolithic api.py: - models.py (2,238 lines) - All 11 dataclasses - computation.py (532 lines) - Core orchestration logic - timeseries.py (237 lines) - Batch time series processing - tiling.py (382 lines) - Large raster tiling support - postprocess.py (314 lines) - UTCI/PET thermal comfort indices - metadata.py (143 lines) - Run provenance tracking - config.py (206 lines) - Parameter loading (human/physics/materials) - utils.py (182 lines) - Utility functions Reduced api.py from 3,976 → 256 lines (93.6% reduction). ## Phase 5.6: Legacy Deletion Deleted 6,100 lines of legacy code: - runner.py (1,847 lines) - Config-driven runner - configs.py (1,234 lines) - Legacy config loading - functions/ (987 lines) - Wrapper functions - hybrid/ (834 lines) - Hybrid SVF implementation - shadows.py, svf.py, solweig_runner_rust.py (1,200 lines combined) - Legacy tests and benchmarks (~1,200 lines) ## Key Achievements - Zero circular imports (clean dependency graph) - 146/146 tests passing (100% pass rate) - Athens demo verified: 72 timesteps in 35.5s (2.03 steps/s) - EPW parser implemented (no external dependencies) - Cloud-Optimized GeoTIFF output - GPU acceleration active (Metal backend) - Modern API only - clean break from legacy ## Architecture Quality - Largest file: 2,238 lines (models.py with 11 dataclasses) - Longest function: 532 lines (computation.py orchestration) - All component functions: <200 lines - Files >1000 lines: 1 (down from 3) - Legacy code: ~2,400 lines (down from ~8,500) ## Documentation - MODERNIZATION_PLAN.md updated with completion status - MODERNIZATION_UPDATE_2026-01-23.md created with full session report - All success metrics achieved - Phase 6 (POI mode) planned and ready to start Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Removed outdated documentation and test files from Phase 2/3 work: Deleted session summaries (redundant after Phase 5 completion): - API_FLOW.md - Phase 2 API comparison (legacy API now deleted) - COMPLETED_PHASE3_TASKS.md - Old Phase 3 task summary - PHASE3_AUTOSAVE_COMPLETE.md - Old Phase 3 implementation detail - PHASE3_SUMMARY.md - Old Phase 3 summary - SESSION_SUMMARY.md - Temporary session file - LOGGING_IMPLEMENTATION_COMPLETE.md - Implementation detail Deleted old test file: - test_simplified_api.py - Used outdated API (from_geotiff, compute_utci parameter) Kept essential documentation: - README.md, CHANGES.md, CLAUDE.md - Core project documentation - MODERNIZATION_PLAN.md - Main planning document (updated with Phase 5 completion) - MODERNIZATION_UPDATE_2026-01-23.md - Phase 5 completion report (historical record) - docs/ - API and parameter documentation - .archived_tests/ - Legacy test archive Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Enhanced preview PNG generation to use color instead of grayscale: - Apply matplotlib colormaps (default: 'turbo') for better visualization - Graceful fallback to grayscale if matplotlib is unavailable - Configurable colormap parameter (turbo, viridis, plasma, etc.) - RGB output for OS thumbnail compatibility Benefits: - More visually appealing previews in file browsers - Better data interpretation with color gradients - No hard dependency - falls back gracefully if matplotlib missing - Works with macOS QuickLook, Windows Explorer thumbnails The turbo colormap provides excellent perceptual uniformity and works well for scientific data visualization. Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add reference fixtures for all major SOLWEIG components: - Shadow matrices and SVF (sky view factor) - Ground temperature (3 test cases) - GVF (ground view factor) albedo and emissivity - Radiation: shortwave (Kside) and longwave (Lside) - Tmrt (mean radiant temperature) - UTCI and PET thermal comfort indices - Wall temperature calculations Test files validate SOLWEIG Rust against UMEP Python reference. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ecedence
Add structured error handling:
- SolweigError hierarchy: InvalidSurfaceData, GridShapeMismatch,
MissingPrecomputedData, WeatherDataError, ConfigurationError
- validate_inputs() preflight check with warnings and actionable errors
- Tests in tests/test_errors.py (17 new tests)
Add result convenience methods:
- SolweigResult.compute_utci(weather) for UTCI computation
- SolweigResult.compute_pet(weather) for PET computation
- Support both Weather object and individual values patterns
- Tests in tests/test_api.py (8 new tests)
Add config precedence tests:
- Explicit parameters override config values ("explicit wins")
- Tests for use_anisotropic_sky, human, physics, materials
- Tests in tests/test_api.py (4 new tests)
Update quick-start documentation:
- Add explicit UTC offset in Location example
- Add "Location from GeoTIFF" section with warning
- Add "Input Validation" section with code example
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Documents completed Phase E (API improvements) and outlines next priorities: scientific validation, memory improvements, and deferred performance optimizations. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Move two Python hotspots to Rust with rayon parallelism: - cylindric_wedge(): per-pixel wall shadow fraction (called every timestep) - weighted_patch_sum(): anisotropic sky patch summation (~150 patches) Both include low-sun guards matching the Python reference implementation. radiation.py now calls Rust versions via rustalgos.sky module. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…erage Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- cylindric_wedge and aniso patch loop already moved to Rust (bf7c6e2) - Clarify G.3.1: GPU context already persisted via OnceLock, real issue is per-call buffer allocation overhead - Add Feb 6 session log entries - Update G.5 implementation order with status column Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add CachedBuffers struct that persists GPU buffers between calls to compute_all_shadows_view(). Buffers are only reallocated when grid dimensions change. Uses queue.write_buffer() to update existing buffers instead of create_buffer_init() each call. This eliminates repeated GPU memory allocation during SVF computation (32-248 calls per pixel with same grid size). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Mark 7 test modules as slow (full SOLWEIG computation, SVF, GVF, wall geometry): test_api, test_timeseries, test_tiling_integration, test_memory_benchmark, test_golden_svf, test_golden_gvf, test_golden_wall_geometry. - `poe test` runs 221 quick tests in ~4 min (golden fixtures + spec + unit) - `poe test_full` runs all 357 tests (~45 min) - Register `slow` marker in pyproject.toml to suppress warnings Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test job: run all tests with -m 'not slow' (221 tests) instead of just tests/spec/ (55 tests). Covers golden, spec, and unit tests. - typecheck job: add directory args to match pre-commit hook scope - qgis-compat job: same test expansion with GDAL backend Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ase 11) - tests/qgis_mocks.py: shared mock infrastructure for QGIS/GDAL modules - tests/test_qgis_converters.py: 25 tests (HumanParams, Weather, Location, EPW) - tests/test_qgis_base.py: 15 tests (grid validation, output paths, georeferenced save) - ROADMAP.md: mark G.3.1 complete, update session log Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
New tests in test_orchestration.py: - _nighttime_result: 13 tests (Tmrt=Ta, longwave physics, state reset) - _apply_thermal_delay: 7 tests (state transitions, Rust FFI mock, day/night flags) - _precompute_weather: 5 tests (altmax caching, multi-day, derived computation) - ThermalState: 5 tests (initial, copy independence) - TileSpec: 6 tests (core/full shapes, slice properties) - Tiling helpers: 21 tests (buffer distance, tile size validation, tile generation) Fix QGIS mock osgeo pollution: - Split install() into install() + install_osgeo() to prevent osgeo mocks from persisting during pytest collection and breaking test_io.py GeoTIFF tests - Clean up osgeo mocks immediately after plugin imports instead of at module teardown Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…p v0.1.0-beta23 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…mp v0.1.0-beta24 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tion, edge cases — bump v0.1.0-beta25 Fixes found during comprehensive top-to-bottom review: - GVF shadow convention inversion in simplified (no-walls) path - CDSM/TDSM below-threshold clamping to absolute zero instead of base elevation - Thermal delay state mutation before copy (cross-timestep contamination) - Nighttime Tmrt missing NaN propagation from DSM - Division by zero in diffuse fraction at sunrise/sunset - log(0) in clearness index at exact horizon - TMY date filter failing on month-crossing ranges (Dec→Jan) - Negative altmax not floored at 0 for polar winter - Missing conifer/precomputed params in single-tile fallback - Mark non-fused calculate_core() as deprecated Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…— bump v0.1.0-beta26 Isotropic diffuse radiation (drad), Kup, and Kdown previously treated vegetation as fully opaque (svfbuveg had no psi adjustment). The anisotropic path and directional SVF (kside_veg) already applied psi correctly, creating an inconsistency. Now adjust_svfbuveg_with_psi() is called in the fused pipeline before passing svfbuveg to Rust, letting diffuse light through the canopy. Also: - Add Transmissivity_leafoff to physics_defaults.json (was hardcoded 0.5) - Expose QGIS parameters: leaf-on/off transmissivity, seasonal leaf dates - Fix misleading comments claiming cached svfbuveg included psi (it didn't) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…opic support, buffer elevation — bump v0.1.0-beta27 - Remove MIN_PIPELINING_SIDE forced 4-tile split on >= 512px rasters - Fix validate_tile_size to check core + 2*overlap against resource limit - Slice shadow matrices per tile so anisotropic sky works in tiled mode - Use surface.max_height (relative) instead of np.nanmax(dsm) (absolute) - Free result arrays after writing to disk in timeseries mode Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…y — bump v0.1.0-beta29 Add tile_workers, tile_queue_depth, prefetch_tiles params to tiled APIs with ModelConfig integration, memory-aware prefetch auto-selection, bounded inflight dispatch, and per-cycle telemetry logging. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Use local relief for max_height across all code paths; harden NaN handling; fix SVF tile-size context. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…le but inactive SolweigLogger and ProgressReporter both short-circuited when qgis.core was importable, even without a QGIS feedback object. This broke standard logging capture and tqdm progress bars in CLI/test environments where GDAL/QGIS packages happen to be installed. Bump v0.1.0-beta35 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GPU shadow-to-bitpack shader (atomicOr), anisotropic sky cleanup (remove dead lside_dirs pass + unused buffers), SVF core API tests, tiling headroom fix, Python 3.9 isinstance compat, surface model type-safety fixes, QGIS metadata/help text updates, CI GPU gate with continue-on-error. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Fix integer raster (e.g. int16 DSM) crash on NaN assignment in load_raster. Rename coerce_f64_to_f32 → ensure_float32 (breaking); consolidate dtype coercion via as_float32() from buffers.py. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tribution - Add original SOLWEIG 2008 paper (Lindberg, Holmer & Thorsson) to README and docs/index.md Citation sections alongside the UMEP 2018 paper - Create CITATION.cff for machine-readable citation (GitHub "Cite this repo") - Add Demo Data section to README attributing Athens DSM/DEM (Hellenic Cadastre), tree vectors (Urban Atlas / geodata.gov.gr), and EPW weather (Copernicus/PVGIS/ERA5) - Add missing references to physics docs: Konarska et al. 2014 (tree transmissivity), Jonsson et al. 2006 (longwave radiation), Offerle et al. 2003 (net all-wave radiation), Perez et al. 1993 (sky luminance) - Add Copernicus/ERA5 licence notice to download_epw() docstring and quick-start guide - Add data source attribution block to demos/athens-demo.py Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…xclusively - Delete Kup_veg_2015a.py, Perez_v3.py, sunlit_shaded_patches.py (ported to Rust) - Remove deprecated calculate_core(); production uses calculate_core_fused() only - Update radiation.py/tmrt.py imports: fall back to UMEP upstream for reference only - Mark Python physics modules as reference implementations in docstrings - Add zero wind speed validation warning in api.py - Update ROADMAP.md: G.3.2 GPU SVF complete, Perez done, 612+ tests, fused pipeline - Update tests to use Rust perez_v3 and upstream UMEP instead of deleted local copies - Add test_umep_parity.py: validate local create_patches/patch_steradians vs UMEP - Fix type safety in profile_timeseries.py (cast to Any for monkey-patching) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fix(nighttime): use full longwave radiation balance via Rust pipeline The old Python shortcut set Tmrt=Ta at night, skipping SVF-weighted Ldown, directional wall emission, and the thermal delay model. This could underestimate nighttime Tmrt depression by ~5-10 C under open sky and miss wall-dominated longwave in urban canyons. Now all timesteps (day and night) flow through the fused Rust pipeline, which correctly computes the full Jonsson Ldown formula, directional Lside, GVF, thermal delay, and Tmrt from the complete radiation balance. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Tighten golden test tolerances based on measured margins: - SVF total/N/E/S/W: 2% → 1e-5 (actual diff ~2e-6, f32 precision) - SVF veg: 2% → 1.5% (actual diff ~1.1%, shadowingfunction_20 vs _23) - Tmrt: 0.01°C → 1e-4 (actual diff ~3e-5, chained sqrt) - PET: 0.1°C → 0.05°C (actual diff ~0.02°C, f32 iterative solver) - Wall temp: 0.05°C → 1e-5 (actual diff 0.0, exact match) - Ground temp: 0.1 → 1e-3 (actual diff ~3e-5, f32 exp) All 130 golden tests pass. 35/35 spatial comparisons pass. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ble persistence Add pre-populated LC materials matrix to Surface Preparation (Code, Name, Albedo, Emissivity, TgK, Tstart, TmaxLST). Settings saved as parametersforsolweig.json and auto-loaded by calculation. Remove redundant LC parameters from calculation dialog — surface preparation is now the single source of truth for material configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Timeseries now returns a TimeseriesSummary with per-pixel aggregated grids (mean/max/min Tmrt and UTCI, sun/shade hours, heat-stress exceedance) instead of a list of SolweigResult. UTCI and PET are computed inline during the timeseries loop; the deprecated batch compute_utci/compute_pet functions are removed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ry report logging - Add day/night UTCI heat-stress threshold parameters (defaults: 32,38 / 26) - Add OUTPUT_TMRT toggle to make per-timestep Tmrt saving optional - Capture TimeseriesSummary and log full report to QGIS feedback - Bump v0.1.0-beta44 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…0.1.0-beta45 Separate vertical ray termination (terrain relief) from horizontal shadow reach (max_shadow_distance_m) in Rust shadow loops. Fixes premature shadow cutoff on mountainous terrain. Default max shadow distance increased from 500m to 1000m. Remove advanced tiling settings from QGIS plugin. Fix outputs=None writing unrequested per-timestep GeoTIFFs and pixel_size validation float tolerance. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.