📍 Processing: luxembourg ================================ 📥 Downloading 1 location(s)... ⬇️ Downloading OSM data for Luxembourg... 🔗 URL: https://download.geofabrik.de/europe/luxembourg-latest.osm.pbf luxembourg.osm.pbf: 0%| | 0.00/45.4M [00:00", line 198, in _run_module_as_main File "", line 88, in _run_code File "/home/hamiltka/heatmap/scripts/load_gpkg.py", line 63, in gdf = load_gpkg(input_path) File "/home/hamiltka/heatmap/scripts/load_gpkg.py", line 28, in load_gpkg gdf = gpd.read_file(path) File "/home/hamiltka/miniconda3/lib/python3.13/site-packages/geopandas/io/file.py", line 316, in _read_file return _read_file_pyogrio( filename, bbox=bbox, mask=mask, columns=columns, rows=rows, **kwargs ) File "/home/hamiltka/miniconda3/lib/python3.13/site-packages/geopandas/io/file.py", line 576, in _read_file_pyogrio return pyogrio.read_dataframe(path_or_bytes, bbox=bbox, **kwargs) ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/hamiltka/miniconda3/lib/python3.13/site-packages/pyogrio/geopandas.py", line 275, in read_dataframe result = read_func( path_or_buffer, ...<14 lines>... **kwargs, ) File "/home/hamiltka/miniconda3/lib/python3.13/site-packages/pyogrio/raw.py", line 198, in read return ogr_read( get_vsi_path_or_buffer(path_or_buffer), ...<15 lines>... datetime_as_string=datetime_as_string, ) File "pyogrio/_io.pyx", line 1313, in pyogrio._io.ogr_read File "pyogrio/_io.pyx", line 227, in pyogrio._io.ogr_open pyogrio.errors.DataSourceError: 'output/luxembourg/luxembourg_roads.gpkg' not recognized as being in a supported file format.; It might help to specify the correct driver explicitly by prefixing the file path with ':', e.g. 'CSV:path'. 📦 Loading layer 'roads' from output/luxembourg/luxembourg_roads.gpkg... ⚠️ Could not load 'roads' layer, trying default layer... 📦 Loading output/luxembourg/luxembourg_roads.gpkg... ❌ Projected roads file not found: output/luxembourg/luxembourg_roads_projected.gpkg ❌ Missing input file: output/luxembourg/luxembourg_roads_projected.gpkg ❌ File not found: output/luxembourg/luxembourg_road_density_2km.gpkg ❌ Roads file not found: output/luxembourg/luxembourg_roads_projected.gpkg ⚠️ No heatmap GeoTIFFs found in output/luxembourg ⚠️ No heatmap GeoTIFFs found in output/luxembourg ⚠️ No heatmap GeoTIFFs found in output/luxembourg ⚠️ No exported artifacts found in output/luxembourg 📦 Exported deliverables to heatmaps/luxembourg/ ================================ ✅ Pipeline complete for luxembourg 📍 Processing: luxembourg ================================ 📥 Downloading 1 location(s)... ✅ File already exists: /home/hamiltka/heatmap/output/luxembourg/luxembourg.osm.pbf (45.4 MB) ============================================================ ✅ Successfully downloaded: 1/1 📁 Downloaded files: • luxembourg: /home/hamiltka/heatmap/output/luxembourg/luxembourg.osm.pbf (45.4 MB) ============================================================ 🔗 Geofabrik URL: https://download.geofabrik.de/europe/luxembourg-latest.osm.pbf ✅ Found existing file: /home/hamiltka/heatmap/output/luxembourg/luxembourg.osm.pbf 🚧 Extracting roads... 📊 PBF file size: 45.4 MB 🔍 Reading OSM metadata... ✅ Extracted 140,594 roads from Luxembourg data. 📊 Memory usage: 157.1 MB 🌐 Reprojected to WGS84 (EPSG:4326). 💾 Saved extracted roads to: /home/hamiltka/heatmap/output/luxembourg/luxembourg_roads.gpkg 📦 Kept PBF files (DELETE_DOWNLOADED_PBF=False) ✅ Process complete! 📁 Folder: /home/hamiltka/heatmap/output/luxembourg 📥 Input: luxembourg.osm.pbf 💾 Output: luxembourg_roads.gpkg 🛣️ Roads extracted: 140,594 ------------------------------------------------------------ access area ... osm_type geometry 0 None None ... way MULTILINESTRING ((6.12775 49.57736, 6.12746 49... 1 None None ... way MULTILINESTRING ((6.12435 49.5784, 6.12414 49.... 2 None None ... way MULTILINESTRING ((6.08361 49.62056, 6.08342 49... 3 None None ... way MULTILINESTRING ((6.08291 49.62049, 6.08259 49... 4 None None ... way MULTILINESTRING ((6.08349 49.61896, 6.08347 49... [5 rows x 43 columns] 📦 Loading layer 'roads' from output/luxembourg/luxembourg_roads.gpkg... ✅ Loaded 140,594 features. CRS: EPSG:4326 🔁 Reprojected to local UTM (EPSG:32632). 💾 Saved projected layer to: /mnt/vision/data/output/luxembourg/luxembourg_roads_projected.gpkg 📦 Loading roads from output/luxembourg/luxembourg_roads_projected.gpkg 🗺️ CRS: EPSG:32632 📏 Generating grid with 2.0 km cells... ✅ Created 1,218 grid cells. 💾 Saved grid to: /mnt/vision/data/output/luxembourg/luxembourg_grid_2km.gpkg 📂 Working on Luxembourg (2.0 km grid) 📥 Roads: luxembourg_roads_projected.gpkg 📥 Grid: luxembourg_grid_2km.gpkg 📊 Computing road segments within each grid cell... ✅ Created 153,371 road pieces clipped to cells. 🧮 Summing clipped lengths per grid cell... ✅ Density calculated with in-cell lengths. 💾 Saved density results to: /mnt/vision/data/output/luxembourg/luxembourg_road_density_2km.gpkg 📦 Loading density data from luxembourg_road_density_2km.gpkg 💾 Saved heatmap to: output/luxembourg/luxembourg_road_density_2km.png 🗺️ Saved GeoTIFF heatmap to: output/luxembourg/luxembourg_road_density_2km.tif 📦 Loading road and grid data for Luxembourg... 📊 Surface categories: {'paved': 72248, 'unpaved': 18300, 'unknown': 15} 📊 Highway categories: {'drive': 57251, 'hike': 51884, 'walk': 28635, 'unknown': 1477, 'bike': 1347} 🎯 Generating density heatmaps for major categories: - highway_category: network type (drive/bike/walk/hike) - surface_category: surface type (paved/unpaved) ============================================================ 🔍 Processing attribute: highway_category ============================================================ 📋 Categories found (5): - bike: 1,347 roads - drive: 57,251 roads - hike: 51,884 roads - unknown: 1,477 roads - walk: 28,635 roads 🎯 Processing: highway_category=drive 📊 Computing density for highway_category=drive... ✅ Found 57,251 roads with highway_category=drive 📍 Cropped to 755 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_highway_category_drive_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_highway_category_drive_density_2km.tif 📍 Saved GeoJSON: luxembourg_highway_category_drive_density_2km_hotspots_500km2.geojson (33 regions) 🎯 Processing: highway_category=hike 📊 Computing density for highway_category=hike... ✅ Found 51,884 roads with highway_category=hike 📍 Cropped to 774 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_highway_category_hike_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_highway_category_hike_density_2km.tif 📍 Saved GeoJSON: luxembourg_highway_category_hike_density_2km_hotspots_500km2.geojson (44 regions) 🎯 Processing: highway_category=unknown 📊 Computing density for highway_category=unknown... ✅ Found 1,477 roads with highway_category=unknown 📍 Cropped to 256 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_highway_category_unknown_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_highway_category_unknown_density_2km.tif 📍 Saved GeoJSON: luxembourg_highway_category_unknown_density_2km_hotspots_500km2.geojson (55 regions) 🎯 Processing: highway_category=walk 📊 Computing density for highway_category=walk... ✅ Found 28,635 roads with highway_category=walk 📍 Cropped to 589 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_highway_category_walk_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_highway_category_walk_density_2km.tif 📍 Saved GeoJSON: luxembourg_highway_category_walk_density_2km_hotspots_500km2.geojson (42 regions) 🎯 Processing: highway_category=bike 📊 Computing density for highway_category=bike... ✅ Found 1,347 roads with highway_category=bike 📍 Cropped to 227 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_highway_category_bike_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_highway_category_bike_density_2km.tif 📍 Saved GeoJSON: luxembourg_highway_category_bike_density_2km_hotspots_500km2.geojson (42 regions) ============================================================ 🔍 Processing attribute: surface_category ============================================================ 📋 Categories found (3): - paved: 72,248 roads - unknown: 15 roads - unpaved: 18,300 roads 🎯 Processing: surface_category=paved 📊 Computing density for surface_category=paved... ✅ Found 72,248 roads with surface_category=paved 📍 Cropped to 763 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_surface_category_paved_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_surface_category_paved_density_2km.tif 📍 Saved GeoJSON: luxembourg_surface_category_paved_density_2km_hotspots_500km2.geojson (26 regions) 🎯 Processing: surface_category=unpaved 📊 Computing density for surface_category=unpaved... ✅ Found 18,300 roads with surface_category=unpaved 📍 Cropped to 746 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_surface_category_unpaved_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_surface_category_unpaved_density_2km.tif 📍 Saved GeoJSON: luxembourg_surface_category_unpaved_density_2km_hotspots_500km2.geojson (42 regions) 🎯 Processing: surface_category=unknown 📊 Computing density for surface_category=unknown... ✅ Found 15 roads with surface_category=unknown 📍 Cropped to 10 cells (out of 1,218 total) 💾 Saved PNG: luxembourg_surface_category_unknown_density_2km.png 🗺️ Saved GeoTIFF: luxembourg_surface_category_unknown_density_2km.tif 📍 Saved GeoJSON: luxembourg_surface_category_unknown_density_2km_hotspots_500km2.geojson (7 regions) ============================================================ ✅ All attribute-specific density heatmaps generated successfully! ============================================================ /home/hamiltka/heatmap/scripts/extract_hotspot_contours.py:313: UserWarning: Column names longer than 10 characters will be truncated when saved to ESRI Shapefile. hotspots_gdf.to_file(shp_path, driver="ESRI Shapefile") /home/hamiltka/miniconda3/lib/python3.13/site-packages/pyogrio/raw.py:723: RuntimeWarning: Normalized/laundered field name: 'perimeter_km' to 'perimeter_' ogr_write( 🎯 Processing 1 heatmaps for Luxembourg... 🔬 Using continuous raster analysis with smooth contours 🗺️ luxembourg_road_density_2km.tif 📊 Threshold: 39565.4 (captures ~500.0 km²) ✅ Found 30 hotspot region(s) 📍 Total area: 444.0 km² 📏 Total perimeter: 379.0 km 💾 Saved overlay: luxembourg_road_density_2km_hotspots.png 💾 Saved GeoJSON: luxembourg_road_density_2km_hotspots_500km2.geojson 💾 Saved Shapefile: luxembourg_road_density_2km_hotspots_500km2.shp ✅ Smooth contour extraction complete. /home/hamiltka/heatmap/scripts/extract_hotspot_contours.py:313: UserWarning: Column names longer than 10 characters will be truncated when saved to ESRI Shapefile. hotspots_gdf.to_file(shp_path, driver="ESRI Shapefile") /home/hamiltka/miniconda3/lib/python3.13/site-packages/pyogrio/raw.py:723: RuntimeWarning: Normalized/laundered field name: 'perimeter_km' to 'perimeter_' ogr_write( 🎯 Processing 1 heatmaps for Luxembourg... 🔬 Using continuous raster analysis with smooth contours 🗺️ luxembourg_road_density_2km.tif 📊 Threshold: 61133.6 (captures 5.0% = 39/778 non-zero pixels) ✅ Found 15 hotspot region(s) 📍 Total area: 126.0 km² 📏 Total perimeter: 144.0 km 💾 Saved overlay: luxembourg_road_density_2km_hotspots.png 💾 Saved GeoJSON: luxembourg_road_density_2km_hotspots_top5pct.geojson 💾 Saved Shapefile: luxembourg_road_density_2km_hotspots_top5pct.shp ✅ Smooth contour extraction complete. /home/hamiltka/heatmap/scripts/extract_hotspot_contours.py:313: UserWarning: Column names longer than 10 characters will be truncated when saved to ESRI Shapefile. hotspots_gdf.to_file(shp_path, driver="ESRI Shapefile") /home/hamiltka/miniconda3/lib/python3.13/site-packages/pyogrio/raw.py:723: RuntimeWarning: Normalized/laundered field name: 'perimeter_km' to 'perimeter_' ogr_write( 🎯 Processing 1 heatmaps for Luxembourg... 🔬 Using continuous raster analysis with smooth contours 🗺️ luxembourg_road_density_2km.tif 📊 Threshold: 47712.9 (captures 10.0% = 78/778 non-zero pixels) ✅ Found 19 hotspot region(s) 📍 Total area: 274.0 km² 📏 Total perimeter: 249.0 km 💾 Saved overlay: luxembourg_road_density_2km_hotspots.png 💾 Saved GeoJSON: luxembourg_road_density_2km_hotspots_top10pct.geojson 💾 Saved Shapefile: luxembourg_road_density_2km_hotspots_top10pct.shp ✅ Smooth contour extraction complete. /home/hamiltka/heatmap/scripts/generate_html_report.py:48: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). timestamp = datetime.utcnow().isoformat() 📄 Saved HTML report to: output/luxembourg/luxembourg_report.html 📦 Exported deliverables to heatmaps/luxembourg/ ================================ ✅ Pipeline complete for luxembourg