F1i$dZddlZddlZddlmZddlZddlmZddlmZddl Z ddl Z ddl mZdZdefd Zed k(rIej&d Zej+dd ej-Zeej0yy)uZ Extracts only road features (highways, streets, etc.) from an OpenStreetMap .pbf file downloaded from Geofabrik. ✅ Features: - Automatic folder creation pe except MemoryError as e: print(f"❌ Memory error during extraction: {e}") print("🔄 Attempting regional processing...") roads = extract_roads_by_regions(filename, location_dir, location) if roads is None: print("💡 Regional processing failed. Try processing a smaller country/region") sys.exit(1)ion - Saves standardized GeoPackage to: output/{LOCATION}/{location}_roads.gpkg N)Path)tqdm)OSM)get_geofabrik_urlc ddl}ddlm}tdt t |}|j }td||\}}} } | |z dz } | |z dz } g} tdD]}tdD]} ||| zz}||dz| zz}||| zz}||dz| zz}||||f}td|dzd |dzd ||jd d id dd|gd}|B|js6tdt|dd|dzd |dz| j|ntd|dzd |dz|j| rPtdtjt!j"| d }tdt|dd|Sy#t$r%}td|dzd |dzd|Yd}~Pd}~wwxYw)z9Extract roads by processing geographic regions separatelyrN)boxu8�️ Processing large dataset by geographic regions...u📍 Dataset bounds: u🔍 Processing region -z/16: highwayTkeepF r namelaneswidthsurfacematerialbridgetunnellayer) custom_filter filter_type keep_nodeskeep_relations bounding_boxextra_attributesu ✅ Found ,z roads in region u ⚪ No roads in region u ❌ Error in region : u� Combining all regions...) ignore_indexu✅ Total extracted: z roads from all regions)gcshapely.geometryrprintrstr get_boundsrangeget_data_by_custom_criteriaemptylenappendcollect Exceptiongpd GeoDataFramepdconcat)filename location_dirlocationr rosmboundsmin_lonmin_latmax_lonmax_latlon_steplat_step all_roadsijregion_min_lonregion_max_lonregion_min_latregion_max_lat region_bbox region_roadsecombined_roadss '/data2/heatmap/scripts/extract_roads.pyextract_roads_by_regionsrGs8 $ DE c(m C^^ F !& *+*0&GWgw'!Q&H'!Q&HI 1XqA# !(1x>#,d"3 &$#(!,& ?   +L4F4FL\):1(==NqQRseSTUVWXUXTYZ[$$\25acU!AaC5AB AN ,-))"))ID*QR %c.&9!%<C F!! G*G  Gr2c  |j}tj}|dz |z }|jdd||dz }|dz } t |}t d| |js t d |jd tjdd 5}|jt|jjd d}d} t!|d5} t#|ddd|d5} |j%| D]-} | j'| | j)t+| / ddddddddd|j-j.dkr t dt d|nt d|t dd} t3t5|}t d|j7ddidddgd } | | j8r t d!t d"t+| d#d$|jd%t d&| j;d'j=d(z d)d*t d+t?| j@ | jGd01} t d2 ||d4z }| jI|d5d67t d8|d} |r9|jr)|jKt d:|jLnt d;|d<t d?t d@|t dA|jLt dB|jLt dCt+| d#t dDt | jOy#t $r.}t d|tjdYd}~jd}~wwxYw#1swYxYw#1swYxYw#1swYxYw#t0$r.}t d|tjdYd}~d}~wwxYw#tB$rS}t d,|t d-tE|||} | t d.tjdYd}~d}~wt0$r.}t d/|tjdYd}~Cd}~wwxYw#t0$r}t d3|Yd}~Jd}~wwxYw#t0$r.}t d9|tjdYd}~Yd}~wwxYw#t0$r}t d=|d>|Yd}~4d}~wwxYw)ENoutputT)parentsexist_okz.osm.pbfz data.gpkgu🔗 Geofabrik URL: u❌ r u ⬇️ Downloading OSM data for z... x)streamtimeoutzcontent-lengthri wbBi)totalunit unit_scale unit_divisordesc) chunk_sizeiu<Downloaded file seems too small — possible download error.u ✅ Download complete: u✅ Found existing file: u❌ Download failed: u 🚧 Extracting roads...u🔍 Reading OSM metadata...r r Fr)rrrrruDNo road data extracted — check the .osm.pbf file or location name.u✅ Extracted rz roads from z data.u📊 Memory usage: )deepiz.1fz MBu📋 Columns available: u$❌ Memory error during extraction: u$� Attempting chunked extraction...uF�💡 Try processing a smaller region or use a machine with more RAMu❌ Road extraction failed: i)epsgu&🌐 Reprojected to WGS84 (EPSG:4326).u⚠️ Reprojection failed: z _roads.gpkgroadsGPKG)rdriveru💾 Saved extracted roads to: u❌ Saving failed: u!🧹 Deleted processed PBF file: u+📦 Kept PBF files (DELETE_DOWNLOADED_PBF=)u⚠️ Could not delete ru ✅ Process complete!u 📁 Folder: u 📥 Input: u 💾 Output: u🛣️ Roads extracted: z<------------------------------------------------------------)(lowerrcwdmkdirrr" ValueErrorsysexitexists capitalizerequestsgetraise_for_statusintheadersopenr iter_contentwriteupdater(statst_sizer+rr#r&r' memory_usagesumlistcolumns MemoryErrorextract_roads_chunkedto_crsto_fileunlinkrhead)r2base_dirr1r0 output_pathurlrDr total_size block_sizefbarchunkrYr3DELETE_DOWNLOADED_PBFs rF extract_roadsrfs~~HxxzHh&1Ltd3 (33H,K) $SE*+ 4X5H5H5J4K5Q Rc$<""$ /?!CD ! (D)Q$#!%$:X. 1 !":!F 3u:."G1) = }}&&3 !_`` -hZ8 9 -hZ8 9 &' E+#h-  ,-//$d+   0 & =EKKcd d s5z!nL9L9L9N8OvVW #E$6$6D$6$A$E$E$G'$QRU#VVYZ[ (emm)<(=>? 2 $ ' 67"z%==  k @ / }=>": X__%6 OO  5hmm_E F ?@U?VVWX Y #$ M, () L () M+**+ ,- %c%j^ 45 (O %**,C  QCj  11)) =<*  %aS)*  X  4QC89 45%h hG = Z [ HHQK  ,QC01   2 ,QC0112  #A3'(   : ( "QC899:sN(AP.AO32O&AO O&O3A P5CP:S"*S3A T- O(#OOO# O&&O0 +O33O=8P P7 #P22P7: S AR S #SS  S0S++S03 T*<#T%%T*- U6U  U__main__z2Download and extract OSM road data for a location.) descriptionz/Location name (e.g., alabama, egypt, thailand).)help)__doc__argparserapathlibrrerpyrosmr geopandasr,pandasr.scripts.geofabrik_registryrrGr#r__name__ArgumentParserparser add_argument parse_argsargsr2rFrs  8 BNNCNb z $X $ $HF  )Z[    D$-- r