Japan

[m-mohr]

see this linkedin post https://www.linkedin.com/posts/hiroo-imaki-b656963b_sourcecooperative-activity-7266885356606676992--RRS?utm_source=share&utm_medium=member_desktop on japan field boundaries. On Source and in GeoParquet. But not fiboa, not 1.1 with bbox and uncompressed.

From @cholmes

[ivorbosloper]

• Created a data survey based on the work referenced in the linkedin post, https://github.com/fiboa/data-survey/blob/main/data/JP.md
• Implementing the converter currently, see https://github.com/fiboa/cli/tree/jp . We have a challenge handling large datasets. Dataframes are fast, but all data is loaded in memory at once. Given the Japan-2024 32GB parquet file, and reading only a limited set of columns in the dataframe, the python converter process consumer 64GB of RAM.

[ivorbosloper]

The author Hiroo Imaki wants to work with us on this issue. I'll try to reach out.

[ivorbosloper]

• Made a Pull-request for converting the published parquet files. We would prefer convert the original Government sources, but this will work for now.
• Also updated the map; https://fiboa.org/map/