Improve GeoParquet support

[aykut-bozkurt]

New metadata fields: We write other metadata info (crs, orientation, edges, bbox) in the GeoParquet spec. Currently, we can only write EPSG:4326 as crs.

Fix: previously, we were writing geometry_types wrong. According to the spec, in the same geometry column, we might have multiple different type of geometry objects but we assumed that we always have a single type.

Geography type: Write metadata for postgis geography type as well in addition to geometry type.

TODOS:

• Add tests.
• Spi run per row reduces perf.
• install postgis_sfcgal extension at ci.
• Update README.md

Closes #132.

[codecov]

Codecov Report

❌ Patch coverage is 60.77519% with 253 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.95%. Comparing base (b533b24) to head (da6ea5a).

Files with missing lines	Patch %	Lines
src/type_compat/geometry.rs	55.27%	178 Missing ⚠️
src/arrow_parquet/pg_to_arrow/geography.rs	0.00%	39 Missing ⚠️
src/arrow_parquet/arrow_to_pg/geography.rs	0.00%	15 Missing ⚠️
src/arrow_parquet/pg_to_arrow/context.rs	85.33%	11 Missing ⚠️
src/arrow_parquet/arrow_to_pg.rs	28.57%	5 Missing ⚠️
src/arrow_parquet/pg_to_arrow.rs	71.42%	2 Missing ⚠️
src/arrow_parquet/pg_to_arrow/map.rs	0.00%	2 Missing ⚠️
src/arrow_parquet/arrow_to_pg/context.rs	85.71%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #148      +/-   ##
==========================================
- Coverage   90.81%   88.95%   -1.87%     
==========================================
  Files          91       93       +2     
  Lines       10368    10855     +487     
==========================================
+ Hits         9416     9656     +240     
- Misses        952     1199     +247     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[antoine-de]

great work 👍

I tested this successfully. The only stuff I had to patch to make gpq validate happy was:

diff --git a/src/type_compat/geometry.rs b/src/type_compat/geometry.rs
index 795a3b3..be28339 100644
--- a/src/type_compat/geometry.rs
+++ b/src/type_compat/geometry.rs
@@ -136,8 +136,11 @@ pub(crate) struct GeometryColumn {
     pub(crate) encoding: GeometryEncoding,
     pub(crate) geometry_types: Vec<GeometryType>,
     pub(crate) crs: Option<serde_json::Value>, // e.g. "EPSG:4326" in pproj format
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) orientation: Option<GeometryOrientation>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) edges: Option<GeometryEdgeType>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) bbox: Option<GeometryBbox>,
 }

The gpq validation was complaining:

 ✓ file must include a "geo" metadata key
 ✓ metadata must be a JSON object
 ✓ metadata must include a "version" string
 ✓ metadata must include a "primary_column" string
 ✓ metadata must include a "columns" object
 ! column metadata must include the "primary_column" name
   ↳ not checked
 ✓ column metadata must include a valid "encoding" string
 ✓ column metadata must include a "geometry_types" list
 ✓ optional "crs" must be null or a PROJJSON object
 ✗ optional "orientation" must be a valid string
   ↳ expected "orientation" for column "geometry" to be a string, got a <nil>: null

I can push the modification / open another MR with the rebase and this patch if you want.

[aykut-bozkurt]

great work 👍

I tested this successfully. The only stuff I had to patch to make gpq validate happy was:

diff --git a/src/type_compat/geometry.rs b/src/type_compat/geometry.rs
index 795a3b3..be28339 100644
--- a/src/type_compat/geometry.rs
+++ b/src/type_compat/geometry.rs
@@ -136,8 +136,11 @@ pub(crate) struct GeometryColumn {
     pub(crate) encoding: GeometryEncoding,
     pub(crate) geometry_types: Vec<GeometryType>,
     pub(crate) crs: Option<serde_json::Value>, // e.g. "EPSG:4326" in pproj format
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) orientation: Option<GeometryOrientation>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) edges: Option<GeometryEdgeType>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub(crate) bbox: Option<GeometryBbox>,
 }

The gpq validation was complaining:

 ✓ file must include a "geo" metadata key
 ✓ metadata must be a JSON object
 ✓ metadata must include a "version" string
 ✓ metadata must include a "primary_column" string
 ✓ metadata must include a "columns" object
 ! column metadata must include the "primary_column" name
   ↳ not checked
 ✓ column metadata must include a valid "encoding" string
 ✓ column metadata must include a "geometry_types" list
 ✓ optional "crs" must be null or a PROJJSON object
 ✗ optional "orientation" must be a valid string
   ↳ expected "orientation" for column "geometry" to be a string, got a <nil>: null

I can push the modification / open another MR with the rebase and this patch if you want.

sure, feel free to complete todos.