Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FlatGeobuf encoding #32

Draft
wants to merge 12 commits into
base: main
Choose a base branch
from
Draft

Add FlatGeobuf encoding #32

wants to merge 12 commits into from

Conversation

StefanBrand
Copy link
Contributor

@StefanBrand StefanBrand commented Jun 10, 2024

Related Issue(s): #24

PR Checklist:

  • This PR has no breaking changes.
  • I have added my changes to the CHANGELOG
    or a CHANGELOG entry is not required.

@m-mohr m-mohr linked an issue Jun 10, 2024 that may be closed by this pull request

## Potential issues in conversion

TBD
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fiboa allowes enums for strings and integers, but flatbuffers don't.
It think we should mention this here.
https://flatbuffers.dev/flatbuffers_guide_writing_schema.html

| binary | Binary |
| string<br />charset: UTF-8 | String |
| array | Json Array |
| object<br />keys: string<br />values: any | Json Object |
Copy link
Contributor

@m-mohr m-mohr Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about structs? I think both could work depending on the specificity of the schema.

I think fiboa, also foe GeoParquet, should do a better job at explaining which schema leads to a struct and which to a map. I'll open an issue: #34

Comment on lines +23 to +24
| date | DateTime (ISO 8601 date) |
| date-time<br />with milliseconds<br />timezone: UTC | DateTime (ISO 8601 date time) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't find datetime data types in the specification. Can you help me finding it? Otherwise it should probably string with a pre-defined format (e.g. ISO 8601 / RFC 3339)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I was refering to this line: https://github.com/flatgeobuf/flatgeobuf/blob/2828e07fd759aa8b44c6b1aaab41320b22dd0d29/src/fbs/header.fbs#L40 It is (probably) an ISO 8601 string, but it is it's own ColumnType in the FlatGeobuf schema.

Related comment: #32 (comment)

Copy link
Contributor

@m-mohr m-mohr Jun 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, that clairifies it. We need to clarify how we handle date only more practically. What happens with the missing components (hour, minute, ...). Maybe add a link somewhere to the allowed types.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't see that link before, looks good. So here the Date issue is still open.

@cholmes
Copy link
Contributor

cholmes commented Jun 19, 2024

Thanks for the contribution! This is awesome to see, and I love flatgeobuf.

My only hesitation is if there's going to be more of a burden to support 3 (or more, as geopackage would come next) 'officially' as part of fiboa. I had thought we'd welcome mappings to flatgeobuf and others, but mark things in some way to say that the mappings are more 'informative' and we're not committing to having all the tooling work with alternate encodings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Define FlatGeoBuf encoding
3 participants