Hi! I’m sick of the mess shapefiles create and it itches me to write a package for reading geometries from sqlite files (they proved to be rather small and fast last time I tested). Given the simplicity of the WKB format it should be rather easy. The question is: What should the geometries be parsed into? Polygons etc. from GeometryBasics? Or a copy of the GeoInterface implementation in Shapefile.jl? Or sth completely different? I really don’t want to implement Point type #1001. All in all I’m not 100% sure how the implementation should look like, so I could also use some guidance
Edit: Maybe it would even be possible to recycle the LibGEOS WKT/WKB parser?
Can you use the geometry types in Meshes.jl for this? The plan is to make it the go-to package for geometric processing in Julia. It fixes several design and mathematical issues in GeometryBasics.jl and is extensively tested, both in single and double floating point precision.
I am finishing a major refactoring of the GeoStats.jl stack to use Meshes.jl so having these shapefiles loaded in the correct geometry types already would be great.
Oh, looks sweet! Actually it reminds me a bit of Grassmann.jl due to the math backing and seems comparable in speed, but with actual usability due to down-to-earth docs and primitives one can understand. Which mapping did you use between Meshes.jl objects and the common types in GIS (MultiPolygon, LineString etc)?
These are the ones I remember out of the box. If you feel that something is missing or that the naming can be improved, feel free to open an issue. We are active there.
Out of curiosity, not that I could bark with the big dogs, but is this idea backed up by @visr and/or @sdanisch? (Dont know who’s responsible in other domains.)
I don’t know whether they support this initiative or not. I hope so. I’ve been working on Meshes.jl because my research depends on it, and because I couldn’t find what I needed in GeometryBasics.jl.
The changes needed to make GeometryBasics.jl more easy to use and more robust weren’t approved by the maintainers because these same changes would break downstream efforts like Makie.jl, which is where most of their development time is spent. Therefore, I created a separate project to improve the situation and to gather more people interested in computational geometry in Julia.
The plan is to submit PRs to all related packages replacing GeometryBasics.jl by Meshes.jl so that everyone can reuse this work and concentrate on the part they are mostly interested (e.g. visualization, FEM). I am working on Meshes.jl because ultimately I want to do GeoStats.jl + FEM + Makie.jl without having to deal with complex conversions between different geometric types.
Aah interesting! I simply didn’t know this functionality existed. In my mind the mapping always looked like:
(Arch)GDAL => raster data
Shapefile.jl => ESRI shapefiles
? => sqlite
I’ll give it a try and if it doesn’t work come back to this. Thanks
Derailing the topic a bit: @yeesian what do you think about the idea of having a single package for geometry primitives across different fields? I really like it and understand the motivation; would be cool to have it.
what do you think about the idea of having a single package for geometry primitives across different fields? I really like it and understand the motivation; would be cool to have it.
I think it could work out, but I’m less convinced that interoperability across packages will be based on a single unifying data representation/format, and more convinced that it has a higher chance of success via a traits-like interface.
My own priorities are focused on enabling use-cases and keeping the overall footprint sustainable for JuliaGeo and JuliaGeometry volunteers per Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure - Ford Foundation. For that reason, I think more about synergies with organizations like https://www.osgeo.org/, shared infrastructure in JuliaPackaging/Yggdrasil, and existing packages rather than implementing entire drivers in Julia from scratch.
I suspect my approach might have a stifling effect on innovation though (e.g. I wasn’t optimistic about LightGraphs.jl and Plots.jl in their early days, and I think I was wildly off the mark there), so I think it’s important to have diversity in viewpoints and for OSS developers to have the freedom to work on the projects that excite them.
I agree that there won’t be a single geometry package that will be used everywhere, so interfaces will remain important. Though I do see value in the community collaborating on a single package that works well for many cases. Perhaps a reasonable analogy would be the Tables interface, with DataFrames as an implementation that works well for most use cases, though not all.
This is also the point I was making in my JuliaCon 2020 lightning talk. There I suggested to go with GeometryBasics.jl, Meshes.jl didn’t exist yet. I was a bit afraid when Meshes.jl was (partially) forked off GeometryBasics.jl, that this would not help the community to collaborate on one. I haven’t seriously looked into Meshes.jl yet, though on the surface it looks quite neat, and @juliohm is clearly putting in a lot of energy there. And I’m happy that he intends to submit PRs to all related packages replacing GeometryBasics.jl by Meshes.jl. It will be quite an effort, with GeometryBasics.jl standing at 33 dependent packages versus Meshes.jl’s 0 at this moment. Before this can really begin probably @sdanisch would need to see if he thinks some of the different choices that were made will cause issues when migrating. I’m thinking for instance of this discussion on inheriting Point from StaticVector that was never concluded.
Regarding the discussion about inheriting from StaticVector my understanding is that this was solely to avoid implementing the methods by hand. The Point in Meshes.jl is nothing but a wrapped SVector with a correct set of methods defined which is not equal to the set of methods defined for vectors. Treating points as vectors is only one of the many conceptual issues I fixed in Meshes.jl.
Regarding the 33 dependencies I don’t think it shouldn’t be a hard task to migrate these to Meshes.jl if these dependants make good use of the api and don’t abuse the vector methods mentioned above. This is crucial for a correct treatment of geometries in more sophisticated use cases. So yes, I plan to submit PRs to the packages gradually as I find the need in my research. For example MeshIO.jl is one that I really want to fix.
I see that there has been a fair amount of discussion going on with respect to the underlying geometry primitives. I would love to see this converge, but having and adhering to a common interface is maybe the first step. Thanks for the insights so far
Actually it reminds me a bit of Grassmann.jl due to the math backing and seems comparable in speed, but with actual usability due to down-to-earth docs and primitives one can understand. Which mapping did you use between Meshes.jl objects and the common types in GIS (MultiPolygon, LineString etc)?