# Spatial Join with GeometryOps is killing my Julia session

**URL:** <https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863>\
**Category:** Geo\
**Tags:** geospatial\
**Created:** [November 20, 2024, 4:58pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863 "2024-11-20T16:58:31Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![alejandromerchan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alejandromerchan/32/10500_2.png) [@alejandromerchan](https://discourse.julialang.org/u/alejandromerchan)\
**Post date:** [November 20, 2024, 4:58pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/1 "2024-11-20T16:58:31Z")

</div>

I’m trying to perform a spatial join in Julia between two files that aren’t that large. One is the California PLSS sections shapefile, which is publicly available [here](https://calpip.cdpr.ca.gov/content/groundwater/shapefiles/Statewide_CA_PLSS_NAD83AlbersCA_20240718.zip) and is 164627 x 17 after some cleaning. The other I cannot share publicly, but it’s a biodiversity database that is 99316 x 42.  
I perform the spatial join as mentioned in this [tutorial](https://juliageo.org/GeometryOps.jl/dev/tutorials/spatial_joins) as:

`spatial_join = innerjoin((plss, biodiversity), by_pred(:geometry, GeometryOps.touches, :geometry))`

And then REPL dies and I don’t get any error message. This happens both in VS Code and directly in the REPL. Any idea what’s going on? Could this be an OOM problem?

I also tried using `GeoStats.jl`, but I run into issues with the projection when trying to open the shapefiles.

My goal in to have a script that performs an analysis with these two files without opening them in ArcGIS. I managed to do it in R, but I’d like to do the same in Julia, but this part has been the bottleneck. I’ll appreciate any help.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [November 20, 2024, 5:11pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/2 "2024-11-20T17:11:15Z")

</div>

> [@alejandromerchan](#):
>
> I also tried using `GeoStats.jl`, but I run into issues with the projection when trying to open the shapefiles.

Hi @alejandromerchan,

Please feel free to start a separate thread and we can try to help. Adding projections is usually easy. There is only one type of projection not covered yet that we are planning to add soon.

---

<div class="post-metadata">

**Author:** ![alejandromerchan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alejandromerchan/32/10500_2.png) [@alejandromerchan](https://discourse.julialang.org/u/alejandromerchan)\
**Post date:** [November 20, 2024, 5:24pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/3 "2024-11-20T17:24:06Z")

</div>

I don’t mind having the conversation here. The error I’m getting for both my files after running  
`plss = GeoIO.load("./PLSS/Statewide_CA_PLSS_NAD83AlbersCA_20240718.shp")`

is

`ERROR: ArgumentError: EPSG/ESRI code for the ESRI ID "NAD_1983_California_Teale_Albers" not found in dictionary. Please check https://github.com/JuliaEarth/CoordRefSystems.jl/blob/main/src/strings.jl If you know the EPSG/ESRI code of a given ESRI WKT string, please submit a pull request.`

I checked the dictionary and there an `"NAD_1983_Contiguous_USA_Albers" => EPSG{5070},`, which I believe should work, but I’m guessing that you have to provide the exact name.

I’d like to make this work, but I also wonder if there’s a way to use the GeoDataFrame that I already have for the `GeoTables.geojoin` operation.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [November 20, 2024, 5:37pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/4 "2024-11-20T17:37:59Z")

</div>

Whenever you encounter an error like this, go to [epsg.io](http://epsg.io) and copy/paste the ID:

> **[Coordinate reference systems for "NAD\_1983\_California\_Teale\_Albers"](https://epsg.io/?q=NAD_1983_California_Teale_Albers)**
>
> EPSG.io: Coordinate systems worldwide (EPSG/ESRI), preview location on a map, get transformation, WKT, OGC GML, Proj.4. https://EPSG.io/ made by @klokantech

There are two results as you can see:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/f/f/ffee04ceedd839b7a706e1813975e847f7ac95ac.png)

If you click on them, and scroll down, you will see that you can choose to export the `ESRI WKT`. In this first result, the string is

`"NAD_1983_California_Teale_Albers_FtUS"`

 ![image](https://global.discourse-cdn.com/julialang/original/3X/0/a/0ae0a6b4fc1ee75b4743df41e916f61f69f7cfb5.png)

In the second result, the string is

`"NAD_1983_California_Teale_Albers_Ft_Intl"`

 ![image](https://global.discourse-cdn.com/julialang/original/3X/3/3/3393f1a0db47683344587b9eb5abe72a45055486.png)

So none of the two results match the string in your file. Let’s try the official EPSG database at [Geodetic Database](https://epsg.org/search/by-name). If we search for “California Albers”, we get the following result:

[https://epsg.org/crs\_3310/NAD83-California-Albers.html](https://epsg.org/crs_3310/NAD83-California-Albers.html)

If that is the CRS of your data, then you have the EPSG code 3310.

After you confirm which of these is your CRS, we can help with next steps.

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [November 20, 2024, 5:44pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/5 "2024-11-20T17:44:58Z")

</div>

I can confirm that your CRS is [NAD83 / California Albers - EPSG:3310](https://epsg.io/3310)

If you export the ESRI WKT you will see the same string stored in your file.

So the next step to get things loaded correctly is to submit a PR to CoordRefSystems.jl adding the pair `"NAD_1983_California_Teale_Albers" => EPSG{3310}` to the dictionary.

---

<div class="post-metadata">

**Author:** ![alejandromerchan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alejandromerchan/32/10500_2.png) [@alejandromerchan](https://discourse.julialang.org/u/alejandromerchan)\
**Post date:** [November 20, 2024, 5:50pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/6 "2024-11-20T17:50:03Z")

</div>

Besed on the .proj file from one of my files, the last one has the right parameters, except the units are meters.

> PROJCS[“NAD\_1983\_California\_Teale\_Albers”,  
> GEOGCS[“GCS\_North\_American\_1983”,  
> DATUM[“D\_North\_American\_1983”,  
> SPHEROID[“GRS\_1980”,6378137.0,298.257222101]],  
> PRIMEM[“Greenwich”,0.0],  
> UNIT[“Degree”,0.0174532925199433]],  
> PROJECTION[“Albers”],  
> PARAMETER[“False\_Easting”,0.0],  
> PARAMETER[“False\_Northing”,-4000000.0],  
> PARAMETER[“Central\_Meridian”,-120.0],  
> PARAMETER[“Standard\_Parallel\_1”,34.0],  
> PARAMETER[“Standard\_Parallel\_2”,40.5],  
> PARAMETER[“Latitude\_Of\_Origin”,0.0],  
> **UNIT[“Meter”,1.0]]**

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [November 20, 2024, 5:51pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/7 "2024-11-20T17:51:25Z")

</div>

Added the entry to the dictionary here:

> <https://github.com/JuliaEarth/CoordRefSystems.jl/pull/208>

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [November 20, 2024, 5:53pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/8 "2024-11-20T17:53:02Z")

</div>

> [@alejandromerchan](#):
>
> , except the units are meters.

The units of the EPSG:3310 are meters:

 ![image](https://global.discourse-cdn.com/julialang/original/3X/d/1/d1106643cce63f979de15acef36dcb877b7131dc.png)

---

<div class="post-metadata">

**Author:** ![juliohm](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/juliohm/32/215266_2.png) [@juliohm](https://discourse.julialang.org/u/juliohm)\
**Post date:** [November 20, 2024, 6:07pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/9 "2024-11-20T18:07:07Z")

</div>

The last ingredient to make things work is to map the newly added `EPSG{3310}` to an actual CRS type. This is what I did here:

> <https://github.com/JuliaEarth/CoordRefSystems.jl/pull/209>

I will merge the PR and release a patch so that you can load your files.

You should be able to update your environment after the version is merged:

> <https://github.com/JuliaRegistries/General/pull/119843>
>
> \- Registering package: CoordRefSystems
> \- Repository: https://github.com/JuliaEar…th/CoordRefSystems.jl
> \- Created by: @juliohm
> \- Version: v0.15.8
> \- Commit: 5f943c8b3d72ecbc8ee4876a1e2ac57879fb39da
> \- Reviewed by: @juliohm
> \- Reference: https://github.com/JuliaEarth/CoordRefSystems.jl/commit/5f943c8b3d72ecbc8ee4876a1e2ac57879fb39da#commitcomment-149319934
> \- Description: Unitful coordinate reference systems for geographic maps in Julia

Please let us know if something else is not working. Preferably in a separate thread.

---

<div class="post-metadata">

**Author:** ![alejandromerchan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alejandromerchan/32/10500_2.png) [@alejandromerchan](https://discourse.julialang.org/u/alejandromerchan)\
**Post date:** [November 20, 2024, 6:20pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/10 "2024-11-20T18:20:31Z")

</div>

Thanks a lot. I’ll let you know how it goes.

---

<div class="post-metadata">

**Author:** ![evetion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/evetion/32/22679_2.png) [@evetion](https://discourse.julialang.org/u/evetion)\
**Post date:** [November 23, 2024, 8:35pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/11 "2024-11-23T20:35:04Z")

</div>

I’m sorry to hear your REPL dies on the operation.

It does seem like an OOM problem, your OS killing the Julia process for taking too much memory. What OS and how much memory do you have? Depending on the OS you should be able to find this in the logs.

In order to help fix the problem, can you share a bit on: what kind of geometry your biodiversity dataset has (points, polygons, etc.), the code used to read the data (I assume GeoDataFrames), and finally the file format of the biodiversity dataset?

Doing 164627 \* 99316 = 16.350.095.132 touch operations (assuming the brute force way without a spatial index) will always require some memory, but much depends on the geometry sizes and geometry format. And I assume you really want to use `touches` instead of `intersects`?

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [November 24, 2024, 12:56am UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/12 "2024-11-24T00:56:32Z")

</div>

@alejandromerchan good to just make a github issue if you hit a problem like this with GeometryOps.jl, that way it will get fixed and we are more likely to get direct notifications and respond quickly.

---

<div class="post-metadata">

**Author:** ![asinghvi17](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/asinghvi17/32/8272_2.png) [@asinghvi17](https://discourse.julialang.org/u/asinghvi17)\
**Post date:** [November 24, 2024, 1:11pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/13 "2024-11-24T13:11:13Z")

</div>

Hmm, I have noticed that FlexiJoins.jl consumes quite a bit of memory, significantly beyond normal DataFrames joins. @aplavin, any thoughts?

Not sure what the best thing to do is here, although you can always manually join with the predicate. But that loses the tree optimizations we have.

---

<div class="post-metadata">

**Author:** ![aplavin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/aplavin/32/222056_2.png) [@aplavin](https://discourse.julialang.org/u/aplavin)\
**Post date:** [November 24, 2024, 1:40pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/14 "2024-11-24T13:40:43Z")

</div>

> [@asinghvi17](#):
>
> Hmm, I have noticed that [FlexiJoins.jl](https://juliahub.com/ui/Packages/General/FlexiJoins) consumes quite a bit of memory, significantly beyond normal DataFrames joins. @aplavin, any thoughts?

Do you have an MWE? Best if that doesn’t require GeometryOps understanding (:

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [November 24, 2024, 4:00pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/15 "2024-11-24T16:00:51Z")

</div>

Unfortunately we don’t even have a GeometryOps.jl MWE here

@alejandromerchan any chance you could put some MWEs together that trigger this error?

---

<div class="post-metadata">

**Author:** ![alejandromerchan](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/alejandromerchan/32/10500_2.png) [@alejandromerchan](https://discourse.julialang.org/u/alejandromerchan)\
**Post date:** [November 25, 2024, 4:13pm UTC](https://discourse.julialang.org/t/spatial-join-with-geometryops-is-killing-my-julia-session/122863/16 "2024-11-25T16:13:03Z")

</div>

Sorry for the late reply, I don’t check Discourse during the weekend. I’ll try to create a MWE and create an issue. As I said, I cannot share one of the files, but I’ll see if I can remove the sensitive information, I can still reproduce the problem.