Unitful error in `geojoin`

Following on from here…

    geo = GeoIO.load("os_bng_grids.gpkg"), layer=4) # Read the grid gpkg file
    abdf = CSV.read("addresses.csv", DataFrame) # Read the address file
    subset!(abdf, [:X_COORDINATE, :Y_COORDINATE] => ByRow((x, y) -> !ismissing(x) && !ismissing(y)))
    points = Point.(zip(abdf.X_COORDINATE, abdf.Y_COORDINATE)) # Convert (x,y) to geometry
    geoabdf = georef(abdf, points) # join x,y geometry back to main dataset
    println(typeof(geoabdf[1, "geometry"]))
    newdf = geojoin(geoabdf, geo, kind=:left, pred=((g1, g2) -> intersects(g1, g2))) # Match grids to addresses based on geometry intersection
    newabdf = values(newdf)
    println(describe(newabdf))

The geojoin fails and the error seems to materialise in Unitful.

ERROR: LoadError: UndefVarError: `U` not defined
Stacktrace:
  [1] unit
    @ C:\Users\TGebbels\.julia\packages\Unitful\GYzMo\src\utils.jl:119 [inlined]
  [2] _absunit(::Core.TypeofBottom, x::SentinelArrays.MissingVector)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:56
  [3] _absunit(x::SentinelArrays.MissingVector)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:53
  [4] (::GeoTables.var"#39#40"{DataFrames.DataFrameColumns{DataFrame}})(var::Symbol)
    @ GeoTables .\none:0
  [5] iterate
    @ .\generator.jl:47 [inlined]
  [6] merge(a::@NamedTuple{}, itr::Base.Generator{Vector{Symbol}, GeoTables.var"#39#40"{DataFrames.DataFrameColumns{DataFrame}}})
    @ Base .\namedtuple.jl:371
  [7] _adjustunits(tab::DataFrame)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:50
  [8] _adjustunits(geotable::GeoTables.GeoTable{DataFrame})
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:42
  [9] _geojoin(gtb1::GeoTables.GeoTable{DataFrame}, gtb2::GeoTables.GeoTable{@NamedTuple{tile_name::Vector{String}, 1km_grid_ref::Vector{String}}}, selector::ColumnSelectors.NoneSelector, aggfuns::Vector{Function}; kind::Symbol, pred::Function, on::Nothing)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\geojoin.jl:86
 [10] _geojoin
    @ C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\geojoin.jl:45 [inlined]
 [11] #geojoin#45
    @ C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\geojoin.jl:39 [inlined]
 [12] addgridtoaddressbase(OSdatapath::String, Status::OSDatasets.AllDatasets)
    @ OSDatasets C:\Users\TGebbels\...\Documents\Julia\Packages\OSDatasets\src\OSaddressbase.jl:91
 [13] top-level scope
    @ c:\Users\TGebbels\...\Documents\Julia\Packages\Test-OSDatasets.jl:20
 [14] include(fname::String)
    @ Base.MainInclude .\client.jl:489
 [15] run(debug_session::VSCodeDebugger.DebugAdapter.DebugSession, error_handler::VSCodeDebugger.var"#3#4"{String})
    @ VSCodeDebugger.DebugAdapter c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\DebugAdapter\src\packagedef.jl:126
 [16] startdebugger()
    @ VSCodeDebugger c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\VSCodeDebugger\src\VSCodeDebugger.jl:45
 [17] top-level scope
    @ c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\debugger\run_debugger.jl:12
 [18] include(mod::Module, _path::String)
    @ Base .\Base.jl:495
 [19] exec_options(opts::Base.JLOptions)
    @ Base .\client.jl:318
 [20] _start()
    @ Base .\client.jl:552
in expression starting at c:\Users\TGebbels\...\Documents\Julia\Packages\Test-OSDatasets.jl:20
 *  Terminal will be reused by tasks, press any key to close it. 

In this case, the points data are of type

Meshes.Point{Meshes.𝔼 {2}, CoordRefSystems.Cartesian2D{CoordRefSystems.NoDatum, Unitful.Quantity{Float64, 𝐋 , Unitful.FreeUnits{(m,), 𝐋 , nothing}}}}

In another example of the same approach, the points data are of type

@NamedTuple{geometry::Point{𝔼 {2}, Cartesian2D{NoDatum, Quantity{Float64, 𝐋 , Unitful.FreeUnits{(m,), 𝐋 , nothing}}}}}

and in this case the geojoin works. I don’t know why the types are different in these cases but I’m assuming the difference is where the error arises.

What have I done wrong?

Hi @TimG can you reduce the code to a MWE?

It is hard to find the source of the error otherwise.

    using DataFrames
    using GeoIO
    using GeoStats
    using CSV

    geo = GeoIO.load("os_bng_grids.gpkg", layer=4) # Read the grid gpkg file
    abdf = CSV.read("AddressBasePlus_FULL_2024-06-12_010.csv", DataFrame) # Read the address file
    subset!(abdf, [:X_COORDINATE, :Y_COORDINATE] => ByRow((x, y) -> !ismissing(x) && !ismissing(y)))
    points = Point.(zip(abdf.X_COORDINATE, abdf.Y_COORDINATE)) # Convert (x,y) to geometry
    geoabdf = georef(abdf, points) # join x,y geometry back to main dataset
    newdf = geojoin(geoabdf, geo, kind=:left, pred=((g1, g2) -> intersects(g1, g2))) # Match grids to addresses based on geometry intersection
    newabdf = values(newdf)
    println(describe(newabdf))

I’ve pulled the address file out of the zip archive, but there isn’t much more I can do to make an MWE.
The address file is 77 columns by 1000000 rows but is straightforward. Unfortunately it is a proprietary, commercial file from Ordnance Survey (the UK mapping agency), so I can’t share it.

I still get the same error:

ERROR: LoadError: UndefVarError: `U` not defined
Stacktrace:
  [1] unit
    @ C:\Users\TGebbels\.julia\packages\Unitful\GYzMo\src\utils.jl:119 [inlined]
  [2] _absunit(::Core.TypeofBottom, x::SentinelArrays.MissingVector)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:56
  [3] _absunit(x::SentinelArrays.MissingVector)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:53
  [4] (::GeoTables.var"#39#40"{DataFrames.DataFrameColumns{DataFrame}})(var::Symbol)
    @ GeoTables .\none:0
  [5] iterate
    @ .\generator.jl:47 [inlined]
  [6] merge(a::@NamedTuple{}, itr::Base.Generator{Vector{Symbol}, GeoTables.var"#39#40"{DataFrames.DataFrameColumns{DataFrame}}})
    @ Base .\namedtuple.jl:371
  [7] _adjustunits(tab::DataFrame)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:50
  [8] _adjustunits(geotable::GeoTable{DataFrame})
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\utils.jl:42
  [9] _geojoin(gtb1::GeoTable{DataFrame}, gtb2::GeoTable{@NamedTuple{tile_name::Vector{String}, 1km_grid_ref::Vector{String}}}, selector::ColumnSelectors.NoneSelector, aggfuns::Vector{Function}; kind::Symbol, pred::Function, on::Nothing)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\geojoin.jl:86
 [10] _geojoin
    @ C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\geojoin.jl:45 [inlined]
 [11] #geojoin#45
    @ C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\geoops\geojoin.jl:39 [inlined]
 [12] top-level scope
    @ c:\Users\TGebbels\...\Documents\Julia\Julia Experimenting\geojoin-mwe.jl:14
 [13] include(fname::String)
    @ Base.MainInclude .\client.jl:489
 [14] run(debug_session::VSCodeDebugger.DebugAdapter.DebugSession, error_handler::VSCodeDebugger.var"#3#4"{String})
    @ VSCodeDebugger.DebugAdapter c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\DebugAdapter\src\packagedef.jl:126
 [15] startdebugger()
    @ VSCodeDebugger c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\VSCodeDebugger\src\VSCodeDebugger.jl:45
 [16] top-level scope
    @ c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\debugger\run_debugger.jl:12
 [17] include(mod::Module, _path::String)
    @ Base .\Base.jl:495
 [18] exec_options(opts::Base.JLOptions)
    @ Base .\client.jl:318
 [19] _start()
    @ Base .\client.jl:552
in expression starting at c:\Users\TGebbels\...\Documents\Julia\Julia Experimenting\geojoin-mwe.jl:14

Can you please share the output of describe(geoabdf)? Any additional information that you can share about this file? crs(geoabdf.geometry)?

Cartesian2D{NoDatum, Quantity{Float64, 𝐋 , Unitful.FreeUnits{(m,), 𝐋 , nothing}}}

Describing the geotable seems to include a lot of the actual data in the output, so I don’t feel able to share. Its just a big block of stuff, so it’s hard to read - maybe it only contains the same info as describing a dataframe. Here is a snippet:

@NamedTuple{variable::String, mean::Union{Missing, Nothing, Float64}, minimum, median::Union{Missing, Nothing, Float64}, maximum, nmissing::Int64}[@NamedTuple{variable::String, mean::Union{Missing, Nothing, Float64}, minimum, median::Union{Missing, Nothing, Float64}, maximum, nmissing::Int64}(("UPRN", 6.821843963482906e10, 70.0, 1.0001e11, 9.06701e11, 0)), @NamedTuple{variable::String, mean::Union{Missing, Nothing, Float64}, minimum, median::Union{Missing, Nothing, Float64}, maximum, nmissing::Int64}(("UDPRN", nothing, CategoricalValue{Int64, UInt32} 9, nothing, CategoricalValue{Int64, UInt32} 57741501, 26893)), @NamedTuple{variable::String, mean::Union{Missing, Nothing, Float64}, minimum, median::Union{Missing, Nothing, Float64}, maximum, nmissing::Int64}(("CHANGE_TYPE", nothing, CategoricalValue{String1, UInt32} String1("I"), nothing, CategoricalValue{String1, UInt32} String1("I"), 0)), 

Here is the output of describe(abdf)

77Γ—7 DataFrame
 Row β”‚ variable                         mean        min                                median      max                                nmissing  eltype
     β”‚ Symbol                           Union…      Any                                Union…      Any                                Int64     Type
─────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
   1 β”‚ UPRN                             6.82184e10  70.0                               1.0001e11   9.06701e11                                0  Float64
   2 β”‚ UDPRN                            2.02082e7   9                                  1.69199e7   57741501                              26893  Union{Missing, Int64}
   3 β”‚ CHANGE_TYPE                                  I                                              I                                         0  String1
   4 β”‚ STATE                            2.00172     1                                  2.0         6                                        13  Union{Missing, Int64}
   5 β”‚ STATE_DATE                                   01/01/1923                                     31/12/2022                               13  Union{Missing, String15}
   6 β”‚ CLASS                                        C                                              ZW99SY                                    0  String7
   7 β”‚ PARENT_UPRN                      7.2639e10   481.0                              1.00339e10  9.06701e11                           818733  Union{Missing, Float64}
   8 β”‚ X_COORDINATE                     4.31573e5   64954.0                            4.33116e5   655330.0                                  0  Float64
   9 β”‚ Y_COORDINATE                     3.13661e5   10201.0                            280818.0    1.21622e6                                 0  Float64
  10 β”‚ LATITUDE                         52.7107     49.9122                            52.4194     60.8243                                   0  Float64
  11 β”‚ LONGITUDE                        -1.55689    -7.51589                           -1.50486    1.75874                                   0  Float64
  12 β”‚ RPC                              1.41578     1                                  1.0         9                                         0  Int64
  13 β”‚ LOCAL_CUSTODIAN_CODE             3809.6      114                                3615.0      9079                                      0  Int64
  14 β”‚ COUNTRY                                      E                                              W                                         0  String1
  15 β”‚ LA_START_DATE                                01/01/2009                                     31/12/2022                                0  String15
  16 β”‚ LAST_UPDATE_DATE                             01/01/2017                                     30/09/2021                                0  String15
  17 β”‚ ENTRY_DATE                                   01/01/1900                                     31/12/2021                                0  String15
  18 β”‚ RM_ORGANISATION_NAME                         & THE DISH RAN AWAY WITH THE SPO…              ZYNK SOFTWARE LTD                    964114  Union{Missing, String}
  19 β”‚ LA_ORGANISATION                              #HAIR & COSMETICS                              Β£5 OR                                971720  Union{Missing, String}
  20 β”‚ DEPARTMENT_NAME                              1 WESSEX REGIMENT (RV) C COY                   YORKSHIRE & THE HUMBER               999702  Union{Missing, String}
  21 β”‚ LEGAL_NAME                                   ANGUS DUNDEE DISTILLERS PLC                    SPECSAVERS OPTICAL SUPERSTORES L…    999989  Union{Missing, String}
  22 β”‚ SUB_BUILDING_NAME                            (BROUGH)                                       ZION BUNGALOW                        897414  Union{Missing, String31}
  23 β”‚ BUILDING_NAME                                (ANDERSON)                                     ZULU FARM COTTAGE                    793418  Union{Missing, String}
  24 β”‚ BUILDING_NUMBER                  48.4367     1                                  24.0        7400                                 213138  Union{Missing, Int64}
  25 β”‚ SAO_START_NUMBER                 34.6874     1                                  10.0        7600                                 933475  Union{Missing, Int64}
  26 β”‚ SAO_START_SUFFIX                             A                                              T                                    998307  Union{Missing, String1}
  27 β”‚ SAO_END_NUMBER                   28.2759     1                                  8.0         609                                  999681  Union{Missing, Int64}
  28 β”‚ SAO_END_SUFFIX                               A                                              J                                    999973  Union{Missing, String1}
  29 β”‚ SAO_TEXT                                     (JOHNSTON) HALF OF 7                           ZONE 6                               895695  Union{Missing, String}
  30 β”‚ ALT_LANGUAGE_SAO_TEXT                        (SO 914-295) NTL COMMNICATION ST…              ZION BUNGALOW                        997090  Union{Missing, String}
  31 β”‚ PAO_START_NUMBER                 49.8531     1                                  24.0        7400                                 171462  Union{Missing, Int64}
  32 β”‚ PAO_START_SUFFIX                             A                                              Y                                    967918  Union{Missing, String1}
  33 β”‚ PAO_END_NUMBER                   84.5488     1                                  40.0        1764                                 991347  Union{Missing, Int64}
  34 β”‚ PAO_END_SUFFIX                               A                                              J                                    999840  Union{Missing, String1}
  35 β”‚ PAO_TEXT                                     (DEVLIN)                                       ZURICH HOUSE                         776157  Union{Missing, String}
  36 β”‚ ALT_LANGUAGE_PAO_TEXT                        1 NEW HOUSE                                    ZONE A ECLIPSE                       985840  Union{Missing, String}
  37 β”‚ USRN                             2.63423e7   200026                             2.41002e7   86000007                                  0  Int64
  38 β”‚ USRN_MATCH_INDICATOR             1.00666     1                                  1.0         2                                         0  Int64
  39 β”‚ AREA_NAME                                                                                                                       1000000  Missing
  40 β”‚ LEVEL                                        -1                                             9                                    804032  Union{Missing, String15}
  41 β”‚ OFFICIAL_FLAG                                N                                              Y                                    340685  Union{Missing, String1}
  42 β”‚ OS_ADDRESS_TOID                              osgb1000002115000030                           osgb5000005165447813                 103743  Union{Missing, String31}
  43 β”‚ OS_ADDRESS_TOID_VERSION          8.00428     1                                  7.0         84                                   103743  Union{Missing, Int64}
  44 β”‚ OS_ROADLINK_TOID                             osgb4000000003210421                           osgb5000005333564248                      0  String31
  45 β”‚ OS_ROADLINK_TOID_VERSION         3.40721     0                                  3.0         35                                        0  Int64
  46 β”‚ OS_TOPO_TOID                                 osgb1000000000063                              osgb5000005334133877                      2  Union{Missing, String31}
  47 β”‚ OS_TOPO_TOID_VERSION             3.56008     1                                  3.0         298                                       2  Union{Missing, Int64}
  48 β”‚ VOA_CT_RECORD                    1.91322e9   1025                               1.90485e8   14717752000                          171902  Union{Missing, Int64}
  49 β”‚ VOA_NDR_RECORD                   4.79055e9   17163                              2.7951e9    14711903000                          965866  Union{Missing, Int64}
  50 β”‚ STREET_DESCRIPTION                           10TH AVENUE                                    ZW9020                                    0  String
  51 β”‚ ALT_LANGUAGE_STREET_DESCRIPTION              32.65 ROW WATERY LANE                          ZW9020                               952443  Union{Missing, String}
  52 β”‚ DEPENDENT_THOROUGHFARE                       A P S INDUSTRIAL PARK                          ZETLAND COURT                        985334  Union{Missing, String}
  53 β”‚ THOROUGHFARE                                 10TH AVENUE                                    ZURICH AVENUE                         55111  Union{Missing, String}
  54 β”‚ WELSH_DEPENDENT_THOROUGHFARE                 A R D BUSINESS PARK                            YR ERW                               999129  Union{Missing, String}
  55 β”‚ WELSH_THOROUGHFARE                           ABBEY COURT                                    ZOAR ROAD                            955146  Union{Missing, String}
  56 β”‚ DOUBLE_DEPENDENT_LOCALITY                    ABBOTS BICKINGTON                              ZONE 3                               995319  Union{Missing, String}
  57 β”‚ DEPENDENT_LOCALITY                           AB KETTLEBY                                    ZOUCH                                675025  Union{Missing, String}
  58 β”‚ LOCALITY                                     ABBESS BEAUCHAMP AND BERNERS ROD…              YTHANWELLS                           782334  Union{Missing, String}
  59 β”‚ WELSH_DEPENDENT_LOCALITY                     ABATY CWM-HIR                                  YSTRADOWEN                           973247  Union{Missing, String}
  60 β”‚ WELSH_DOUBLE_DEPENDENT_LOCALITY              ABEREIDDY                                      YNYSMEUDWY                           999268  Union{Missing, String31}
  61 β”‚ TOWN_NAME                                    AB KETTLEBY                                    ZOUCH                                    87  Union{Missing, String31}
  62 β”‚ ADMINISTRATIVE_AREA                          ABERDEEN CITY                                  YORK                                      0  String31
  63 β”‚ POST_TOWN                                    ABBOTS LANGLEY                                 YSTRAD MEURIG                         26893  Union{Missing, String31}
  64 β”‚ WELSH_POST_TOWN                              ABERAERON                                      YSTRAD MEURIG                        950219  Union{Missing, String31}
  65 β”‚ POSTCODE                                     AB10 1AN                                       ZE3 9JX                               26893  Union{Missing, String15}
  66 β”‚ POSTCODE_LOCATOR                             AB10 1AN                                       ZE3 9JX                                   0  String15
  67 β”‚ POSTCODE_TYPE                                L                                              S                                     26893  Union{Missing, String1}
  68 β”‚ DELIVERY_POINT_SUFFIX                        1A                                             9T                                    26893  Union{Missing, String3}
  69 β”‚ ADDRESSBASE_POSTAL                           C                                              N                                         0  String1
  70 β”‚ PO_BOX_NUMBER                                1                                              9994                                 998990  Union{Missing, String7}
  71 β”‚ WARD_CODE                                    E05000932                                      W05001800                                 5  Union{Missing, String15}
  72 β”‚ PARISH_CODE                                  E04000001                                      W04001085                            577481  Union{Missing, String15}
  73 β”‚ RM_START_DATE                                01/01/2019                                     31/12/2021                            26893  Union{Missing, String15}
  74 β”‚ MULTI_OCC_COUNT                  0.08942     0                                  0.0         507                                       0  Int64
  75 β”‚ VOA_NDR_P_DESC_CODE                          CA                                             TX                                   965866  Union{Missing, String7}
  76 β”‚ VOA_NDR_SCAT_CODE                199.207     3                                  203.0       999                                  966219  Union{Missing, Int64}
  77 β”‚ ALT_LANGUAGE                                 CYM                                            CYM                                  952433  Union{Missing, String3}

I think you will have to figure this one out yourself given the data privacy issues.

Try to provide the crs option in the georef call with the CSV file, to make sure that both geotables are georeferrenced on the same system:

georef(csv, ("x", "y"), crs = crs(geotable.geometry))

If that solves the issue, then we need to fix something on our side.

No, that fails…

#points = Point.(zip(abdf.X_COORDINATE, abdf.Y_COORDINATE)) # Convert (x,y) to geometry
geoabdf = georef(abdf, ("X_COORDINATE", "Y_COORDINATE"); crs = crs(geo.geometry)) # join x,y geometry back to main dataset

with the following error

ERROR: LoadError: MethodError: no method matching CoordRefSystems.ShiftedCRS{CoordRefSystems.TransverseMercator{0.9996012717, Quantity{Float64, Unitful.Dimensions{()}(), Unitful.FreeUnits{(Unitful.Unit{:Degree, Unitful.Dimensions{()}()}(0, 1//1),), Unitful.Dimensions{()}(), nothing}}(49.0), Quantity{Float64, Unitful.Dimensions{()}(), Unitful.FreeUnits{(Unitful.Unit{:Degree, Unitful.Dimensions{()}()}(0, 1//1),), Unitful.Dimensions{()}(), nothing}}(-2.0), OSGB36, Quantity{Float64, Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}(), Unitful.FreeUnits{(Unitful.Unit{:Meter, Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}()}(0, 1//1),), Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}(), nothing}}}, Quantity{Float64, Unitful.Dimensions{()}(), Unitful.FreeUnits{(Unitful.Unit{:Degree, Unitful.Dimensions{()}()}(0, 1//1),), Unitful.Dimensions{()}(), nothing}}(0.0), Quantity{Float64, Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}(), Unitful.FreeUnits{(Unitful.Unit{:Meter, Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}()}(0, 1//1),), Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}(), nothing}}(400000.0), Quantity{Float64, Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}(), Unitful.FreeUnits{(Unitful.Unit{:Meter, Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}()}(0, 1//1),), Unitful.Dimensions{(Unitful.Dimension{:Length}(1//1),)}(), nothing}}(-100000.0), OSGB36}(::Float64, ::Float64)

Closest candidates are:
  (::Type{CoordRefSystems.ShiftedCRS{CRS, lonβ‚’, xβ‚’, yβ‚’, Datum}} where {CRS, lonβ‚’, xβ‚’, yβ‚’, Datum})(::Any)
   @ CoordRefSystems C:\Users\TGebbels\.julia\packages\CoordRefSystems\R3eqw\src\crs\projected\shifted.jl:12

Stacktrace:
  [1] (::GeoTables.var"#point#119"{DataType})(::Float64, ::Vararg{Float64})
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\georef.jl:81
  [2] #4
    @ .\generator.jl:36 [inlined]
  [3] iterate
    @ .\generator.jl:47 [inlined]
  [4] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{SentinelArrays.ChainedVector{Float64, Vector{Float64}}, SentinelArrays.ChainedVector{Float64, Vector{Float64}}}}, Base.var"#4#5"{GeoTables.var"#point#119"{DataType}}}) 
    @ Base .\array.jl:834
  [5] map(::Function, ::SentinelArrays.ChainedVector{Float64, Vector{Float64}}, ::SentinelArrays.ChainedVector{Float64, Vector{Float64}})
    @ Base .\abstractarray.jl:3409
  [6] georef(table::DataFrame, names::Vector{Symbol}; crs::Type)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\georef.jl:82
  [7] georef
    @ C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\georef.jl:70 [inlined]
  [8] georef(table::DataFrame, names::Tuple{String, String}; crs::Type)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\8up7e\src\georef.jl:94
  [9] top-level scope
    @ c:\Users\TGebbels\...\Documents\Julia\Julia Experimenting\geojoin-mwe.jl:14
 [10] include(fname::String)
    @ Base.MainInclude .\client.jl:489
 [11] run(debug_session::VSCodeDebugger.DebugAdapter.DebugSession, error_handler::VSCodeDebugger.var"#3#4"{String})
    @ VSCodeDebugger.DebugAdapter c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\DebugAdapter\src\packagedef.jl:126
 [12] startdebugger()
    @ VSCodeDebugger c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\VSCodeDebugger\src\VSCodeDebugger.jl:45
 [13] top-level scope
    @ c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\debugger\run_debugger.jl:12
 [14] include(mod::Module, _path::String)
    @ Base .\Base.jl:495
 [15] exec_options(opts::Base.JLOptions)
    @ Base .\client.jl:318
 [16] _start()
    @ Base .\client.jl:552
in expression starting at c:\Users\TGebbels\...\Documents\Julia\Julia Experimenting\geojoin-mwe.jl:14

This file has the format of the original address file but I’ve changed the content and reduced it to just a few rows. It still exhibits the problem. Hopefully, this’ll allow you to recreate the issue.

UPRN,UDPRN,CHANGE_TYPE,STATE,STATE_DATE,CLASS,PARENT_UPRN,X_COORDINATE,Y_COORDINATE,LATITUDE,LONGITUDE,RPC,LOCAL_CUSTODIAN_CODE,COUNTRY,LA_START_DATE,LAST_UPDATE_DATE,ENTRY_DATE,RM_ORGANISATION_NAME,LA_ORGANISATION,DEPARTMENT_NAME,LEGAL_NAME,SUB_BUILDING_NAME,BUILDING_NAME,BUILDING_NUMBER,SAO_START_NUMBER,SAO_START_SUFFIX,SAO_END_NUMBER,SAO_END_SUFFIX,SAO_TEXT,ALT_LANGUAGE_SAO_TEXT,PAO_START_NUMBER,PAO_START_SUFFIX,PAO_END_NUMBER,PAO_END_SUFFIX,PAO_TEXT,ALT_LANGUAGE_PAO_TEXT,USRN,USRN_MATCH_INDICATOR,AREA_NAME,LEVEL,OFFICIAL_FLAG,OS_ADDRESS_TOID,OS_ADDRESS_TOID_VERSION,OS_ROADLINK_TOID,OS_ROADLINK_TOID_VERSION,OS_TOPO_TOID,OS_TOPO_TOID_VERSION,VOA_CT_RECORD,VOA_NDR_RECORD,STREET_DESCRIPTION,ALT_LANGUAGE_STREET_DESCRIPTION,DEPENDENT_THOROUGHFARE,THOROUGHFARE,WELSH_DEPENDENT_THOROUGHFARE,WELSH_THOROUGHFARE,DOUBLE_DEPENDENT_LOCALITY,DEPENDENT_LOCALITY,LOCALITY,WELSH_DEPENDENT_LOCALITY,WELSH_DOUBLE_DEPENDENT_LOCALITY,TOWN_NAME,ADMINISTRATIVE_AREA,POST_TOWN,WELSH_POST_TOWN,POSTCODE,POSTCODE_LOCATOR,POSTCODE_TYPE,DELIVERY_POINT_SUFFIX,ADDRESSBASE_POSTAL,PO_BOX_NUMBER,WARD_CODE,PARISH_CODE,RM_START_DATE,MULTI_OCC_COUNT,VOA_NDR_P_DESC_CODE,VOA_NDR_SCAT_CODE,ALT_LANGUAGE
10010179499,10814737,I,2,19/07/2002,CR09,,495978,201413,51.7032657,-0.6124841,2,440,E,12/12/2007,28/02/2023,19/07/2002,BLAH BLAH,BLAH BLAH,,,,,18,,,,,,,18,,,,,,7300552,1,,,Y,osgb1000002185811105,13,osgb5000005302535768,0,osgb1000018546611,4,,148099162,BLAH BLAH,,,BLAH BLAH,,,,,,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,HP5 1EZ,HP5 1EZ,S,1B,D,,E05013137,E04001569,19/03/2012,0,CS,249,
10010179499,18653210,I,2,04/12/2008,RD04,,541782,298370,52.5647089,0.090338,2,515,E,18/01/2008,20/05/2023,16/04/2001,,,,,,,2,,,,,,,2,,,,,,14502001,1,,,Y,osgb1000002277732691,4,osgb4000000027944135,4,osgb1000009964114,5,451031012,,BLAH BLAH,,,BLAH BLAH,,,,,,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,PE15 8SS,PE15 8SS,S,1H,D,,E05015427,E04001661,19/03/2012,0,,,
10010179499,23410983,I,2,06/12/2007,RD04,,568344,190419,51.5874812,0.4286444,1,1505,E,18/01/2008,28/05/2024,06/12/2007,,,,,,,12,,,,,,,12,,,,,,2002207,1,,,,osgb1000002223332672,9,osgb5000005320682746,0,osgb1000000975101,2,887842022,,BLAH BLAH,,,BLAH BLAH,,,,BLAH BLAH,BLAH BLAH,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,SS15 4FA,SS15 4FA,S,2A,D,,E05015639,,19/03/2012,0,,,
10010179499,4927951,I,2,16/04/2001,RD02,,579255,212015,51.7780985,0.5970503,2,1510,E,13/03/2008,26/07/2020,16/04/2001,,,,,,,19,,,,,,,19,,,,,,3902160,1,,,Y,osgb1000002278083444,5,osgb4000000027968639,4,osgb1000004723240,4,1153438023,,BLAH BLAH,,,BLAH BLAH,,,,BLAH BLAH,,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,CM3 2DA,CM3 2DA,S,1L,D,,E05012964,E04012932,19/03/2012,0,,,
10010179499,23526015,I,2,16/04/2001,RD03,,581706,187123,51.5537312,0.6196076,2,1520,E,15/10/2007,28/05/2024,16/04/2001,,,,,,,18,,,,,,,18,,,,,,6300633,1,,,,osgb1000002223617368,6,osgb4000000030164906,2,osgb1000001540977,4,277627023,,BLAH BLAH,,,BLAH BLAH,,,,,BLAH BLAH,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,SS7 2HP,SS7 2HP,S,1F,D,,E05015697,,19/03/2012,0,,,
10010179499,5019676,I,2,21/11/2002,RD03,,571286,195505,51.6322832,0.4735494,1,1525,E,26/02/2008,28/05/2024,21/11/2002,,,,,,,14,,,,,,,14,,,,,,6601588,1,,,Y,osgb1000002223385527,7,osgb4000000030274539,3,osgb1000001078541,5,61810022,,BLAH BLAH,,,BLAH BLAH,,,,BLAH BLAH,BLAH BLAH,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,CM11 1NY,CM11 1NY,S,1G,D,,E05004112,E04003971,19/03/2012,0,,,
10010179499,11287438,I,2,16/08/2004,RD06,10035035617,409069,422808,53.7016441,-1.8641099,2,4710,E,07/02/2008,28/02/2023,16/08/2004,,,,,BLAH BLAH,BLAH BLAH,,,,,,BLAH BLAH,,,,,,BLAH BLAH,,5306563,1,,2,Y,osgb1000002128552130,12,osgb4000000010753033,8,osgb1000002037998331,2,3338933000,,BLAH BLAH,,,BLAH BLAH,,,,BLAH BLAH,BLAH BLAH,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,HX3 0RN,HX3 0RN,S,1P,D,,E05001383,,19/03/2012,0,,,
12010456389,23039953,I,2,09/10/2017,CO01,10010456374,416790,141647,51.1738131,-1.7611999,1,3940,E,10/12/2007,26/07/2022,03/09/2007,BLAH BLAH,,,,BLAH BLAH,BLAH BLAH,,,,,,BLAH BLAH,,,,,,BLAH BLAH,,33502282,1,,,Y,osgb1000002119591075,13,osgb4000000023270147,6,osgb1000002109143665,2,,2437797000,BLAH BLAH,,,BLAH BLAH,,,,BLAH BLAH,BLAH BLAH,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,SP4 7SD,SP4 7SD,S,1U,D,,E05013400,E04011623,19/03/2012,0,CO,203,
10062116711,27723586,I,2,10/10/2005,RD06,,581519,214009,51.7952791,0.6308741,2,1510,E,13/03/2008,28/02/2023,10/10/2005,,,,,,,31,,,,,,,31,,,,,,3902918,1,,1,Y,osgb1000002278304265,10,osgb4000000028289148,2,osgb1000002314277642,7,3908301000,,BLAH BLAH,,,BLAH BLAH,,,,,,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,CM8 1GW,CM8 1GW,S,1U,D,,E05012966,E04012935,19/03/2012,0,,,
10012918120,50417803,I,2,19/02/2007,RD02,,454681,367966,53.2059785,-1.1827785,1,3025,E,23/10/2007,20/05/2023,19/02/2007,,,,,,,10,,,,,,,10,,,,,,24622060,1,,,,osgb1000002309113727,5,osgb5000005217919494,1,osgb1000002112845671,1,6059604000,,BLAH BLAH,,,BLAH BLAH,,,,BLAH BLAH,,,,BLAH BLAH,BLAH BLAH,BLAH BLAH,,NG20 8YH,NG20 8YH,S,3A,D,,E05014643,E04007876,19/03/2012,0,,,

I can reproduce the error now. Will take a look into it.

1 Like

@TimG some columns of your table are completely filled with missing. That was causing an internal error during aggregation of unitful values.

A patch release should be available soon with the fix:

3 Likes

Thanks very much for this @juliohm.

I tried again with

GeoTables v1.23.0
Meshes v0.48.1

This line

    geoabdf = georef(abdf, ("X_COORDINATE", "Y_COORDINATE"); crs=crs(geo.geometry))

still failed in the same way as shown above.

I also tried:

    geoabdf = georef(abdf, ("X_COORDINATE", "Y_COORDINATE"); crs=EPSG{27700})

but this gave the following:

ERROR: LoadError: MethodError: no constructors have been defined for CoordRefSystems.EPSG{27700}
Stacktrace:
  [1] (::GeoTables.var"#point#117"{DataType})(::Float64, ::Vararg{Float64})
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\y9u5Y\src\georef.jl:81
  [2] #4
    @ .\generator.jl:36 [inlined]
  [3] iterate
    @ .\generator.jl:47 [inlined]
  [4] collect(itr::Base.Generator{Base.Iterators.Zip{Tuple{SentinelArrays.ChainedVector{Union{Missing, Float64}, SentinelArrays.SentinelVector{Float64, Float64, Missing, Vector{Float64}}}, SentinelArrays.ChainedVector{Union{Missing, Float64}, SentinelArrays.SentinelVector{Float64, Float64, Missing, Vector{Float64}}}}}, Base.var"#4#5"{GeoTables.var"#point#117"{DataType}}})
    @ Base .\array.jl:834
  [5] map(::Function, ::SentinelArrays.ChainedVector{Union{Missing, Float64}, SentinelArrays.SentinelVector{Float64, Float64, Missing, Vector{Float64}}}, ::SentinelArrays.ChainedVector{Union{Missing, Float64}, SentinelArrays.SentinelVector{Float64, Float64, Missing, Vector{Float64}}})
    @ Base .\abstractarray.jl:3409
  [6] georef(table::DataFrame, names::Vector{Symbol}; crs::Type)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\y9u5Y\src\georef.jl:82
  [7] georef
    @ C:\Users\TGebbels\.julia\packages\GeoTables\y9u5Y\src\georef.jl:70 [inlined]
  [8] georef(table::DataFrame, names::Tuple{String, String}; crs::Type)
    @ GeoTables C:\Users\TGebbels\.julia\packages\GeoTables\y9u5Y\src\georef.jl:94
  [9] georef
    @ C:\Users\TGebbels\.julia\packages\GeoTables\y9u5Y\src\georef.jl:94 [inlined]
 [10] addgridtoaddressbase2(OSdatapath::String, Status::OSDatasets.AllDatasets)
    @ OSDatasets C:\Users\TGebbels\...\Documents\Julia\Packages\OSDatasets\src\OSaddressbase.jl:99
 [11] top-level scope
    @ c:\Users\TGebbels\...\Documents\Julia\Packages\Test-OSDatasets.jl:25
 [12] include(fname::String)
    @ Base.MainInclude .\client.jl:489
 [13] run(debug_session::VSCodeDebugger.DebugAdapter.DebugSession, error_handler::VSCodeDebugger.var"#3#4"{String})
    @ VSCodeDebugger.DebugAdapter c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\DebugAdapter\src\packagedef.jl:126
 [14] startdebugger()
    @ VSCodeDebugger c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\packages\VSCodeDebugger\src\VSCodeDebugger.jl:45
 [15] top-level scope
    @ c:\Users\TGebbels\.vscode\extensions\julialang.language-julia-1.105.2\scripts\debugger\run_debugger.jl:12
 [16] include(mod::Module, _path::String)
    @ Base .\Base.jl:495
 [17] exec_options(opts::Base.JLOptions)
    @ Base .\client.jl:318
 [18] _start()
    @ Base .\client.jl:552
in expression starting at c:\Users\TGebbels\...\Documents\Julia\Packages\Test-OSDatasets.jl:25

Just for good measure, I also tried

geoabdf = georef(abdf, ("X_COORDINATE", "Y_COORDINATE"); crs=British_National_Grid)

but that simply said ERROR: LoadError: UndefVarError: 'British_National_Grid' not defined.

On the other hand, using LatLon as the crs, does seem to work

geoabdf = georef(abdf, ("LATITUDE", "LONGITUDE"); crs=LatLon)

But in this case the geojoin has to convert between LatLon to BNG each time.

Am I still doing something wrong?

Hi @TimG you dont need to pass the crs option anymore. The latest version of geojoin will adjust the CRS of the input tables if necessary.

In any case, we also added the option to pass a EPSG or ESRI code in georef. Should be available soon:

1 Like