libnetcdf not defined on M1

On a M1 Macbook I get errors trying to use NetCDF from Julia. Any idea which libnetcdf it is looking for? Isn’t NetCDF in Julia self-sufficient? GitHub - JuliaGeo/NetCDF.jl: NetCDF support for the julia programming language does not mention any external system package dependencies.

Thank you.

[~]᳅ julia
_
_ _ ()_ | Documentation: https://docs.julialang.org
() | () () |
_ _ | | __ _ | Type “?” for help, “]?” for Pkg help.
| | | | | | |/ ` | |
| | |
| | | | (
| | | Version 1.7.0 (2021-11-30)
/ |_|||_’_| | Official https://julialang.org/ release
|__/ |

(@v1.7) pkg> add NetCDF
Updating registry at ~/.julia/registries/General.toml
Resolving package versions…
Updating ~/.julia/environments/v1.7/Project.toml
[30363a11] + NetCDF v0.11.4
Updating ~/.julia/environments/v1.7/Manifest.toml
[3c3547ce] + DiskArrays v0.3.0
[59287772] + Formatting v0.4.2
[692b3bcd] + JLLWrappers v1.4.0
[30363a11] + NetCDF v0.11.4
[21216c6a] + Preferences v1.2.3
[0234f1f7] + HDF5_jll v1.12.0+1
[7243133f] + NetCDF_jll v400.702.400+0
[458c3c95] + OpenSSL_jll v1.1.13+0

julia> using NetCDF

julia> ncinfo(“test.nc”)
ERROR: UndefVarError: libnetcdf not defined
Stacktrace:
[1] nc_open
@ ~/.julia/packages/NetCDF/CS38M/src/netcdf_c.jl:55 [inlined]
[2] nc_open(fname::String, omode::UInt16)
@ NetCDF ~/.julia/packages/NetCDF/CS38M/src/netcdf_helpers.jl:103
[3] open(fil::String; mode::UInt16, readdimvar::Bool, add_finalizer::Bool)
@ NetCDF ~/.julia/packages/NetCDF/CS38M/src/NetCDF.jl:913
[4] open(f::typeof(show), args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
@ NetCDF ~/.julia/packages/NetCDF/CS38M/src/NetCDF.jl:999
[5] open
@ ~/.julia/packages/NetCDF/CS38M/src/NetCDF.jl:999 [inlined]
[6] ncinfo(fil::String)
@ NetCDF ~/.julia/packages/NetCDF/CS38M/src/NetCDF.jl:1101
[7] top-level scope
@ REPL[4]:1

It’s not yet supported (except as mentioned in the end). NetCDF.jl has a dependency on NetCDF_jll. JLL packages, are often written in C (which CAN possibly support M1, not a guarantee independently of Julia), to me JLL is “code” for C, but could be other languages.

When people say ARM, or M1, is supported by Julia, it means for pure Julia code (one reason to have that, I still like the ability, and no reason not to use with C, except security, and might not be supported), and Julia itself.

There you can see under Platforms, e.g.

  • macOS x86_64 ( x86_64-apple-darwin )

You probably can use x86 Julia (and then indirectly the Rosetta transpiler).

I’m not sure what it takes to get NetCDF_jll updated. It (and other JLLs, but HDF5_jll and OpenSSL_jll are already taken care of, so this seems to be your last hurdle for that package) need to compile too for M1. It’s clearly not automatic, but might be around the corner. Maybe you can help with (trying) compiling libnetcdf and/or investigating.

In that case I noticed the JLL dependency in the Projects.toml file, but it could have been a dependency of one of those… so I use JuliaHub to see all the dependencies usually, if I need to. It should work, but isn’t always updated with some info, so the Manifest file might be a better place to look…

Currently, the NetCDF JLL package (the compiled binary) isn’t built for M1: Yggdrasil/common.jl at a9b4210baf9e548f30cf77426c8b2b58104341ed · JuliaPackaging/Yggdrasil · GitHub

Hopefully this is just a matter of updating the build script.

Looks like the limiting factor is HDF5, which doesn’t yet support cross-compiling: [HDF5] Cross-compile for all platforms by giordano · Pull Request #567 · JuliaPackaging/Yggdrasil · GitHub

However, it looks like HDF5 now has precompiled binaries for MacOS on ARM via conda-forge, so once the HDF5 build script is updated, then it should be possible to update NetCDF to build on MacOS/ARM as well.

It is already

It seems we/you, know why this didn’t work out of the box, AND the solution(s). I was going by:

HDF5_jll.jl is available for the following platforms:

  • macOS aarch64 ( aarch64-apple-darwin ) […]

Still, this is sub-optimal for new users. I’m thinking, did the user get a warning? NetCDF_jll was seemingly installed with no (working) support. Should it not have installed? I’m not sure that’s possible, at best to install, and provide a warning on install time. Then that package needs to implement it, or strictly something else (Yggrasil?) for all of them.

I was curious if we do better/worse than Python, it has the same problem:

In general, Python (and R) has a lot of C/otherlang or C++ (very common for R, less so for Python?) dependencies. That works very smoothly most of the time, and users may not even know that Python isn’t doing the heavy lifting. For new platforms, such as M1, maybe they have greater problems, more to fix.

Thank you all for your responses!