HDF5 cannot compile in M1 pro mac

The julia version is v1.7.0 rc-2, installed by official dmg.
OS: OSX Monterey

Hope to learn about how to resolve this problem, thanks!

The log:

julia> using HDF5
[ Info: Precompiling HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f]
ERROR: LoadError: UndefVarError: libhdf5 not defined
Stacktrace:
[1] top-level scope
@ ~/.julia/packages/HDF5/pIJra/src/api_types.jl:119
[2] include(mod::Module, _path::String)
@ Base ./Base.jl:420
[3] include(x::String)
@ HDF5 ~/.julia/packages/HDF5/pIJra/src/HDF5.jl:1
[4] top-level scope
@ ~/.julia/packages/HDF5/pIJra/src/HDF5.jl:51
[5] include
@ ./Base.jl:420 [inlined]
[6] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
@ Base ./loading.jl:1318
[7] top-level scope
@ none:1
[8] eval
@ ./boot.jl:373 [inlined]
[9] eval(x::Expr)
@ Base.MainInclude ./client.jl:453
[10] top-level scope
@ none:1
in expression starting at /Users/John/.julia/packages/HDF5/pIJra/src/api_types.jl:119
in expression starting at /Users/John/.julia/packages/HDF5/pIJra/src/HDF5.jl:1
ERROR: Failed to precompile HDF5 [f67ccb44-e63f-5c2f-98bd-6dc0ccc4ba2f] to /Users/John/.julia/compiled/v1.7/HDF5/jl_Ku0anm.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
[2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
@ Base ./loading.jl:1466
[3] compilecache(pkg::Base.PkgId, path::String)
@ Base ./loading.jl:1410
[4] _require(pkg::Base.PkgId)
@ Base ./loading.jl:1120
[5] require(uuidkey::Base.PkgId)
@ Base ./loading.jl:1013
[6] require(into::Module, mod::Symbol)
@ Base ./loading.jl:997

1 Like

I have the same issue. I even try directly giving the path to libhdf5 but it still doesn’t work.

Edit - Got it to work. I removed the .julia folder in your home directory completely, upgraded to 1.7.0 rc-3, and in my terminal exported the path to libhdf5 like this export JULIA_HDF5_PATH=/opt/homebrew/Cellar/hdf5/1.12.1 which I installed with homebrew, obviously by the path. Then started Julia and ran these commands using Pkg; Pkd.add("HDF5"); Pkg.build("HDF5").

It also worked for me. Many thanks for the solution!

Thank you so much for this solution. I’m trying to follow these steps. I have julia version 1.7.2, which is what I think is recommended for M1 chip.

I used homebrew to install hdf5 as you recommend, and added to the path.

I then ran using Pkg; Pkg.add("HDF5") without any problem.

However, when I ran Pkg.build("HDF5") I got the following error:

Error building HDF5:

[ Info: using system HDF5

ERROR: LoadError: libhdf5 could not be found

Do you know what might be going wrong?

It looks like it’s not using the home brew hdf5. There is an environment variable within julia (not your OS) called JULIA_LIBHDF5 or something similar that you need to set to point to the brew hdf5 install. I am mobile right now but respond if you can’t get it and I’ll be able to check exactly what my settings are later tonight.

Thank you so much! I really appreciate your saying this. yes, if you could check your settings I would be super grateful. I am trying to follow this website, which talks about environment variables in Julia.

Environment Variables · The Julia Language.

It talks about editing the file ~/.julia/config/startup.jl

But when I go into my .julia folder, my only options are artifacts, conda, logs, pluto_notebooks, registries, compiled, environments, packages, prefs, and scratchspaces

startup.jl is optional. If it isn’t there, create one.

right, but there is also not even a config folder in my settings

@giordano Shouldn’t there be another way of doing this with HDF5_jll.jl and a Preferences.jl override?

I’m also confused because I thought we have

Depends on the version of HDF5_jll.jl they’re using. May not be the last one. You can see it with

]status -m HDF5_jll

@mkitti you still need to point Julia to the brew install of hdf5. @Xavier_Gonzalez Just put in your .zshrc or .bashrc or whatever shell you use export JULIA_HDF5_PATH=/opt/homebrew/Cellar/hdf5/1.12.1_1 make sure to match the version number at the end. you can check your version number by just doing ls -l /opt/homebrew/Cellar/hdf5/. also start a new shell and then a new Julia instance when you go to test. then do add HDF5, build HDF5 etc

@kylebeggs I’m uncertain why homebrew is still needed. It could certainly still be useful, but I’m not sure why it is necessary as of January 10th, 2022.

As I posted above, HDF5_jll now appears to provide support aarch64-apple-darwin and giordano is not objecting to that assertion.

image

Oh I see, yes it’s possible it’s not needed anymore. I was just saying what worked for me (which was before Jan 2022)