Problems with environment - Tree Hash Mismatch!

Sure, it gives:

$ df "${HOME}"
Filesystem     1K-blocks     Used Available Use% Mounted on
gpfs_home_labs 104857600 61083648  43773952  59% /home/labs

But isn’t there something special someone needs to do on the cluster? I know that everytime I log in to the cluster I need to load a module of julia to be able to run it. I am not sure how to configure these things…

Ugh, sorry, I forgot you need the -T flag to see the type:

df -T "${HOME}"

Not really, you can follow the instructions about how to use Julia on Linux

1 Like

the type is gpfs.

It is tempting to try to install it myself, but I am afraid to do something wrong… I might just wait for someone to do it…

That’s very likely the issue. Julia v1.6 should be able to deal with it (well, it should still show some warning messages, but it won’t be a fatal error, I believe).

It’d be a good occasion for you to learn how to fiddle a bit with programs in Linux systems :wink: You may ask for help to install Julia in a different thread if you have problems. But I also understand if you don’t really feel comfortable doing it yourself.

2 Likes

Thanks! I already addressed the cluster people asking them teach me how to install julia myself:)
I hope that will solve the issue!
I will update here as soon as I figure it out.

I have julia 1.6.1 installed on the cluster!

I instantiated my project on the cluster and got errors (only attached those that failed):

Precompiling project...
  βœ— Xorg_libxcb_jll
  βœ— Xorg_xcb_util_jll
  βœ— Xorg_libX11_jll
  βœ— Xorg_libXfixes_jll
  βœ— Xorg_libxkbfile_jll
  βœ— Xorg_libXrender_jll
  βœ— Xorg_libXext_jll
  βœ— Xorg_xcb_util_wm_jll
  βœ— Xorg_xcb_util_keysyms_jll
  βœ— Xorg_xcb_util_image_jll
  βœ— Xorg_xcb_util_renderutil_jll
  βœ— Xorg_libXcursor_jll
  βœ— Xorg_libXi_jll
  βœ— Libglvnd_jll
  βœ— Xorg_libXinerama_jll
  βœ— Xorg_libXrandr_jll
  βœ— Cairo_jll
  βœ— Xorg_xkbcomp_jll
  βœ— Xorg_xkeyboard_config_jll
  βœ— GLFW_jll
  βœ— xkbcommon_jll
  βœ— Qt5Base_jll
  βœ— GR_jll
  119 dependencies successfully precompiled in 618 seconds (7 already precompiled)
  23 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages

do you know what is the problem?

I’d start from doing what the message suggests :slightly_smiling_face:

1 Like

I tried, but I don’t get anything…

julia> import Pkg; Pkg.precompile()
Precompiling project...
  βœ— Xorg_libxcb_jll
  βœ— Xorg_xcb_util_jll
  βœ— Xorg_libX11_jll
  βœ— Xorg_libXfixes_jll
  βœ— Xorg_libxkbfile_jll
  βœ— Xorg_libXrender_jll
  βœ— Xorg_libXext_jll
  βœ— Xorg_xcb_util_image_jll
  βœ— Xorg_xcb_util_renderutil_jll
  βœ— Xorg_xcb_util_wm_jll
  βœ— Xorg_xcb_util_keysyms_jll
  βœ— Xorg_xkbcomp_jll
  βœ— Libglvnd_jll
  βœ— Xorg_libXinerama_jll
  βœ— Xorg_libXi_jll
  βœ— Cairo_jll
  βœ— Xorg_libXcursor_jll
  βœ— Xorg_libXrandr_jll
  βœ— Xorg_xkeyboard_config_jll
  βœ— GLFW_jll
  βœ— xkbcommon_jll
  βœ— Qt5Base_jll
  βœ— GR_jll
  0 dependencies successfully precompiled in 197 seconds (126 already precompiled)

Try to load one of those packages, as suggested by the error message I quoted before already :slightly_smiling_face: note that in order to load a package it must be in the environment, so you may have to ]add it

1 Like

It also does not give much information…

(Ising2d) pkg> add Cairo_jll
    Updating registry at `~/.julia/registries/General`
β”Œ Warning: Some registries failed to update:
β”‚     β€” `~/.julia/registries/General` β€” registry dirty
β”” @ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:1229
   Resolving package versions...
   Installed XML2_jll ─────────── v2.9.11+0
   Installed MutableArithmetics ─ v0.2.18
   Installed FFTW ─────────────── v1.4.0
   Installed Polynomials ──────── v2.0.9
  Downloaded artifact: Glib
  Downloaded artifact: Gettext
  Downloaded artifact: Expat
  Downloaded artifact: GLFW
  Downloaded artifact: Libiconv
  Downloaded artifact: XML2
  Downloaded artifact: Libffi
  Downloaded artifact: Libmount
  Downloaded artifact: Zstd
  Downloaded artifact: Libuuid
  Downloaded artifact: PCRE
    Updating `~/Mpemba/Ising2d/Project.toml`
  [6e4b80f9] ↓ BenchmarkTools v1.0.0 β‡’ v0.7.0
  [a93c6f00] ↓ DataFrames v1.1.1 β‡’ v1.0.1
  [7a1cc6ca] ↑ FFTW v1.3.2 β‡’ v1.4.0
  [033835bb] ↓ JLD2 v0.4.5 β‡’ v0.4.4
  [91a5bcdd] ↓ Plots v1.14.0 β‡’ v1.13.1
  [295af30f] ↓ Revise v3.1.16 β‡’ v3.1.15
  [2913bbd2] ↓ StatsBase v0.33.8 β‡’ v0.33.6
  [83423d85] + Cairo_jll v1.16.0+6
    Updating `~/Mpemba/Ising2d/Manifest.toml`
  [6e4b80f9] ↓ BenchmarkTools v1.0.0 β‡’ v0.7.0
  [35d6a980] ↓ ColorSchemes v3.12.1 β‡’ v3.12.0
  [3da002f7] ↓ ColorTypes v0.11.0 β‡’ v0.10.12
  [34da2185] ↓ Compat v3.30.0 β‡’ v3.27.0
  [a93c6f00] ↓ DataFrames v1.1.1 β‡’ v1.0.1
  [7a1cc6ca] ↑ FFTW v1.3.2 β‡’ v1.4.0
  [5789e2e9] ↓ FileIO v1.9.0 β‡’ v1.8.0
  [42e2da0e] ↓ Grisu v1.0.2 β‡’ v1.0.0
  [cd3eb016] ↓ HTTP v0.9.8 β‡’ v0.9.6
  [033835bb] ↓ JLD2 v0.4.5 β‡’ v0.4.4
  [aa1ae85d] ↓ JuliaInterpreter v0.8.16 β‡’ v0.8.14
  [6f1432cf] ↓ LoweredCodeUtils v2.1.0 β‡’ v2.0.0
  [d8a4904e] ↓ MutableArithmetics v0.2.19 β‡’ v0.2.18
  [bac558e1] ↓ OrderedCollections v1.4.1 β‡’ v1.4.0
  [91a5bcdd] ↓ Plots v1.14.0 β‡’ v1.13.1
  [f27b6e38] ↓ Polynomials v2.0.10 β‡’ v2.0.9
  [21216c6a] ↓ Preferences v1.2.2 β‡’ v1.2.1
  [08abe8d2] ↓ PrettyTables v1.0.1 β‡’ v1.0.0
  [295af30f] ↓ Revise v3.1.16 β‡’ v3.1.15
  [992d4aef] ↓ Showoff v1.0.3 β‡’ v1.0.2
  [a2af1166] ↓ SortingAlgorithms v1.0.0 β‡’ v0.3.1
  [90137ffa] ↓ StaticArrays v1.2.0 β‡’ v1.1.2
  [82ae8749] - StatsAPI v1.0.0
  [2913bbd2] ↓ StatsBase v0.33.8 β‡’ v0.33.6
  [f269a46b] ↓ TimeZones v1.5.5 β‡’ v1.5.3
  [5c2747f8] ↓ URIs v1.3.0 β‡’ v1.2.0
  [2e619515] ↓ Expat_jll v2.2.10+0 β‡’ v2.2.7+6
  [0656b61e] ↓ GLFW_jll v3.3.4+0 β‡’ v3.3.3+0
  [78b55507] ↓ Gettext_jll v0.21.0+0 β‡’ v0.20.1+7
  [7746bdde] ↓ Glib_jll v2.68.1+0 β‡’ v2.59.0+4
  [e9f186c6] ↓ Libffi_jll v3.2.2+0 β‡’ v3.2.1+4
  [94ce4f54] ↓ Libiconv_jll v1.16.1+0 β‡’ v1.16.0+7
  [4b2f31a3] ↓ Libmount_jll v2.35.0+0 β‡’ v2.34.0+3
  [38a345b3] ↓ Libuuid_jll v2.36.0+0 β‡’ v2.34.0+7
  [2f80f16e] ↓ PCRE_jll v8.44.0+0 β‡’ v8.42.0+4
  [02c8fc9c] ↓ XML2_jll v2.9.12+0 β‡’ v2.9.11+0
  [3161d3a3] ↓ Zstd_jll v1.5.0+0 β‡’ v1.4.8+0
Precompiling project...
  βœ— Xorg_libxcb_jll
  βœ— Xorg_xcb_util_jll
  βœ— Xorg_libX11_jll
  βœ— Xorg_xcb_util_wm_jll
  βœ— Xorg_xcb_util_keysyms_jll
  βœ— Xorg_xcb_util_image_jll
  βœ— Xorg_xcb_util_renderutil_jll
  βœ— Xorg_libXfixes_jll
  βœ— Xorg_libxkbfile_jll
  βœ— Xorg_libXrender_jll
  βœ— Xorg_libXext_jll
  βœ— Libglvnd_jll
  βœ— Xorg_libXinerama_jll
  βœ— Cairo_jll
  βœ— Xorg_libXrandr_jll
  βœ— Xorg_libXcursor_jll
  βœ— Xorg_libXi_jll
  βœ— Xorg_xkbcomp_jll
  βœ— Xorg_xkeyboard_config_jll
  βœ— GLFW_jll
  βœ— xkbcommon_jll
  βœ— Qt5Base_jll
  βœ— GR_jll
  78 dependencies successfully precompiled in 561 seconds (47 already precompiled)
  23 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages

Because you aren’t loading the package as I asked, you only installed it. You now need to do

using Cairo_jll

to load the package

1 Like

I actually now did precompile again, and I get

(Ising2d) pkg> precompile
Precompiling project...
  βœ— Xorg_libxcb_jll
  βœ— Xorg_xcb_util_jll
  βœ— Xorg_libX11_jll
  βœ— Xorg_libXfixes_jll
  βœ— Xorg_libxkbfile_jll
  βœ— Xorg_libXrender_jll
  βœ— Xorg_libXext_jll
  βœ— Xorg_xcb_util_image_jll
  βœ— Xorg_xcb_util_renderutil_jll
  βœ— Xorg_xcb_util_wm_jll
  βœ— Xorg_xcb_util_keysyms_jll
  βœ— Xorg_libXcursor_jll
  βœ— Xorg_xkbcomp_jll
  βœ— Libglvnd_jll
  βœ— Xorg_libXinerama_jll
  βœ— Xorg_libXrandr_jll
  βœ— Xorg_libXi_jll
  βœ— Cairo_jll
  βœ— Xorg_xkeyboard_config_jll
  βœ— GLFW_jll
  βœ— xkbcommon_jll
  βœ— Qt5Base_jll
  βœ— GR_jll
  0 dependencies successfully precompiled in 227 seconds (125 already precompiled)

ERROR: The following 1 direct dependency failed to precompile:

Cairo_jll [83423d85-b0ee-5818-9007-b63ccbeb887a]

ERROR: LoadError: LoadError: InitError: could not load library "/home/labs/orenraz/roiho/.julia/artifacts/323b2adff3bbf2980e9f1f62990860b61470af16/lib/libgobject-2.0.so"
/home/labs/orenraz/roiho/.julia/artifacts/323b2adff3bbf2980e9f1f62990860b61470af16/lib/libgobject-2.0.so: undefined symbol: g_date_copy
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:114
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:114
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/bkwIo/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ Glib_jll ~/.julia/packages/Glib_jll/nZezQ/src/wrappers/x86_64-linux-gnu.jl:36
  [5] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:674
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:760
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:998
  [8] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:914
  [9] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:901
 [10] include(mod::Module, _path::String)
    @ Base ./Base.jl:386
 [11] top-level scope
    @ ~/.julia/packages/JLLWrappers/bkwIo/src/toplevel_generators.jl:170
 [12] include
    @ ./Base.jl:386 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1213
 [14] top-level scope
    @ none:1
 [15] eval
    @ ./boot.jl:360 [inlined]
 [16] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [17] top-level scope
    @ none:1
during initialization of module Glib_jll
in expression starting at /home/labs/orenraz/roiho/.julia/packages/Cairo_jll/OlXYv/src/wrappers/x86_64-linux-gnu.jl:4
in expression starting at /home/labs/orenraz/roiho/.julia/packages/Cairo_jll/OlXYv/src/Cairo_jll.jl:2

Before going on with other guesses, can you please show the value of the LD_LIBRARY_PATH environment variable inside Julia, for example with

ENV["LD_LIBRARY_PATH"]

?

1 Like

sure:

julia> ENV["LD_LIBRARY_PATH"]
"/apps/RH7U2/gnu/julia/1.6.1/lib:/apps/RH7U2/gnu/python/3.6.3/lib:/usr/share/lsf/10.1/linux3.10-glibc2.17-x86_64/lib:/apps/RH7U2/general/IMOD/4.10.8/IMOD/lib:"

What happens if you start Julia with

LD_LIBRARY_PATH="" julia

so that this environment variable is cleared before starting Julia?

4 Likes

seems to be working fine! no errors!

julia> using Cairo_jll
[ Info: Precompiling Cairo_jll [83423d85-b0ee-5818-9007-b63ccbeb887a]

does this mean I should always start julia like that?
is there a way to configure this environment variable?

I’d say yes, in your cluster.

I don’t recommend changing the environment variable for the rest of your system, I’d create an alias for Julia, or a small shell wrapper

I guess you are right.
actually, most of my use is to run julia scripts. So adding LD_LIBRARY_PATH="" as a prefix before running the script is quite easy.

so apparently running

LD_LIBRARY_PATH="" julia script.jl

does not make the errors go away.
Namely, I still see this error in the output (since I am importing PyPlot):

β”Œ Warning: No working GUI backend found for matplotlib
β”” @ PyPlot ~/.julia/packages/PyPlot/XHEG0/src/init.jl:165

@giordano, do you know what should be done in this case?
Is there a way to set it from β€œinside” the script?

To be clear, you are experiencing multiple different unrelated issues:

  1. uncommon filesystem on which computation of the git tree sha1 fails
  2. LD_LIBRARY_PATH environment variable being set which causes JLL packages to not be loaded correctly
  3. matplotlib GUI not working for some other reasons which at the moment I have no clue about (might be loosely related to point 2 though)

My suggestion would be to use some other plotting libraries. Honestly I’m not familiar with internals of PyPlot.jl to know what’s going on. If you really want to use PyPlot.jl I suggest you to open a new thread, this has been resolved long ago (the original problem with the git tree sha1 was solved by upgrading to Julia v1.6)

2 Likes