Issues with Plots and Libtiff_jll

General Usage
, ,
1

I recently updated the Plots package in my working environment just to find that I couldn’t plot anymore as the installation was suddenly broken.

> using Plots
...
WARNING: Method definition airyai(ColorTypes.Color{T, 1} where T) in module ColorVectorSpace at /home/gzagatti/.julia/packages/ColorVectorSpace/QI5vM/src/ColorVectorSpace.jl:315 overwritten in module SpecialFunctionsExt at /home/gzagatti/.julia/packages/ColorVectorSpace/tLy1N/ext/SpecialFunctionsExt.jl:15.
  ** incremental compilation may be fatally broken for this module **
...
> plot(1)
GKS: libtiff.so.6: cannot open shared object file: No such file or directory

I found out the issue was that when Plots was updated so was Libtiff_jll. Now Libtiff_jll links to version 4.5 of libtiff which is not available on the official apt repositories for Ubuntu 22 as you can see from here.

I considered the following options, but I am not completely satisfied with any of them.

  1. Add a compat entry to my Project.toml environment as described in this discourse thread.

    It is annoying to add a compat entry to every single project that depends on Plots. The compat entry is only specific to my development environment and not to the project itself. Therefore, I would not like to block upstream updates because my machine does not have access to the right dynamic library.

    Package resolution should also take into account the dynamic libraries when determining the dependencies it needs. It is weird that version 4.5 of Libtiff_jll was selected when in fact the system does not satisfy its requirements.

    If Julia cannot determine whether non-Julia dependencies are available on the system, I would like to at least have a local-only compat entry to tell Julia about the limitations of my system. Similar to adding an entry to .git/info/exclude when I want to exclude local files from my git repo but don’t want to commit them to .gitignore.

  2. Install libtiff with Homebrew and modify ENV in startup.jl:

    # startup.jl
if isfile("/home/linuxbrew/.linuxbrew/lib/libtiff.so.6")
  libtiff = dirname(realpath("/home/linuxbrew/.linuxbrew/lib/libtiff.so.6"))
  ld_library_path = get(ENV, "LD_LIBRARY_PATH", "")
  ld_library_path = length(ld_library_path) > 0 ? ":$ld_library_path" : ""
  ENV["LD_LIBRARY_PATH"] = "$libtiff$ld_library_path"
end

    This didn’t work. Despite the modification to ENV["LD_LIBRARY_PATH"] at the very beginning of startup.jl, Julia would still complain.

    GKS: libtiff.so.6: cannot open shared object file: No such file or directory
    1. Install libtiff with Homebrew and start julia with a modified LD_LIBRARY_PATH.
    LD_LIBRARY_PATH=/home/linuxbrew/.linuxbrew/Cellar/libtiff/4.5.1/lib:${LIBRARY_PATH} julia

    This worked. However, it does not feel elegant to start julia with a hacked LD_LIBRARY_PATH, especially when ldconfig -v points to libtiff.so.6 installed via Homebrew. ldconfig is installed as part of glibc with Homebrew and exposed in my path via /home/linuxbrew/.linuxbrew/sbin.

    If I have to modify LD_LIBRARY_PATH, I would prefer to use LD_LIBRARY_PATH=/home/linux/.linuxbrew/lib:${LIBRARY_PATH} since this is where Homebrew exposes its dynamic libraries to the system. However, I get a different error:

    InitError: could not load library "/home/gzagatti/.julia/artifacts/57a2adf6148eab124016172e61e7850ec8bcc217/lib/libpangocairo-1.0.so"
/home/linuxbrew/.linuxbrew/lib/libpangoft2-1.0.so.0: undefined symbol: hb_font_set_synthetic_slant
during initialization of module Pango_jll

    Now, Pango_jll is complaining.

At the end I went for option (3) because that is what worked and I didn’t need to modify every single Project.toml that I came accross. However, I would prefer to go for option (2) because at least everything remains within Julia. It would be great if Julia could pick the modifications in ENV at startup.

Even better, it would be great to have a file in ~/.julia/config that tells Julia about our local development constraints.

It would be great to hear your thoughts on this issue. Perhaps someone has a better solution or I have missed some obvious configuration.

Thanks for all the great work on BinaryBuilder.jl and Yggdrasil which I had no idea it existed until I came across this error.

2

There clearly is a bug here but it’s not that some specific version of libtiff is missing from your system libraries. It would be helpful to be able to reproduce this. I’m also on Ubuntu 22 and this works:

/tmp$ mkdir test_environment
/tmp$ cd test_environment/
/tmp/test_environment$ julia --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.9.3 (2023-08-24)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(test_environment) pkg> add Plots Libtiff_jll
[...]

(test_environment) pkg> status
Status `/tmp/test_environment/Project.toml`
  [91a5bcdd] Plots v1.39.0
  [89763e89] Libtiff_jll v4.5.1+1

julia> using Plots, Libtiff_jll

julia> plot(1)  # works

julia> Libtiff_jll.libtiff_path
"/home/gunnar/.julia/artifacts/e18a60e84fd8aefa967cb9f1d11dc6cbd9cac88b/lib/libtiff.so"

The mechanism to point Julia to system or custom libraries is 8. Artifacts · Pkg.jl but unless you have very specific needs it’s unlikely that this is something you want to explore.

Changing ENV["LD_LIBRARY_PATH"] inside julia happens much too late since that environment variable was read by the dynamic loader before starting julia.

3

GR_jll correctly specifies the compat bound for Libtiff_jll, so I’m missing what the problem could be:

4

Thank you both for your fast response.

@GunnarFarneback I followed your step-by-step test_environment without any issues. It turns out the problem involved more than the Plots and Libtiff_jll.

As I use Kitty as my terminal, I tend to pre-load a convenient package KittyTerminalImages in my startup script to display plots directly on the terminal (which is particularly convenient in a remote server). It turns out that KittyTerminalImages will install Libtiff_jll version 4.4 because it depends on RSvg which depends on gdk_pixbuf_jll which has a constraint on Libtiff_jll.

However, KittyTerminalImages is loaded from my base environment @1.9 while Plots usually come from my working Project.toml. Therefore, to reproduce the bug you have to do the following:

> cd ~/.julia/environments/v1.9
> julia --startup-file=no
julia> ]
(@v1.9) pkg> add https://github.com/gzagatti/KittyTerminalImages.jl.git#inteporlation-update
...
julia> exit()
> cd test_environment
> julia --startup-file=no
julia> using KittyTerminalImages
julia> ]
(@v1.9) pkg> activate .
(test_environment) pkg>
julia> using Plots
...
WARNING: Method definition zeta(ColorTypes.Color{T, 1} where T) in module ColorVectorSpace at /home/gzagatti/.julia/packages/ColorVectorSpace/QI5vM/src/ColorVectorSpace.jl:315 overwritten in module SpecialFunctionsExt at /home/gzagatti/.julia/packages/ColorVectorSpace/tLy1N/ext/SpecialFunctionsExt.jl:15.
  ** incremental compilation may be fatally broken for this module **

GKS: libtiff.so.6: cannot open shared object file: No such file or directory

Strangely enough, if I do the inverse I have no problems:

> cd test_environment
> julia --startup-file=no
julia> ]
(@v1.9) pkg> activate .
(test_environment) pkg>
julia> using Plots, KittyTerminalImages

Therefore, my strategy will be to stop pre-loading KittyTerminalImages and only load the package after Plots is loaded. A bit less convenient, but more elegant IMO.

In any case, I think we might have a bug here because each package should read its own artifact. But please correct me if I’m wrong.

5

I think the problem is that Pkg allows stacking incompatible environments.

1 Like