Relocatable Makie app

ohmsweetohm1 · September 23, 2024, 12:24pm

I want to share some results with a college so that he can have a look at the data himself.
Plotly can export an HTML which is interactive, but WGLMakie can not (as far as I known). It is mainly time series charts where it would be nice to pan and zoom around.

I have this very basic app to test the approach, but it appears to be not relocatable.

module MakieTestApp

using GLMakie

function julia_main()::Cint
    # do something based on ARGS?

    fig1 = Figure()
    ax1 = Axis(fig1[1, 1])
    lines!(ax1, cumsum(randn(100)))
    lines!(ax1, cumsum(randn(100)))
    lines!(ax1, cumsum(randn(100)))

    screen2 = GLMakie.Screen()
    display(screen2, fig1)

    fig2 = Figure()
    ax1 = Axis(fig2[1, 1])
    lines!(cumsum(randn(100)))
    lines!(cumsum(randn(100)))
    lines!(cumsum(randn(100)))

    screen1 = GLMakie.Screen()
    display(screen1, fig2)

    wait(screen1)
    wait(screen2)

    return 0 # if things finished successfully
end

end # module MakieTestApp

import PackageCompiler
PackageCompiler.create_app(
    "MakieTestApp",
    "MakieTestAppCompiled",
)

When I run the app after compiling I see this screen:

I had an issue with this Glib warning here already: GLib-GIO-WARNING · Issue #4049 · MakieOrg/Makie.jl · GitHub

Nevertheless, on another machine, I get this:

SystemError: opening file "C:\\Users\\KaisermayerV\\.julia\\packages\\GLMakie\\QOOnq\\assets\\shader\\postprocessing/fullscreen.vert": No such file or directory

What can I do?

BTW there app directory has 1.5GB!

sdanisch · September 23, 2024, 12:32pm

We do have a test for this:

github.com

MakieOrg/Makie.jl/blob/master/relocatability.jl#L38-L50


      
          create_app(joinpath(pwd(), "MakieApp"), "executable"; force=true, incremental=true, include_transitive_dependencies=false)
          exe = joinpath(pwd(), "executable", "bin", "MakieApp")
          @test success(`$(exe)`)
          julia_pkg_dir = joinpath(Base.DEPOT_PATH[1], "packages")
          @test isdir(julia_pkg_dir)
          mvd_julia_pkg_dir = julia_pkg_dir * ".old"
          # Move package dir so that we can test relocatability (hardcoded paths to package dir being invalid now)
          try
              mv(julia_pkg_dir, mvd_julia_pkg_dir)
              @test success(`$(exe)`)
          catch e
              mv(mvd_julia_pkg_dir, julia_pkg_dir)
          end

Which should test for not existing ./julia/packages/**…
Not sure what’s going wrong in your case… Could you maybe try the code from the test and see if that passes, and then figure out what you do differently from there?

sdanisch · September 23, 2024, 12:35pm

Oh I just realize our test may be incorrect, since the GLMakie source code doesn’t live in .julia/packages/** on the CI.

ohmsweetohm1 · September 23, 2024, 12:35pm

Maybe; the build options are different

ohmsweetohm1 · September 23, 2024, 2:50pm

Nop, same problem!

ohmsweetohm1 · September 25, 2024, 9:43am

Any recommendations?
I considered WGLMakie but the HTML export appears to be not sufficient for my needs. I want a zoomable time series plot.

sdanisch · September 25, 2024, 10:45am

I’m guessing, that this part isn’t relocatable anymore:

github.com

MakieOrg/Makie.jl/blob/master/GLMakie/src/GLMakie.jl#L51-L76


      
          function ShaderSource(path)
              typ = GLAbstraction.shadertype(splitext(path)[2])
              source = read(path, String)
              name = String(path)
              return ShaderSource(typ, source, name)
          end
          
          const GL_ASSET_DIR = RelocatableFolders.@path joinpath(@__DIR__, "..", "assets")
          const SHADER_DIR = RelocatableFolders.@path joinpath(GL_ASSET_DIR, "shader")
          const LOADED_SHADERS = Dict{String, ShaderSource}()
          
          const SHADER_PATHS = Dict{String,Union{RelocatableFolders.Path, String}}()
          
          # Turns out, loading shaders is so slow, that it actually makes sense to memoize it :-O
          # when creating 1000 plots with the PlotSpec API, timing drop from 1.5s to 1s just from this change:
          function shader_path(name)
              return get!(SHADER_PATHS, name) do
                  return joinpath(SHADER_DIR, name)
              end
          end

This file has been truncated. show original

I’m not 100% sure why, so this is where I would spend the time debugging.

ohmsweetohm1 · September 25, 2024, 10:54am

Building new paths from, for example, ASSETS in the above example will return a String containing the resolved path rather than a Path object. Doing this at the module-level will result in hardcoded paths that will run into relocatability issues as discussed above. Always create a new @path for each resource you wish to reference rather than building them in parts, e.g.

So I’m guessing this is the problem:

github.com

MakieOrg/Makie.jl/blob/9eca66abc21fd8300e9532f8eaa90b58c643e10c/GLMakie/src/GLMakie.jl#L68C16-L68C42


      
          return joinpath(SHADER_DIR, name)

ohmsweetohm1 · September 26, 2024, 6:57am

julia> typeof(GLMakie.SHADER_DIR)
RelocatableFolders.Path

julia> typeof(joinpath(GLMakie.SHADER_DIR, "postprocessing/fullscreen.vert"))
String

ohmsweetohm1 · October 18, 2024, 7:06pm

This is fixed in the current master of GLMakie.

Thx @sdanisch

ohmsweetohm1 · October 29, 2024, 12:24pm

BTW, since the relocatability is a job for each and every package, I still do not see that a general Julia app will ever be relocatable out of the box - unless Julia handles this at a high level.

fatteneder · October 29, 2024, 3:44pm

The current plan is to get a helper into Julia 1.12 to deal with the exact issue from this post.