Bundling Julia GUI application for Windows

Janis_Erdmanis · October 23, 2023, 9:48pm

I am wondering has anyone has tried to bundle a Julia application for Universal Windows Platform (UWP) using the MSIX bundle?

I am currently working on a cross-platform bundler for Julia GUI applications called AppBundler.jl. I have already made some progress on the MacOS and Linux versions using the Snap format, but strict confinement is still an issue. Now, I am focusing on the Windows version and aiming to use the MSIX bundle. This would enable the publishing of apps on the Windows store, which is pretty cool. However, I have realized that there are many assumptions I have made that are not true on Windows, such as being able to use Powershell in the same way as Bash to write a launcher script.

My last attempt now is to write a launcher in C# using a template for the UWP GUI application. Distilling to essentials, I am now trying to start Julia, placing the julia folder next to the entry point and running it with a corresponding C# code:

string DIR = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
string JULIA = Path.Combine(DIR, "julia", "bin", "julia.exe");

Process.Start(JULIA);

string USER_DIR = ApplicationData.Current.LocalFolder.Path;
string LOG_PATH = Path.Combine(USER_DIR, "output.txt");

File.WriteAllText(LOG_PATH, JULIA);

The good news is that the binary was found, and the terminal flashed on the screen. However, I got an output, which says could not load library “libpcre2-8” The parameter is incorrect. The full error can be seen in the screenshot. I have no idea how I could proceed from there.

mkitti · October 24, 2023, 12:28am

It is looking for libpcre2-8.dll which on Windows should be in the bin directory where the executable lives.

You may have a problem with the DLL search path because that may be restricted for UWP apps.

Ping me later for details and links

mkitti · October 24, 2023, 6:02pm

This pull request and discussion may be relevant to your effort:

github.com/JuliaLang/julia

Try LOAD_LIBRARY_SEARCH_DLL_LOAD_DIR | LOAD_LIBRARY_SEARCH_SYSTEM32 | LOAD_LIBRARY_SEARCH_USER_DIRS for DLL loading on Windows

JuliaLang:master ← mkitti:mk/load_library_search_user_dirs

opened 04:09AM - 02 Dec 22 UTC

mkitti

+25 -1

Currently, we use `LOAD_WITH_ALTERED_SEARCH_PATH` to load DLL libraries on Windo…ws. With this pull request, if loading with the above fails, we use `LOAD_LIBRARY_SEARCH_USER_DIRS`. This is a minimalist version of the idea proposed on [Discourse](https://discourse.julialang.org/t/use-modern-secure-windows-api-for-dlopen/90171/5). Rather than replacing `LOAD_WITH_ALTERED_SEARCH_PATH` with `LOAD_LIBRARY_SEARCH_DEFAULT_DIRS`, we only use `LOAD_LIBRARY_SEARCH_USER_DIRS` if loading with ``LOAD_WITH_ALTERED_SEARCH_PATH` fails. This allows the user to manually build a search list of directories via `AddDllDirectory`. ```julia @ccall "kernel32".AddDllDirectory(library::Cwstring)::Ptr{Nothing} ``` https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexw

Janis_Erdmanis · October 25, 2023, 2:07pm

This would be something I could try out if it would be available. I tried to set the PATH variable to before starting the process as follows:

string currentPath = Environment.GetEnvironmentVariable("PATH");
string newPath = Path.Combine(DIR, "julia", "bin") + ";" + currentPath;
Environment.SetEnvironmentVariable("PATH", newPath, EnvironmentVariableTarget.Process);
Process.Start(JULIA);

but that had no effect. What else I could try?

Janis_Erdmanis · October 25, 2023, 10:59pm

I finally managed to get something. Using Advanced Installer software I made a bundle of julia in full trust mode which runs. The next step is to see whether it will work with GUI applications as well and investigate what configuration made it possible so I could do bundling in more automated way.

Janis_Erdmanis · October 26, 2023, 9:50am

There couldn’t be such typos as I am not reffering to theese libraries individually.

The libpcre2-8 library is part of Julia bundle and usually it is referenced from julia.exe itself. What I am merelly doing is that I am taking the Julia zip bundle for Windows, unzip it, and place it in MSIX folder structure and pack it with makeappx or in this case within C# project. I even tried to modify the PATH variable to include the julia/bin folder before starting the julia.exe but without any results.

It might be the case that the way Julia gets a relative path of DLLs is faulty within UWP container environment.

Janis_Erdmanis · October 27, 2023, 1:17pm

I expected that I will have a smooth ride after cracking how to run Julia in a full trust UWP envirnment, but another issue have just surfaced. When Julia does precompilation, for instance, when adding a Revise package a following error is thrown:

julia> using Revise
[ Info: Precompiling Revise [295af30f-e4ad-537b-8983-00126c2a3abe]
ERROR: LoadError: failed process: Process(setenv(
`'C:\Program Files\WindowsApps\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j\libexec\julia\lld.exe'
-flavor gnu -m i386pep -Bdynamic --enable-auto-image-base 
--allow-multiple-definition '' -shared -o 'C:\Users\jerdmanis\.julia\compiled\v1.9\OrderedCollections\jl_D1E8.tmp' 
--whole-archive 'C:\Users\jerdmanis\.julia\compiled\v1.9\OrderedCollections\jl_D1E7.tmp' 
--no-whole-archive '-LC:\Program Files\WindowsApps\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j\lib' 
'-LC:\Program Files\WindowsApps\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j\lib\julia' 
'-LC:\Program Files\WindowsApps\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j\bin' 
-ljulia -ljulia-internal -lopenlibm -lssp -lgcc_s -lgcc -lmsvcrt`,
["WINDIR=C:\\Windows", "PATH=C:\\Program Files\\WindowsApps\\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j\\lib\\julia;
C:\\Program Files\\WindowsApps\\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j\\bin;C:\\Program Files\\Parallels\\Parallels Tools\\Applications;C:\\Windows\\system32;C:\\Windows;
C:\\Windows\\System32\\Wbem;
C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files\\dotnet\\;
C:\\Program Files (x86)\\Windows Kits\\10\\Windows Performance Toolkit\\;C:\\Users\\jerdmanis\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\jerdmanis\\.dotnet\\tools;C:\\Users\\jerdmanis\\AppData\\Local\\Programs\\Julia-1.9.3\\bin;
C:\\Users\\jerdmanis\\JuliaPack\\julia\\lib", "USERDOMAIN_ROAMINGPROFILE=JANISERDMAN0F68", "LOCALAPPDATA=C:\\Users\\jerdmanis\\AppData\\Local", "HOMEPATH=\\Users\\jerdmanis", "PROCESSOR_IDENTIFIER=ARMv8 (64-bit) Family 8 Model 0 Revision   0,  ", "NUMBER_OF_PROCESSORS=4", "PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC", "SYSTEMROOT=C:\\Windows", "APPDATA=C:\\Users\\jerdmanis\\AppData\\Roaming" 
…  
"COMMONPROGRAMFILES(ARM)=C:\\Program Files (Arm)\\Common Files", "PROCESSOR_LEVEL=0", "JULIA_LOAD_PATH=C:\\Users\\jerdmanis\\.julia\\environments\\v1.9\\Project.toml;C:\\Program Files\\WindowsApps\\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j\\share\\julia\\stdlib\\v1.9", "SYSTEMDRIVE=C:", "=C:=C:\\Program Files", "PROGRAMW6432=C:\\Program Files", 
"TEMP=C:\\Users\\JERDMA~1\\AppData\\Local\\Temp", "HOMEDRIVE=C:", "OPENBLAS_MAIN_FREE=1", "PROCESSOR_ARCHITECTURE=AMD64"]),
 ProcessExited(3221225781)) [3221225781]

I do have access to Julia REPL, what could I check to troubleshoot the issue?

One thing I noticed is that running lld.exe with Julia shell from UWP container (still under full trust) does not give any output wheras running it from ordinary Julia shell gives output:

julia> run(`lld.exe`)
lld is a generic driver.
Invoke ld.lld (Unix), ld64.lld (macOS), lld-link (Windows), wasm-ld (WebAssembly) instead

I could also reproduce this strange behaviour by running Julia with adminstrateive rights from C:\Program Files\WindowsApps\JanisErdmanis.Julia_1.0.0.0_neutral__s0by06ay2112j where no output for lld.exe is shown. Afterwards I copied all files from the WindowsApp folder to a home directory and the output for lld.exe command returned.