How to dynamically embed Julia in a Windows app

General Usage
,
1

I have written a plugin for the commercial statistical software Stata that bridges to Julia in order to exploit its speed. In an issue thread on juliaup, @davidanthoff and @staticfloat have helped me figure out how best to do this. But I’m still getting crashes on some Windows systems and am stuck.

Here is a MWE. It loads libjulia.dll, gets the address for the jl_init() function, and then tries to call it. I compile this in Visual Studio while adding no extra libraries to link to but adding the Julia include folder so it finds juliah.h. It crashes on the last step.

#include <julia.h>
#include <iostream>
#include "windows.h"

typedef void (* myfn_t)(void);

int main() {
    HINSTANCE hDLL = LoadLibraryA("C:\\Users\\maihp\\.julia\\juliaup\\julia-1.9.4+0.x64.w64.mingw32\\bin\\libjulia.dll");
    myfn_t JL_init = (void (*)(void))GetProcAddress(hDLL, "jl_init");
    std::cout << "Didn't crash 1!\n";
    (JL_init)();
    std::cout << "Didn't crash 2!\n";
    return 0;
}

When I run the result, test.exe, in CMD, I get:

Microsoft Windows [Version 10.0.19045.2965]
(c) Microsoft Corporation. All rights reserved.

... [snip]

C:\Users\maihp\source\repos\julia.ado\x64\Release>test.exe
Didn't crash 1!
fatal: error thrown and no exception handler available.
InitError(mod=:Sys, error=ErrorException("could not load library "libpcre2-8"
The specified module could not be found. "))
ijl_errorf at C:/workdir/src\rtutils.c:77
ijl_load_dynamic_library at C:/workdir/src\dlload.c:369
jl_get_library_ at C:/workdir/src\runtime_ccall.cpp:48
jl_get_library_ at C:/workdir/src\runtime_ccall.cpp:39 [inlined]
ijl_load_and_lookup at C:/workdir/src\runtime_ccall.cpp:61
jlplt_pcre2_compile_8_33629.clone_1 at C:\Users\maihp\.julia\juliaup\julia-1.9.4+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
compile at .\pcre.jl:161
compile at .\regex.jl:75
match at .\regex.jl:376
match at .\regex.jl:376 [inlined]
match at .\regex.jl:395 [inlined]
splitdrive at .\path.jl:38
joinpath at .\path.jl:264
joinpath at .\path.jl:327 [inlined]
abspath at .\path.jl:449 [inlined]
__init_build at .\sysinfo.jl:128
__init__ at .\sysinfo.jl:120
jfptr___init___42494.clone_1 at C:\Users\maihp\.julia\juliaup\julia-1.9.4+0.x64.w64.mingw32\lib\julia\sys.dll (unknown line)
jl_apply at C:/workdir/src\julia.h:1880 [inlined]
jl_module_run_initializer at C:/workdir/src\toplevel.c:75
_finish_julia_init at C:/workdir/src\init.c:855
ijl_init_with_image at C:/workdir/src\jlapi.c:66 [inlined]
ijl_init_with_image at C:/workdir/src\jlapi.c:55 [inlined]
ijl_init at C:/workdir/src\jlapi.c:82
unknown function (ip: 00007ff70da2103a)
unknown function (ip: 00007ff70da214bf)
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)

To be clear, I have compiled on the same computer on which I’m getting the crash.

This is happening in Windows 10 Pro 22H2. At least one user reports the same thing in “Windows 11 Pro 64-bit”. It does not happen to me on another laptop with Windows 11 Pro 22H2. All systems have Julia 1.9.4.

Thanks for any help!

P.S. The motive for run-time rather than load-time linking to the Julia libraries is that in a juliaup world, there can be multiple versions on a machine, none of whose shared libraries are visible to the linker at load time.

5 Likes
2

The first question to answer is whether Windows actually cannot find “libpcre2-8.dll” or if there anorher problem, perhaps with dependencies.

You will then want to use gflags and a Windows debugger as described here:

1 Like
3

Essentially we will want the Debugging Tools for Windows:

Then we will need gflags, the global flags editor.

Run gflags by itself to get a dialog box with options.

Next you want to enable loader snaps to trace the Windows DLL loader.

So we do gflags /i stata.exe +sls or whatever the entry point is.

Next you have to run the program under a debugger.

1 Like
4

Thanks @mkitti. Yes, libpcre2-8.dll is in my Julia bin folder, next to julia.exe and all the other dll’s. I should have mentioned that.

I’ve already run it in the Visual Studio debugger. The error in Visual Studio is

Exception thrown at 0x0000000000000000 in Project1.exe: 0xC0000005: Access violation executing location 0x0000000000000000.

My guess is that the error is much easier to understand than fix. I think it is occurring because Windows doesn’t know to look for DLL’s in the Julia bin directory. Because when using juliaup, no Julia bin directory (and there will be more than 1 if you have multiple installation channels) is in PATH. Only the directory holding the juliaup executable, or a shortcut to it, is in PATH. My little program is able to load libjulia.dll because I provide it the exact path. But if that DLL contains code that wants to call a function in another Julia DLL, like libjulia-internal.dll or libpcre2-8.dll, then it’s stuck.

I tried inserting

HINSTANCE hDLLlibpcre28 = LoadLibraryA("C:\\Users\\maihp\\.julia\\juliaup\\julia-1.9.4+0.x64.w64.mingw32\\bin\\libpcre2-8.dll");

to load the missing library that way, but it didn’t change the error.

So for me this leads to the question in the title of the post: how does one embed Julia in a Windows app for distribution? Is there some way to tell Windows, when loading a DLL at run time, where to look for other DLLs that the explicitly loaded one is referencing?

Or to turn that around, do I need to tell my Windows users not to use juliaup? According to my limited understanding, in Linux and macOS, there is just one shared library file, libjulia.so, so this issue does not arise. (I think…)

If I could somehow guarantee that all the Julia dll’s are copied into the same folder as Stata.exe–the application for which I’ve written the plugin–that might work. But that is not really practical.

1 Like
5

Could the problem also be related to different C runtime libraries? I believe Julia uses msvcrt.dll (which is the very old version of it that ships as part of Windows for backwards compat reasons that MS would really like everyone to not use), but Visual Studio presumably would use more modern versions of the runtime C library. @staticfloat would probably know more :slight_smile:

I don’t think this is related to PATH, Julia itself also works if the dlls are not on the PATH.

2 Likes
6

I’m unclear if you used gflags or not and what the error has to fo with the stacktrace. Did you try to continue execution past the “access violation”?

Here is how the DLL search order works on Windows:

7

Right. I believe that when Julia is installed with juliaup, the Julia dll’s appear nowhere in the list of places that Windows checks.

8

Set the gflags like I suggested above and then the debugger will tell you exactly where Windows is looking for libraries or why it cannot load anything.

9

This error is also raised when one tries to bundle Julia in a MSIX bundle for which I oppened #52007 issue. From what I skimmed through the solution seems to be recompiling julia and all it’s libraries with /ZW flag.

10

I think I solved it for my case!

This substitutes for the LoadLibraryA() call in my example:

    SetCurrentDirectoryA(libdir);
    hDLL = LoadLibraryExA(fulllibpath, NULL, LOAD_WITH_ALTERED_SEARCH_PATH);

where libdir is the path to the /bin folder and fulllibpath is that + “libjulia.dll”.

I think it is also possible to do it with AddDllDirectory() and then LoadLibraryExA(fulllibpath, NULL, LOAD_LIBRARY_SEARCH_USER_DIRS) but AddDllDirectory() requires a wide/Unicode string and I’d rather avoid the complication.

11

You may be interested in Use modern, secure, Windows API for dlopen

1 Like
12

Not exactly answering your question but given what you’re doing you may find GitHub - GunnarFarneback/DynamicallyLoadedEmbedding.jl: Embed Julia with dynamical loading of libjulia at runtime. interesting, if you’re not already familiar with it.

(I should add some documentation about juliaup there, since I’ve recently updated the application which the package is based on.)

1 Like
13

@GunnarFarneback, I did not know about this! It looks well done and is reassuring to me.

It appears that there is one novelty in what I have done, which might interest you. It flows entirely from the guidance of @staticfloat.

It’s a response to the problem that an application distributed without Julia has to find the shared Julia libraries somehow. The task is complicated by the multiplexing of juliaup, which tends to further obscure the location of the libraries, because there could be several locations, for different Julia versions simultaneously installed.

The solution is to run a shell command to look first for the executable, and ask it which libjulia it is using:

julia -e "using Libdl; println(dlpath(\"libjulia\"))" > tempfile

In my context, I have little choice to us use a temporary file to capture the library path. There are probably more graceful solutions in other contexts.

14

So what my application does after the juliaup integration is the following (it’s a requirement that the users have juliaup installed):

  1. Look up julia version from an application configuration file, for this example let’s assume it’s 1.9.3.
  2. Run external process juliaup add 1.9.3.
  3. Run external process julia +1.9.3 -E Sys.BINDIR and read the output into bindir.
  4. Windows: SetDllDirectoryW(bindir), julia_library_name = "libjulia.dll"
  5. Linux: bindir.replace("/bin", "/lib"), julia_library_name = bindir + "/libjulia.so"
  6. Continue with what’s already in DynamicallyLoadedEmbedding.

It would be nicer if I could just ask juliaup for the location of libjulia for a given version. It should have the needed information and be faster to query than spinning up a Julia process only for that reason.

15

Yes, agreed, we discussed a possible API for that in Make Windows & Mac installation put Julia runtime libraries in path · Issue #758 · JuliaLang/juliaup · GitHub.

The idea I had there was that you could essentially ask Juliaup “give me the libjulia that I should be using if X is my working folder”, and then Juliaup would look at all the overrides/defaults etc and figure out which Julia version to use. I think your use case might be a bit different, in that you want to control which Julia version to use somewhere in your apps config, right? So we should actually design the API to support that case as well.

2 Likes
16

Yes, that’s right.