My simple embedding example works on Linux, fails in Windows

embedding

#1

I’m working on a project right now in which I need to call Julia from a C++ Windows application. I’m not a Windows expert but need it for this project. I’m using julia 1.0.2 and on the C++ side I’m building with Microsoft Visual studio 2017. I was running into some memory access violation exceptions in the following scenario:

  1. Call julia function that returns mutable struct using jl_call (or by directly calling an @ccallable function)
  2. Use that mutable struct as input in subsequent jl_call (or call to @ccallable function)

In the course of narrowing down the problem I went back to basics and found that the following MWE runs successfully on linux building with g++ but fails on Windows:

#include <iostream>
#include <julia.h>
JULIA_DEFINE_FAST_TLS() // only define this once, in an executable (not in a shared library) if you want fast code.

int main(int argc, char *argv[])
{
    /* required: setup the Julia context */
    jl_init();

    /* run Julia commands */
    int64_t n = 10;
    jl_value_t* jl_n = jl_box_int64(n);
    jl_eval_string("using SparseArrays");
    jl_module_t *SparseArrays = (jl_module_t*)jl_eval_string("SparseArrays");
    jl_function_t *sprandn = jl_get_function(SparseArrays, "sprandn");
    jl_atexit_hook(0);
    return 0;
} 

When building this with visual studio 2017 on Windows it crashes with the following message and backtrace

Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
Exception: EXCEPTION_ACCESS_VIOLATION at 0x43a3ac9da -- ptrhash_get at /home/Administrator/buildbot/worker/package_win64/build/src/support\ptrhash.c:26
in expression starting at no file:0
ptrhash_get at /home/Administrator/buildbot/worker/package_win64/build/src/support\ptrhash.c:26
jl_get_binding_ at /home/Administrator/buildbot/worker/package_win64/build/src\module.c:230 [inlined]
jl_get_binding at /home/Administrator/buildbot/worker/package_win64/build/src\module.c:261 [inlined]
jl_get_global at /home/Administrator/buildbot/worker/package_win64/build/src\module.c:470
jl_get_function at c:\users\patrick\appdata\local\julia-1.0.2\include\julia\julia.h:1352
main at c:\users\patrick\source\repos\testembedding\testembeddingapp\testembeddingapp.cpp:19
invoke_main at d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78
__scrt_common_main_seh at d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
__scrt_common_main at d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:330
mainCRTStartup at d:\agent\_work\3\s\src\vctools\crt\vcstartup\src\startup\exe_main.cpp:16
BaseThreadInitThunk at C:\WINDOWS\System32\KERNEL32.DLL (unknown line)
RtlUserThreadStart at C:\WINDOWS\SYSTEM32\ntdll.dll (unknown line)
Allocations: 4018 (Pool: 4009; Big: 9); GC: 0

Stepping through the code with the visual studio debugger I saw the following line runs but returns a null pointer

jl_module_t *SparseArrays = (jl_module_t*)jl_eval_string("SparseArrays");

So the following line

jl_function_t *sprandn = jl_get_function(SparseArrays, "sprandn");

crashes when it tries to use the null pointer.

I’ve gotten some other embedding examples to work with the Microsoft toolchain including ones where I pass arrays originally defined in C++ to julia, modify them in place in julia and then subsequently work with them in C++ but I can’t get anything that returns composite object to C++ and then tries to reuse it in subsequent julia calls to work on Windows. No problems on Linux with g++

So, my questions are

  1. Should I expect this to work with Microsoft visual C++? I haven’t tried minGW. I’ve never used it before but perhaps it’s time to learn. I have no idea if this is a Windows issue or a MVSC issue.
  2. Is there anything extra I can do to make this work on Windows? Am I missing something or is this a julia bug? Scouring the docs/discourse/stack overflow hasn’t yielded anything.

For completeness, here’s a little longer MWE that works on Linux but crashes when built with the microsoft toolchain on Windows

#include <iostream>
#include <julia.h>
JULIA_DEFINE_FAST_TLS() // only define this once, in an executable (not in a shared library) if you want fast code.

int main(int argc, char *argv[])
{
    /* required: setup the Julia context */
    jl_init();

    /* run Julia commands */
    int64_t n = 10;
    jl_value_t* jl_n = jl_box_int64(n);
    jl_eval_string("using SparseArrays");
    jl_module_t *SparseArrays = (jl_module_t*)jl_eval_string("SparseArrays");
    jl_function_t *sprandn = jl_get_function(SparseArrays, "sprandn");
    jl_function_t *mul = jl_get_function(jl_main_module, "*");
    jl_value_t *f64_array = jl_apply_array_type((jl_value_t*)jl_float64_type, 1);
    jl_value_t **args;
    JL_GC_PUSHARGS(args, 3);
    double* x;
    x = new double[10];
    for (int i = 0; i<10; i++) {
        x[i] = i;    
    }
    for (int i = 0; i<10; i++) {
        std::cout << "x[i] is " << x[i] << std::endl;    
    }
    jl_array_t *jl_x = jl_ptr_to_array_1d(f64_array, x, 10, 0);
    args[0] = (jl_value_t*)jl_x;
    
    jl_value_t *A = jl_call3(sprandn, jl_n, jl_n, jl_box_float64(0.2));
    args[1] = A;
    std::cout << "A pointer address is " << A << std::endl;
    std::cout << "A.n is " << jl_unbox_int64(jl_get_field(A,"n")) << std::endl;
    
    jl_array_t *jl_b = (jl_array_t*)jl_call2(mul, A, (jl_value_t*)jl_x);
    args[2] = (jl_value_t*)jl_b;
    std::cout << "jl_b pointer address is " << jl_b << std::endl;
    double *b = (double*)jl_array_data(jl_b);
    for (int i = 0; i<10; i++) {
        std::cout << "b " << i << " is " << b[i] << std::endl;
    }
    
    JL_GC_POP();
    delete[] x;
    /* strongly recommended: notify Julia that the
         program is about to terminate. this allows
         Julia time to cleanup pending write requests
         and run all finalizers
    */
    jl_atexit_hook(0);
    return 0;
} 

Thanks, Patrick


#2

Seems like a julia bug to me. I think you should expect this to work with MSVC, the juila API is plain C so there should be no ABI issues.

Does something even more basic such as

jl_value_t *val = jl_eval_string("1");

work and give you the expected value?


#3

Thanks for the reply @Chris_Foster. This fell off my radar in the run up to the holidays. I’ve since gotten some more complex examples to compile with visual studio and run on Windows but not this particular one. So it’s not that I can’t do anything with MSVC. Not sure what’s going on with this particular example. I’ll dig into it a bit more and file an issue.