Calling Julia from C: printing array of strings?

frylock · June 22, 2023, 9:20pm

I’m aware that there are other, perhaps better, ways of doing this - but I’m trying to broaden my understanding of how the various pieces work.

I have a C program that contains an array of strings, and I’d like to print those strings as if I had done:

julia> println(["one", "two", "three"])
["one", "two", "three"]

This is my program so far:

#include <julia.h>
#include <stdlib.h>

JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[]) {
    const char *stuff[] = {"one", "two", "three"};

    jl_init();

    jl_value_t *array_type = jl_apply_array_type((jl_value_t *)jl_char_type, 1);
    jl_array_t *julia_stuff = jl_ptr_to_array_1d(array_type, stuff, 3, 0);
    jl_function_t *prnt = jl_get_function(jl_base_module, "println");
    jl_call1(prnt, (jl_value_t *)julia_stuff);

    jl_atexit_hook(0);
    return EXIT_SUCCESS;
}

It prints:

['\x25\x09\xc0\x04', '\x00\x00\x56\x29', '\x25\x09\xc0\x08']

mkitti · June 22, 2023, 10:42pm

Well there are many issues here.

I see that you are trying to use jl_char_type. This is not the same as char in C. Rather those are 32-bit integer values.

If you want char as in C in Julia, you need to use Cchar.

julia> sizeof(Char)
4

julia> sizeof(Cchar)
1

This should immediately hint to you that strings in Julia are handled differently than they are in C. For C-compatible strings we have Cstring. Here is an excerpt from the help for Cstring.

Cstring

A C-style string composed of the native character type Cchars. Cstrings are NUL-terminated. For C-style strings composed of the native wide character type, see Cwstring. For more information about string interoperability with C, see the manual.

What you did is basically equivalent to the following Julia code.

julia> stuff = pointer(pointer.(["one", "two", "three"]))
Ptr{Ptr{UInt8}} @0x00000256121036f0

julia> unsafe_wrap(Array{Char}, Ptr{Char}(stuff), 3; own = false) |> println
['\x10\xf2\xf4\xe0', '\x00\x00\x02\x56', '\x10\xf2\xf4\xf8']

julia> unsafe_wrap(Array{Char}, Ptr{Char}(stuff), 3; own = false) |> typeof
Vector{Char} (alias for Array{Char, 1})

What you want to do is something like this.

julia> stuff = pointer(pointer.(["one", "two", "three"]))
Ptr{Ptr{UInt8}} @0x000002579d66f4c0

julia> ptrs = unsafe_wrap(Vector{Ptr{UInt8}}, stuff, 3; own = false)
3-element Vector{Ptr{UInt8}}:
 Ptr{UInt8} @0x0000025611834a08
 Ptr{UInt8} @0x0000025611834a20
 Ptr{UInt8} @0x0000025611834a38

julia> println(unsafe_string.(ptrs))
["one", "two", "three"]

stevengj · June 22, 2023, 10:48pm

This corresponds to a Julia Char type, which is a 32-bit type that is not the same as an 8-bit C char. (Nor is a Julia String equivalent to an array of Char.) The equivalent of char in Julia is usually taken as UInt8 (though technically it is Cchar if you care about signedness).

However, const char *stuff[] = {"one", "two", "three"} is not stored in memory as the equivalent of Vector{Vector{UInt8}} in Julia, so you anyway can’t use jl_ptr_to_array_1d in this way.

(This is the pitfall of inter-language calling — you need to have some understanding of how types are represented in memory in both languages.)

My generic advice for calling Julia from C is to (a) call C from Julia instead if possible and (b) in any case, write most of the “glue” code in Julia where it is much easier. It is (relatively) easy to write a Julia wrapper function with a C-compatible API and get a C function pointer that you can call on the C side as needed — I wrote an example here.

mkitti · June 22, 2023, 11:09pm

Here is a C program that does what you want.

#include <julia.h>
#include <stdlib.h>

JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[]) {
    const char *stuff[] = {"one", "two", "three"};

    jl_init();

    jl_value_t *array_type = jl_apply_array_type((jl_value_t *)jl_uint8pointer_type, 1);
    jl_array_t *julia_stuff = jl_ptr_to_array_1d(array_type, stuff, 3, 0);
    jl_function_t *prnt = jl_eval_string("x->println(unsafe_string.(x))");
    jl_call1(prnt, (jl_value_t *)julia_stuff);

    jl_atexit_hook(0);
    return EXIT_SUCCESS;
}

Running this, I get the following.

$ ./frylock_test.exe
["one", "two", "three"]

I concur with stevengj. You want to do as much as possible on the Julia side. If you want to initiate this from C though, you should make extensive use of jl_eval_string.

I made a few more variants.

In “frylock_test2.c”, I shift more of the work to the Julia side and just box the pointer.

$ cat frylock_test2.c
#include <julia.h>
#include <stdlib.h>

JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[]) {
    const char *stuff[] = {"one", "two", "three"};

    jl_init();

    jl_value_t *ptr = jl_box_uint8pointer((uint8_t *)stuff);
    jl_function_t *wrap = jl_eval_string("x->unsafe_wrap(Vector{Ptr{UInt8}}, Ptr{Ptr{UInt8}}(x), 3; own = false)");
    jl_value_t *julia_stuff = jl_call1(wrap, (jl_value_t *)ptr);
    jl_function_t *prnt = jl_eval_string("x->println(unsafe_string.(x))");
    jl_call1(prnt, julia_stuff);

    jl_atexit_hook(0);
    return EXIT_SUCCESS;
}

In “frylock_test3.c”, I did the broadcasting manually based on the @code_lowered Julia.

$ cat frylock_test3.c
#include <julia.h>
#include <stdlib.h>

JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[]) {
    const char *stuff[] = {"one", "two", "three"};

    jl_init();

    jl_value_t *array_type = jl_apply_array_type((jl_value_t *)jl_uint8pointer_type, 1);
    jl_array_t *julia_stuff = jl_ptr_to_array_1d(array_type, stuff, 3, 0);
    jl_function_t *julia_println = jl_get_function(jl_base_module, "println");
    jl_function_t *julia_broadcasted = jl_get_function(jl_base_module, "broadcasted");
    jl_function_t *julia_materialize = jl_get_function(jl_base_module, "materialize");
    jl_function_t *julia_unsafe_string = jl_get_function(jl_base_module, "unsafe_string");
    jl_value_t *ans = jl_call2(julia_broadcasted, julia_unsafe_string, (jl_value_t *)julia_stuff);
    ans = jl_call1(julia_materialize, ans);
    ans = jl_call1(julia_println, ans);

    jl_atexit_hook(0);
    return EXIT_SUCCESS;
}

In “frylock_test4.c”, I do the conversion from C strings to Julia strings via jl_cstr_to_string.

$ cat frylock_test4.c
#include <julia.h>
#include <stdlib.h>

JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[]) {
    const char *stuff[] = {"one", "two", "three"};
    jl_value_t *julia_strings[3];

    jl_init();

    for(int i = 0; i < 3; ++i) {
      julia_strings[i] = jl_cstr_to_string(stuff[i]);
    }

    jl_value_t *array_type = jl_apply_array_type((jl_value_t *)jl_string_type, 1);
    jl_array_t *julia_stuff = jl_ptr_to_array_1d(array_type, julia_strings, 3, 0);
    jl_function_t *julia_println = jl_get_function(jl_base_module, "println");
    jl_call1(julia_println, (jl_value_t *)julia_stuff);

    jl_atexit_hook(0);
    return EXIT_SUCCESS;
}

I did this in MSYS2 on Windows, so my compilation steps and running steps were as follows.

gcc frylock_test.c -I "/path/to/julia/include/julia" -L "/path/to/julia/bin" -ljulia -o frylock_test -g
PATH="/path/to/julia/bin" frylock_test

frylock · June 23, 2023, 12:07pm

Wow! thanks folks!

frylock · June 23, 2023, 1:15pm

In frylock_test3.c your lines:

jl_function_t *julia_broadcasted = jl_get_function(jl_base_module, "broadcasted");
jl_function_t *julia_materialize = jl_get_function(jl_base_module, "materialize");

… when running in the REPL, I figured I could get help on those with ? materialize … but help isn’t available? (however the C code worked just fine). I thought jl_get_function got Julia functions? Where are those at?

EDIT: Never mind, turns out that they have to be looked up as:
? Base.broadcasted and ? Base.materialize.

mkitti · June 23, 2023, 6:32pm

Perhaps it would help to explain how I found that because you normally do not use broadcasted and materialize directly in Julia.

Basically I asked for the unoptimized lowered code.

julia> ptrs = pointer.(["one", "two", "three"])
3-element Vector{Ptr{UInt8}}:
 Ptr{UInt8} @0x0000022b4664b5a0
 Ptr{UInt8} @0x0000022b4664b5b8
 Ptr{UInt8} @0x0000022b4664b5d0

julia> unsafe_string.(ptrs)
3-element Vector{String}:
 "one"
 "two"
 "three"

julia> @code_lowered unsafe_string.(ptrs)
CodeInfo(
1 ─ %1 = Base.broadcasted(Main.unsafe_string, x1)
│   %2 = Base.materialize(%1)
└──      return %2
)

vchuravy · June 23, 2023, 11:11pm

    jl_function_t *julia_unsafe_string = jl_get_function(jl_base_module, "unsafe_string");

There is a C-API function for this: jl_pchar_to_string

So you could also do something like:

const char *stuff[] = {"one", "two", "three"};
jl_value_t *val = NULL;
jl_value_t *arr = (jl_value_t*)jl_alloc_array_1d(jl_string_type);

JL_GC_PUSH2(&val, &arr);

for (int i = 0; i< 3; i++) {
   val = jl_pchar_to_string(stuff[i]);
   jl_array_ptr_1d_push(arr, val);
}
val = NULL;

jl_(arr); # or use same as above to access the real `show`

mkitti · June 26, 2023, 5:09pm

Ah, I see that jl_cstr_to_string that I used in frylock_test4.c just calls jl_pchar_to_string:

github.com

JuliaLang/julia/blob/863057689bc3444c363147a101770250b6b8977c/src/array.c#L519


      
                      jl_throw(jl_memory_exception);
                  s = jl_gc_big_alloc_noinline(ptls, allocsz);
              }
              jl_set_typetagof(s, jl_string_tag, 0);
              maybe_record_alloc_to_profile(s, len, jl_string_type);
              *(size_t*)s = len;
              jl_string_data(s)[len] = 0;
              return s;
          }
          
          JL_DLLEXPORT jl_value_t *jl_pchar_to_string(const char *str, size_t len)
          {
              jl_value_t *s = jl_alloc_string(len);
              if (len > 0)
                  memcpy(jl_string_data(s), str, len);
              return s;
          }
          
          JL_DLLEXPORT jl_value_t *jl_cstr_to_string(const char *str)
          {
              return jl_pchar_to_string(str, strlen(str));

But jl_pchar_to_string takes two arguments…

frylock · June 26, 2023, 8:26pm

I had to do some minor tinkering to get it to work on my system, but here it is in context:

#include <julia.h>
#include <stdlib.h>
#include <stdio.h>

JULIA_DEFINE_FAST_TLS
#define MAX_LEN 128

int main(int argc, char *argv[]) {
    const char *stuff[] = {"hello", "there", "my", "friend"};

    jl_init();
    jl_value_t *val = NULL;
    jl_value_t *array_type = jl_apply_array_type((jl_value_t *)jl_string_type, 1);
    jl_array_t *array = jl_alloc_array_1d((jl_value_t *)array_type, 0);
    JL_GC_PUSH2(&val, &array);

    for (size_t i = 0; i < 4; ++i) {
        val = jl_pchar_to_string(stuff[i], strnlen(stuff[i], MAX_LEN));
        jl_array_ptr_1d_push(array, val);
    }
    val = NULL;

    // Have Julia print your vector.
    jl_function_t *prnt = jl_get_function(jl_base_module, "println");
    jl_call1(prnt, (jl_value_t *)array);

    jl_atexit_hook(0);
    return EXIT_SUCCESS;
}

frylock · June 26, 2023, 8:29pm

And, just for fun, here is your code snippet printing out the contents of a std::vector<std::string>:

#include <iostream>
#include <string>
#include <vector>

#include <julia.h>

using std::string;

JULIA_DEFINE_FAST_TLS

int main(int argc, char *argv[]) {
    std::vector<string> stuff = {"hello", "there", "my", "friend"};

    jl_init();
    jl_value_t *val = nullptr;
    jl_value_t *array_type = jl_apply_array_type((jl_value_t *)jl_string_type, 1);
    jl_array_t *array = jl_alloc_array_1d((jl_value_t *)array_type, 0);
    JL_GC_PUSH2(&val, &array);

    for (auto s : stuff) {
        val = jl_pchar_to_string(s.c_str(), s.length());
        jl_array_ptr_1d_push(array, val);
    }
    val = nullptr;

    // Have Julia print your vector.
    jl_function_t *prnt = jl_get_function(jl_base_module, "println");
    jl_call1(prnt, (jl_value_t *)array);

    jl_atexit_hook(0);
    return EXIT_SUCCESS;
}