Having problems embedding Julia 1.0 in PureData

New to Julia
1

My teacher who created PureData also created a great external for PureData which allows interfacing between PureData and Julia. This was done to leverage Julia’s fantastic numerical computation abilities with PureData’s awesome real-time graphical signal-processing capabilities. His external worked for Julia 0.6 but it does not work for Julia 1.0. He suspects it has something to do with an updated initialization routine but he is not sure. Prior to Julia 0.6, he had to make a change to the call to Julia’s initialization routine from jl_init(“/usr”) to jl_init();
He suspects something similar is causing problems as a result of the error log from Julia. Does anyone have any advice on this matter?

Here is the error log (jl_init mention at the end of first line):

/Users/nakultiruviluamala/Documents/PD/externals/julia/JuliaNew/julia.d_fat: dlopen(/Users/nakultiruviluamala/Documents/PD/externals/julia/JuliaNew/julia.d_fat, 10): Symbol not found: _jl_init
  Referenced from: /Users/nakultiruviluamala/Documents/PD/externals/julia/JuliaNew/julia.d_fat
  Expected in: flat namespace
 in /Users/nakultiruviluamala/Documents/PD/externals/julia/JuliaNew/julia.d_fat

Many thanks,
Nakul

1 Like
2

For updating code, it is recommended to use Julia 0.7 and work through the deprecation messages. You can also try FemtoCleaner to catch common deprecations.

1 Like
3

If libjulia is linked at build time, the problem is that you need to run julia-config.jl to find out appropriate compiler and linker flags, as documented in Embedding Julia · The Julia Language.

If libjulia is dynamically loaded you have run into Dynamically loaded embedding doesn't know the spelling of jl_init · Issue #28824 · JuliaLang/julia · GitHub. Reading through https://github.com/JuliaLang/julia/pull/28886/files should point you to a solution and help you with some more issues you may encounter next.

This won’t help at all with calling into Julia from an external program but is great advice once the embedding problems have been solved.

2 Likes
4

I think the jl_init__threaded() worked! It compiles now and it doesn’t throw any errors in PureData. I’ll ask my teacher if he thinks the whole thing checks out to his satisfaction when I see him on Thursday.

Thanks a bunch!

1 Like
6

I’ve encountered a problem. There appears to be a scoping hassle, as any variable is undefined, and therefore function calls are not possible.

I get this when I call a function: exception occured in eval_string: UndefVarError which I didn’t use to get in 0.6 with the same script. I didn’t find any differences in the embedding procedure in the manual from 0.6 to 1.0 so I’m at a loss for what to do.

Here’s the C code for PureData, which contains Julia calls. I’m not exactly sure how to proceed (or how to use FemtoCleaner for this):

#include "m_pd.h"
#include <julia/julia.h>
#include <unistd.h>

static t_class *julia_class;
static int initted;

typedef struct _julia
{
    t_object x_obj;
    t_canvas *x_canvas;
} t_julia;

static void *julia_new(void)
{
    t_julia *x = (t_julia *)pd_new(julia_class);
    outlet_new(&x->x_obj, &s_);
    x->x_canvas = canvas_getcurrent();
    if (!initted)
    {
        initted = 1;
    }
    return (x);
}

static jl_value_t *julia_do_eval(t_julia *x, int argc, t_atom *argv)
{
    char cmdbuf[MAXPDSTRING+ 5];
    char atombuf[MAXPDSTRING];
    int i;
    jl_value_t *jv;
    cmdbuf[0] = 0;
    for (i = 0; i < argc; i++)
    {
    	strcat(cmdbuf, " ");
    	atom_string(&argv[i], atombuf, MAXPDSTRING);
    	strncat(cmdbuf, atombuf, MAXPDSTRING);
        cmdbuf[MAXPDSTRING-1] = 0;
    }
    strcat(cmdbuf, ";\n");
    jv = jl_eval_string(cmdbuf);
    if (jl_exception_occurred())
    {
        post("exception occured in eval_string: %s",
            jl_typeof_str(jl_exception_occurred()));
        return (0);
    }
    else return (jv);
}

static jl_value_t *julia_do_call(t_julia *x, int argc, t_atom *argv)
{
    char cmdbuf[MAXPDSTRING+ 5];
    char atombuf[MAXPDSTRING];
    int i;
    jl_value_t *jv;
    cmdbuf[0] = 0;
    for (i = 0; i < argc; i++)
    {
    	if (i)
            strcat(cmdbuf, " ");
        else
            strcat(cmdbuf, "ans = ");
    	atom_string(&argv[i], atombuf, MAXPDSTRING);
    	strncat(cmdbuf, atombuf, MAXPDSTRING);
        cmdbuf[MAXPDSTRING-1] = 0;
        if (!i)
    	    strncat(cmdbuf, "(", MAXPDSTRING);
        else if (i < argc-1)
    	    strncat(cmdbuf, ",", MAXPDSTRING);
        cmdbuf[MAXPDSTRING-1] = 0;
    }
    strncat(cmdbuf, ");\n", MAXPDSTRING);
    cmdbuf[MAXPDSTRING-1] = 0;
    /* fprintf(stderr, "%s", cmdbuf); */
    jv = jl_eval_string(cmdbuf);
    if (jl_exception_occurred())
    {
        post("exception occured in eval_string: %s",
            jl_typeof_str(jl_exception_occurred()));
        return (0);
    }
    else return (jv);
}

static void julia_eval(t_julia *x, t_symbol *s, int argc, t_atom *argv)
{
    jl_value_t *jv = julia_do_eval(x, argc, argv);
}

static void julia_call(t_julia *x, t_symbol *s, int argc, t_atom *argv)
{
    jl_value_t *jv = julia_do_call(x, argc, argv);
}

static void julia_source(t_julia *x, t_symbol *filesym)
{
    char buf[MAXPDSTRING], *bufptr, filename[MAXPDSTRING], command[MAXPDSTRING];
    int fd = open_via_path(canvas_getdir(x->x_canvas)->s_name,
         filesym->s_name, "", buf, &bufptr, MAXPDSTRING, 0);
    if (fd < 0)
    {
        pd_error(x, "%s: can't open file", filesym->s_name);
        return;
    }
    close(fd);
    strncpy(filename, buf, MAXPDSTRING);
    filename[MAXPDSTRING-2] = 0;
    strcat(filename, "/");
    strncat(filename, bufptr, MAXPDSTRING-strlen(filename));
    filename[MAXPDSTRING-1] = 0;
    post("file: %s", filename);
    snprintf(command, MAXPDSTRING, "include(\"%s\");\n", filename);
    command[MAXPDSTRING-1] = 0;
    jl_eval_string(command);
}

static void julia_set(t_julia *x, t_symbol *s, int argc, t_atom *argv)
{
    char cmdbuf[MAXPDSTRING];
    int i;
    if (argc < 2 || argv[0].a_type != A_SYMBOL)
    {
        pd_error(s, "set: usage: array-name value...");
        return;
    }
    snprintf(cmdbuf, MAXPDSTRING, "%s = [", argv[0].a_w.w_symbol->s_name);
    cmdbuf[MAXPDSTRING-1] = 0;
    for (i = 1; i < argc; i++)
    {
    	atom_string(&argv[i], cmdbuf + strlen(cmdbuf),
            MAXPDSTRING-strlen(cmdbuf) - 5);
    	if (i < argc-1)
            strcat(cmdbuf, ",");
    }
    strcat(cmdbuf, "];\n");
    jl_eval_string(cmdbuf);
    if (jl_exception_occurred())
    {
        post("exception occured in eval_string: %s",
            jl_typeof_str(jl_exception_occurred()));
        return;
    }
}

static void julia_get(t_julia *x, t_symbol *s, int argc, t_atom *argv)
{
    jl_value_t *a = julia_do_eval(x, argc, argv);
    if (!a)
        return;
    if (!jl_is_array(a))
    {
        if (jl_typeis(jl_typeof(a), jl_float64_type))
            outlet_float(x->x_obj.ob_outlet, jl_unbox_float64(a));
        else if (jl_is_int64(a))
            outlet_float(x->x_obj.ob_outlet, jl_unbox_int64(a));
        else post("julia_get: unhandled type %s",
            jl_typename_str(jl_array_eltype(a)));
    }
    else if (jl_array_eltype(a) != jl_float64_type && 
        jl_array_eltype(a) != jl_int64_type)
    {
        post("julia_get: unhandled array element type %s",
            jl_typename_str(jl_array_eltype(a)));
        return;
    }
    else
    {
        int n = jl_array_len(a), i, useint =
            (jl_array_eltype(a) == jl_int64_type);
        t_atom *at = alloca(n * sizeof(t_atom));
        for (i = 0; i < n; i++)
            SETFLOAT(at+i, 
                (useint ? ((long long *)jl_array_data(a))[i]
                    : ((double*)jl_array_data(a))[i]));
        outlet_list(x->x_obj.ob_outlet, 0, n, at);
    }
}

static void julia_toarray(t_julia *x, t_symbol *var, t_symbol *arrayname)
{
    t_garray *a;
    t_word *w;
    int nw;
    t_atom at;
    jl_value_t *ja;

    SETSYMBOL(&at, var);
    ja = julia_do_eval(x, 1, &at);
    if (!ja)
        return;

    if (!(a = (t_garray *)pd_findbyclass(arrayname, garray_class)))
    {
        pd_error(x, "julia: %s: no such array",
            arrayname->s_name);
    }
    else if (!garray_getfloatwords(a, &nw, &w))
    {
        pd_error(x, "julia: %s: needs floating-point array",
            arrayname->s_name);
    }
    else if (!jl_is_array(ja))
    {
        post("julia_getarray: %s was not an array",
            arrayname->s_name);
        return;
    }
    else if (jl_array_eltype(ja) != jl_float64_type && 
        jl_array_eltype(ja) != jl_int64_type)
    {
        post("julia_get: unhandled array element type %s",
            jl_typename_str(jl_array_eltype(ja)));
        return;
    }
    else
    {
        int n = jl_array_len(ja), i, useint =
            (jl_array_eltype(ja) == jl_int64_type);
        if (n != nw)
        {
            post("resizing");
            garray_resize(a, n);
            if (!garray_getfloatwords(a, &nw, &w))
            {
                post("%s: resize failed", arrayname->s_name);
                return;
            }
        }
        for (i = 0; i < n; i++)
            w[i].w_float = (useint ? ((long long *)jl_array_data(ja))[i]
                : ((double*)jl_array_data(ja))[i]);
        garray_redraw(a);
     }
}

static void julia_fromarray(t_julia *x, t_symbol *var, t_symbol *arrayname)
{
    t_garray *a;
    t_word *w;
    int nw, i;
    t_atom at;
    jl_value_t *ja;
    char cmdstring[MAXPDSTRING];
    jl_value_t *array_type;
    jl_array_t *array_thing;
    double *array_stuff;
    jl_value_t *func;
    
    snprintf(cmdstring, MAXPDSTRING,
        "function (x); global %s; %s = x; end\n",
        var->s_name, var->s_name);
    cmdstring[MAXPDSTRING-1] = 0;
    func = jl_eval_string(cmdstring);
    if (jl_exception_occurred())
    {
        post("exception occured in eval_string: %s",
            jl_typeof_str(jl_exception_occurred()));
        return;
    }
    if (!func)
        return;

    if (!(a = (t_garray *)pd_findbyclass(arrayname, garray_class)))
    {
        pd_error(x, "julia: %s: no such array",
            arrayname->s_name);
        return;
    }
    else if (!garray_getfloatwords(a, &nw, &w))
    {
        pd_error(x, "julia: %s: needs floating-point array",
            arrayname->s_name);
        return;
    }
    array_type = jl_apply_array_type((jl_value_t *)jl_float64_type, 1);
    array_stuff = (double*)malloc(sizeof(double) * nw);
    for (i = 0; i < nw; i++)
        array_stuff[i] = w[i].w_float;
    array_thing = jl_ptr_to_array_1d(array_type, array_stuff, nw, 1);

    ja = jl_call1((jl_function_t *)func, (jl_value_t *)array_thing);    
    if (jl_exception_occurred())
    {
        post("exception setting array: %s",
             jl_typeof_str(jl_exception_occurred()));
        return;
    }
}

void julia_setup(void)
{
    julia_class = class_new(gensym("julia"), (t_newmethod)julia_new, 0,
        sizeof(t_julia), 0, A_NULL);
    class_addmethod(julia_class, (t_method)julia_eval,
        gensym("eval"), A_GIMME, A_NULL);
    class_addmethod(julia_class, (t_method)julia_call,
        gensym("call"), A_GIMME, A_NULL);
    class_addmethod(julia_class, (t_method)julia_source,
        gensym("source"), A_SYMBOL, A_NULL);
    class_addmethod(julia_class, (t_method)julia_get,
        gensym("get"), A_GIMME, A_NULL);
    class_addmethod(julia_class, (t_method)julia_set,
        gensym("set"), A_GIMME, A_NULL);
    class_addmethod(julia_class, (t_method)julia_toarray,
        gensym("toarray"), A_SYMBOL, A_SYMBOL, A_NULL);
    class_addmethod(julia_class, (t_method)julia_fromarray,
        gensym("fromarray"), A_SYMBOL, A_SYMBOL, A_NULL);
#if (JULIA_VERSION_MAJOR > 0 || JULIA_VERSION_MINOR >= 6)
    jl_init__threading();
#else 
    jl_init("/usr");
#endif
}
7

How do you separate context for different julia nodes? It seems that every instance share the same module. I think it’s better to create different modules for each node with jl_new_module. Also, new modules doesnt` have reference for Base, they are bare, you should link them to Base. Actually, don’t use jl_new_module, instead write custom function like this from modules.c.

JL_DLLEXPORT jl_value_t *jl_f_new_module(jl_sym_t *name, uint8_t std_imports)
{
    // TODO: should we prohibit this during incremental compilation?
    jl_module_t *m = jl_new_module(name);
    JL_GC_PUSH1(&m);
    m->parent = jl_main_module; // TODO: this is a lie
    jl_gc_wb(m, m->parent);
    if (std_imports)
        jl_add_standard_imports(m);
    JL_GC_POP();
    // TODO: should we somehow try to gc-root this correctly?
    return (jl_value_t*)m;
}