Problem calling string parse + eval from C++

embedding
metaprogramming
#1

s::String|>Meta.parse|>eval seems good in REPL,but not works when calling julia from C++. I have no idea why?
but the following works in both:
getproperty(Main,s::String|>Symbol) return DataType

define.jl:

#!/usr/bin/env julia

function define(a::String)
	exp="mutable struct $a\n\t a::Int\nend\n"
	println("before")
	exp|>Meta.parse|>eval
	println("after")
end

define.cpp:

#define JULIA_ENABLE_THREADING
#include <julia.h>
JULIA_DEFINE_FAST_TLS()

int main(int argc,char *argv[])
{
	jl_init();
	jl_eval_string("Base.MainInclude.include(\"define.jl\")");

	jl_function_t *obj=jl_get_function(jl_main_module, "define");
	jl_value_t *a=jl_cstr_to_string("Thost");
	JL_GC_PUSH1(&a);
	jl_call1(obj,a);
	JL_GC_POP();
};

built by:

g++ -o define -fPIC -I$JULIA_DIR/include/julia -L$JULIA-DIR/lib define.cpp -ljulia /usr/lib/libstdc++.so.6

when run the define.jl in REPL, “before" and “after” are both output.
when run the cpp by ./define, only “before” is output.

Parse String to DataType
#2

It seems that eval does not work, when calling from C++ using jl_call. Anyone knows why?

#3

“It does not work” questions rarely garner a useful response. What, specifically, did you try, what was the expected output, and what output did you get?

3 Likes
#4

:sweat_smile:Thank you! the following is my code:

function define(a::String)
	isdefined(Main,a|>Symbol) && return a
	exp="mutable struct $a\n"
	for (k,v) in tfull[a]
		line=isdigit.([k...])|>all ? "$v::NTuple{$k,Cchar}" : "$v::$(get(typet,k) do; define(k); end)"
		exp*="\t"*line*"\n"
	end
	exp*"end\n"|>Meta.parse|>eval
	println(a)
	a
end

The function dynamically and recursively defines a nested struct.
when called from C++ using jl_call, I cannot get the output of “println(a)", the problem came from eval in previous line. But the function is fine when runing in REPL. Actually, it is the same problem with that in the top post.

#5

So is the problem with println? That is, if you just replace the whole function with println(a), do you still not get output when you call it with jl_call? (When you say “I cannot get the output”, do you mean that it runs but produces no output, or that it gives some kind of error?)

#6

After remove the rest code, the println(a) get right output, so I think it is the problem of eval.

#7

I’m afraid that without being more clear about what you’re doing—what C++ code are you running?—and what happens—what is the exact output?—you are not going to get much more help.

Also, metaprogramming with strings like this is very much not recommended. It’s the only option in many dynamic languages, but Julia has a first class representation of code with which you can do real metaprogramming.

1 Like
#8

The code in C++ side

jl_function_t *obj=jl_get_function(jl_main_module, "define");
jl_value_t *a=jl_cstr_to_string("CThost");
JL_GC_PUSH1(&a);
jl_call1(obj,a);
JL_GC_POP();

May I have more your advice about the my previous Julia codes about metaprogramming? Thanks

#9

I can’t run any of your example code because it is incomplete. This is what I get when I try to run it:

julia> define("Foo")
ERROR: UndefVarError: tfull not defined
Stacktrace:
 [1] define(::String) at ./REPL[26]:4
 [2] top-level scope at REPL[28]:1

In order to give you more specific advice on how to do metaprogramming correctly, I would have to spend time to try to modify your code so that it works and then rewrite it to use expression metaprogramming instead of string metaprogramming. I’m willing to do the latter but not the former. Similarly, your C++ code is incomplete and not something I can paste into a text file and compile with gcc or clang. In order to diagnose or test it, I would have to reconstruct the rest of the C++ file in order to be able to compile it. That is not something I’m willing to spend my time on, especially since you presumably have a C++ source file that compiles already.

You are asking for help—which people are willing to give, as the many replies here indicate—but by not posting self-contained, complete code, you are creating a situation where those who would help you, not only need to do the work inherent in helping you but also need to do the extra work of turning your descriptions of code and incomplete snippets into something that can be diagnosed or tested or even understood. I’m perfectly willing to help but I am not willing to do that additional and unnecessary work that you could eliminate by posting self-contained examples to ask questions about.

#10

What Stefan is saying is:

2 Likes
#11

Got it. I modify the top post and past the simplified code and describe the output results.

#12

Your post looks good now: there’s enough information to compile and run the code locally and figure out what happened here. Here’s the cause of this confusing behavior:

  1. One thing you’ve already noticed is that Main.include is not present in embedded julia (I think this is a bug and someone else does too: https://github.com/JuliaLang/julia/issues/28825). For exactly the same reason, Main.eval is also not present so your use of eval inside define will fail, but only in embedded julia.
  2. Because eval is not present, a MethodError is thrown and you never get to see after printed.
  3. Because you don’t do any error handling on the embedded side (ie, you never call jl_exception_occurred()), you didn’t notice the MethodError.

The very simplest workaround is to use Base.eval(Main, Meta.parse(exp)) inside define.

However, as Stefan already mentioned, there’s much better ways to construct expressions in Julia than string munging. For example, in this simple case you could do it by interpolating the struct name into the expression:

 function define(struct_name::Symbol) 
    @eval mutable struct $struct_name
        a::Int
    end
    getproperty(Main, struct_name)  # Look up and return the newly generated type
end

Then call as follows:

define(:Thost)

Or from C:

    jl_function_t *obj = jl_get_function(jl_main_module, "define");
    jl_value_t *struct_name = (jl_value_t*)jl_symbol("Thost");
    JL_GC_PUSH1(&struct_name);
    jl_value_t *struct_type = jl_call1(obj, struct_name);

Having written the above, I think it’s worth asking why you want to define types dynamically like this at all? There’s good reasons of course, but if you can avoid it that’s usually even better. (One good reason would be to reflect types into julia which are only known dynamically from the C++ side.)

1 Like
#13

As you said, but more than 400 static structs in C++ side are demanded to reflect to julia, and I don’t wanna make code so long,:sweat_smile:.

#14

Root cause will be fixed in https://github.com/JuliaLang/julia/pull/32062