# Local expr with local variables - aka metaprogramming C++

**URL:** <https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501>\
**Category:** General Usage\
**Tags:** ccall, macros, cxx, eval\
**Created:** [June 7, 2021, 8:21am UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501 "2021-06-07T08:21:40Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 7, 2021, 8:21am UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/1 "2021-06-07T08:21:40Z")

</div>

Hi all,

I am trying to use a C++ library by creating C++ code dynamically (i.e. meta-programming C++). I’m using `Cxx` package for this.

In `Cxx` there’s a macro-string named `icxx_str` which basically wraps the code inside the argument string in a C++ function and executes it.

It looks like this ([source code](https://github.com/JuliaInterop/Cxx.jl/blob/master/src/cxxstr.jl#L736)):

```julia
macro icxx_str(str, args...)
    annotations = length(args) > 0 ? first(args) : ""
    esc(process_cxx_string(str, false, false, __source__ , annotations))
end

```

A simple usage:

```julia
# cxx is for global scope
cxx"include <iostream>;"

# icxx can interpolate variables
a_variable = "hello, world!"
icxx"std::cout << $a_variable << std::endl;"

```

I am creating the code dynamically, e.g.

```julia
function test()
    a_variable = [4, 5]
    namespace = "std"
    cppcode = "$namespace::cout << \$(conversion_fn(a_variable)) << $namespace.endl;"
end

```

Here, `cppcode` is a string where some variables are interpolated (`$namespace`), while other variables (`conversion_fn(a_variable)`) are meant to be passed as arguments and thus should be interpolated by `icxx_str`, which also takes care of proper type conversion. `conversion_fn` takes care of converting `Vector{Int64}` to a type that `icxx` knows how to convert (namely: `cxxt"std::vector<double>"`)

Now, I need to put variable `cppcode` as argument to `icxx_str`; however, if I do `@icxx_str(cppcode)` it obviously tries to use `"cppcode"` as string.

Thanks to the Slack chat, I came up with `@eval @icxx_str($cppcode)`, which works if there are no local variables that must be interpolated by `icxx_str`, because `@eval` works in the global scope. Indeed, it works in the REPL, but not in a function!

How can I do?

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 7, 2021, 12:42pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/2 "2021-06-07T12:42:51Z")

</div>

Ok, right now I’m doing the following, which is very ugly and very pythonic (~= no metaprogramming):

```julia
function make_test_fn(namespace)
    eval(Meta.parse("""
        function test(a_variable=[4, 5])
            icxx"$namespace::cout << \$(conversion_fn(a_variable)) << $namespace.endl;"
        end
    """))
end

```

* * *

To ensure that the function is created at inclusion/import time:

```julia
for namespace in ["std", "other_name_space"]
    eval(Meta.parse("""
        function test(a_variable=[4, 5])
            icxx"$namespace::cout << \$(conversion_fn(a_variable)) << $namespace.endl;"
        end
    """))
end

```

---

<div class="post-metadata">

**Author:** ![Skoffer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/skoffer/32/378_2.png) [@Skoffer](https://discourse.julialang.org/u/Skoffer)\
**Post date:** [June 7, 2021, 1:12pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/3 "2021-06-07T13:12:13Z")

</div>

Why not use [Mustache.jl](https://github.com/jverzani/Mustache.jl) or any other template engine? It gives flexibility and there is no need to use `eval(Meta.parse` combination.

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 7, 2021, 1:20pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/4 "2021-06-07T13:20:19Z")

</div>

IMHO, Mustache is just too complex for simply defining a function. Julia should have the ability to run `eval` in the local scope, instead.

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [June 7, 2021, 4:10pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/5 "2021-06-07T16:10:20Z")

</div>

> [@sapo](#):
>
> Julia should have the ability to run `eval` in the local scope, instead.

Julia does not have this ability because it would break everything that makes it fast, so it will probably never support it, it is a design decision.

> [@sapo](#):
>
> Ok, right now I’m doing the following, which is very ugly and very pythonic

You should not be using `Meta.parse` but instead the `quote` block. You are adding an unnecessary extra phase of syntax parsing.

Anyways, taking a step back and looking at the whole picture, I think that the right way to do that is using Julia to create C++ files, call the C++ compiler over them, and then use Cxx to link to the compiled libraries. See: [Examples · Cxx.jl](https://juliainterop.github.io/Cxx.jl/stable/examples/#Using-C-with-shared-libraries-1)

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 7, 2021, 4:20pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/6 "2021-06-07T16:20:39Z")

</div>

> [@Henrique\_Becker](#):
>
> Julia does not have this ability because it would break everything that makes it fast, so it will probably never support it, it is a design decision.

There are many things that Julia can do and that lower the performance. Having the ability to run `eval` in local scope just adds an option for these somehow rare cases.

> [@Henrique\_Becker](#):
>
> You should not be using `Meta.parse` but instead the `quote` block. You are adding an unnecessary extra phase of syntax parsing.

Sorry, I’m pretty new to Julia. Can you provide me an example?

> [@Henrique\_Becker](#):
>
> Anyways, taking a step back and looking at the whole picture, I think that the right way to do that is using Julia to create C++ files, call the C++ compiler over them, and then use Cxx to link to the compiled libraries. See: [Examples · Cxx.jl](https://juliainterop.github.io/Cxx.jl/stable/examples/#Using-C-with-shared-libraries-1)

That’s weird… Cxx is nice because allows to almost completely avoid worrying about compilation and C++ stuffs. After all, `icxx` _is_ creating a C++ file and compiling, isn’t it?

---

<div class="post-metadata">

**Author:** ![Syx\_Pek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/syx_pek/32/6364_2.png) [@Syx\_Pek](https://discourse.julialang.org/u/Syx_Pek)\
**Post date:** [June 7, 2021, 5:18pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/7 "2021-06-07T17:18:32Z")

</div>

I would like to note a few things.

1. The main issue is here is the confusion of compile-time and runtime. What you’re trying to do is a kind of double-compilation. First we create a Julian function, which generates C++ code, which then gets recompiled into a Julian expression and called. This workflow is a little unusual, and should really be carefully thought whether it is necessary.
2. Having a local eval is very problematic for code optimization. This is not something that one can just turn on (easily) and say that the code should always run fast when no eval is ever touched. The rough idea is that having a local eval dramatically changes the guarantees one has in the code. Is it possible to introspect to make sure no local evals exist and do everything the current julian way to keep everything fast, while having local evals when necessary as a slow route. Probably. But it is probably not worth (and explicitly chosen against as a design feature) to do so.
3. The recommended workflow I have for your particular problem is the following. Using string interpolation to generate the necessary C++ code entirely, then later in at the end of your script, generate the `Meta.parse` calls with `Cxx.jl` in global scope. In some sense, it is wrong to want to make it work in a function because it is the wrong “time” (see again the confusion of runtime and compile-time in 1)

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [June 7, 2021, 5:27pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/8 "2021-06-07T17:27:32Z")

</div>

> [@sapo](#):
>
> There are many things that Julia can do and that lower the performance. Having the ability to run `eval` in local scope just adds an option for these somehow rare cases.

I think I did not express myself clearly enough. Having the possibility of executing `eval` in local scope would not make things slow only when you use this feature, it would make the _whole language_ much slower, independently if this feature is used or not. Basically, the fact that, for example, Python allow things similar to `eval` in local scope that makes it much slower than Julia.

> [@sapo](#):
>
> Sorry, I’m pretty new to Julia. Can you provide me an example?

You should not be doing metaprogramming in Julia without [reading the relevant section of the manual](https://docs.julialang.org/en/v1/manual/metaprogramming/#Expressions-and-evaluation).

> [@sapo](#):
>
> After all, `icxx` _is_ creating a C++ file and compiling, isn’t it?

Probably. The problem here is that `icxx` is probably a macro because this forces the user to put the C++ code directly into the source, and then this step of C++ compilation and linking may be done in a precompiling phase. You are trying to bypass this by using `eval` and, therefore, it seems to me that your use case is not the use case that `icxx` focus on simplifying/solving. You would be better using low level machinery yourself if you want to generate C++ code on the fly because it seems `icxx` was not made to help in this specific situation.

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 8, 2021, 10:48am UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/9 "2021-06-08T10:48:18Z")

</div>

> [@Syx\_Pek](#):
>
> The main issue is here is the confusion of compile-time and runtime. What you’re trying to do is a kind of double-compilation. First we create a Julian function, which generates C++ code, which then gets recompiled into a Julian expression and called. This workflow is a little unusual, and should really be carefully thought whether it is necessary.

You’re right, the example here is confusing. In my real code, the function is generated at inclusion/import time.

> [@Henrique\_Becker](#):
>
> You should not be doing metaprogramming in Julia without [reading the relevant section of the manual](https://docs.julialang.org/en/v1/manual/metaprogramming/#Expressions-and-evaluation).

I know about it. I have this problem though:

```julia
function_name = "my_function"
fn = quote
function $function_name()
   println("Hello")
end
end # quote
eval(fn)

```

creates error for the way function name is used

---

<div class="post-metadata">

**Author:** ![Syx\_Pek](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/syx_pek/32/6364_2.png) [@Syx\_Pek](https://discourse.julialang.org/u/Syx_Pek)\
**Post date:** [June 8, 2021, 11:07am UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/10 "2021-06-08T11:07:04Z")

</div>

On the last problem. You can just

```julia
julia> function_name = "my_function";

julia> fnexpr = quote
           function $(Symbol(function_name))()
               println("Hello")
           end
       end;

julia> eval(fnexpr)

```

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 8, 2021, 11:22am UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/11 "2021-06-08T11:22:49Z")

</div>

> [@sapo](#):
>
> I am trying to use a C++ library by creating C++ code dynamically (i.e. meta-programming C++). I’m using `Cxx` package for this.

Perhaps we can suggest a better approach than the one you’re currently trying to use if you tell us what library you’re trying to use?

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 8, 2021, 12:37pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/12 "2021-06-08T12:37:23Z")

</div>

Well, the wrapper is almost completely done… by the way, it’s Essentia C++ library

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 8, 2021, 12:48pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/13 "2021-06-08T12:48:19Z")

</div>

> [@Syx\_Pek](#):
>
> On the last problem. You can just
> 
> ```julia
> julia> function_name = "my_function";
> 
> julia> fnexpr = quote
> function $(Symbol(function_name))()
> println("Hello")
> end
> end;
> 
> julia> eval(fnexpr)
> 
> ```

Ok, and for the original example?

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [June 8, 2021, 1:50pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/14 "2021-06-08T13:50:16Z")

</div>

> [@sapo](#):
>
> Ok, and for the original example?

What is different in the original example for you to be in doubt?

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 8, 2021, 1:57pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/15 "2021-06-08T13:57:05Z")

</div>

The variable is interpolated inside a `icxx"[stuffs]"`, that is: the interpolation must output a string with not quote, like in “$var”…

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [June 8, 2021, 2:08pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/16 "2021-06-08T14:08:15Z")

</div>

The quoting is to avoid the re-parsing of Julia code. The whole `icxx` literal is single Julia expression and can be assembled in separate and then interpolated inside the quote that avoids the re-parsing of the whole `function` structure.

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 8, 2021, 2:27pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/17 "2021-06-08T14:27:38Z")

</div>

`icxx` is a macro which wants a string, not an expression. the following doesn’t work (this is the problem described in the first post)

```julia
eval(quote
    icxx$cppcode
end)

eval(quote
    @icxx_str($cppcode)
end)

```

---

<div class="post-metadata">

**Author:** ![Henrique\_Becker](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/henrique_becker/32/15443_2.png) [@Henrique\_Becker](https://discourse.julialang.org/u/Henrique_Becker)\
**Post date:** [June 8, 2021, 2:53pm UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/18 "2021-06-08T14:53:47Z")

</div>

I meant building a string `"@icxx_str(\"$(escape_string(cppcode))\")"` and then interpolating it, i.e., interpolating the macro too. I have not a good environment to test it right now, does it not work? Anyway, this is just for performance, as you need to compile a C++ code behind the scenes (i.e., it is not just parsing a Julia code) the gain can be small.

---

<div class="post-metadata">

**Author:** ![sapo](https://avatars.discourse-cdn.com/v4/letter/s/ec9cab/32.png) [@sapo](https://discourse.julialang.org/u/sapo)\
**Post date:** [June 9, 2021, 8:32am UTC](https://discourse.julialang.org/t/local-expr-with-local-variables-aka-metaprogramming-c/62501/19 "2021-06-09T08:32:31Z")

</div>

Yes, I don’t think that there could be a lot of improvement since my function just calls an `icxx` macro. Also, having one string with all code inside it is more readable than splitting the body of the function into multiple statements. I think I will use the `Meta.parse` way for readability.