# \[Yggdrasil\] How to call Julia?

**URL:** <https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586>\
**Category:** Package Management\
**Tags:** yggdrasil\
**Created:** [September 6, 2023, 9:10pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586 "2023-09-06T21:10:54Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![dourouc05](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dourouc05/32/2646_2.png) [@dourouc05](https://discourse.julialang.org/u/dourouc05)\
**Post date:** [September 6, 2023, 9:10pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/1 "2023-09-06T21:10:54Z")

</div>

Hi,

I’m starting a new Yggdrasil script (is that what you are supposed to call them?) to build a large and complex C++ library, [OR-Tools](https://github.com/google/or-tools). The end goal is to provide a Julia wrapper around it.

At some point during the build, I would need to generate Julia code (automatically, that is) using ProtoBuf.jl, during the build process of the library to ensure that the generated code is up-to-date with respect to the library.

However, in my `script` (in `build_tarballs.jl`), I cannot call `julia`: the command is not recognised! Here is the exact error message ([full log](https://buildkite.com/julialang/yggdrasil/builds/4881#018a5b48-2a59-4984-95ff-c64848f3054b)):

```julia
[13:45:47] ---> julia -e "using ProtoBuf; protojl()"
[13:45:47] /bin/bash: line 50: julia: command not found
[13:45:47] ---> julia -e "using ProtoBuf; protojl()"
[13:45:47] ---> julia -e "using ProtoBuf; protojl()"
[13:45:47] Previous command exited with 127
[13:45:47] Child Process exited, exit code 127

```

Here was my `script`:

```julia
cd $WORKSPACE/srcdir/or-tools*
# Apply patches.
mkdir build
cmake -S. -Bbuild …
cmake --build build
cmake --build build --target install
julia -e "using ProtoBuf; protojl()"

```

I know that the provided Julia script has zero chance of being correct, but the failure is that Julia cannot be built. How is that possible? Do I have to use some magic dependency? I can only find `libjulia_jll` that I could depend on, but that does not seem even remotely what I need (it only exposes `libjulia`, not the standard Julia interpreter).

Thanks!

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [September 6, 2023, 9:26pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/2 "2023-09-06T21:26:54Z")

</div>

I think you can just interpolate `Base.julia_cmd()` into the build string. e.g.

```julia
script = """
cd \$WORKSPACE/srcdir/or-tools*
# Apply patches.
mkdir build
cmake -S. -Bbuild …
cmake --build build
cmake --build build --target install
$(Base.julia_cmd()) -e "using ProtoBuf; protojl()"
"""

```

Note that if you want to interpolate into the string, you can’t use `raw""" ... """`, so you have to escape any literal `$` symbols you want to stay in the script, e.g. for `$WORKSPACE`.

---

<div class="post-metadata">

**Author:** ![dourouc05](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dourouc05/32/2646_2.png) [@dourouc05](https://discourse.julialang.org/u/dourouc05)\
**Post date:** [September 6, 2023, 10:09pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/3 "2023-09-06T22:09:16Z")

</div>

Thanks, I’m giving this a try!

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [September 6, 2023, 10:12pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/4 "2023-09-06T22:12:52Z")

</div>

Aren’t you better off making `ORTools_jll.jl` which just builds the library, and then `ORTools.jl` which provides the Julia wrapper?

---

<div class="post-metadata">

**Author:** ![dourouc05](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dourouc05/32/2646_2.png) [@dourouc05](https://discourse.julialang.org/u/dourouc05)\
**Post date:** [September 6, 2023, 10:33pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/5 "2023-09-06T22:33:24Z")

</div>

That’s the plan, ORTools.jl will contain the actual MOI wrappers.

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [September 6, 2023, 10:39pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/6 "2023-09-06T22:39:29Z")

</div>

So why does the jll need Julia code?

---

<div class="post-metadata">

**Author:** ![dourouc05](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dourouc05/32/2646_2.png) [@dourouc05](https://discourse.julialang.org/u/dourouc05)\
**Post date:** [September 6, 2023, 11:01pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/7 "2023-09-06T23:01:05Z")

</div>

For the _generated_ code, unrelated to MOI, using ProtoBuf.jl. Storing this code near the JLL ensures that it always corresponds to the installed binary, with no risk that the generated code is out of date and no manual operation to update it (unlike CPLEX.jl or Gurobi.jl) — although doing everything in the .jl package would be simpler.

---

<div class="post-metadata">

**Author:** ![dourouc05](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dourouc05/32/2646_2.png) [@dourouc05](https://discourse.julialang.org/u/dourouc05)\
**Post date:** [September 10, 2023, 2:29pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/8 "2023-09-10T14:29:22Z")

</div>

I’ve tried this solution, but the corresponding Julia binary [is not accessible](https://buildkite.com/julialang/yggdrasil/builds/5000#018a773e-ecc7-40f3-90c7-aa990d36a747):

```julia
/bin/bash: line 78: /cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.7/julia-1.7-latest-linux-x86_64/bin/julia: No such file or directory

```

The full value of `$(Base.julia_cmd())` is:

```julia
/cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.7/julia-1.7-latest-linux-x86_64/bin/julia -Cnative -J/cache/julia-buildkite-plugin/julia_installs/bin/linux/x64/1.7/julia-1.7-latest-linux-x86_64/lib/julia/sys.so -g1

```

The `/cache/` mounting is not available at all (I’ve tried `ls` on it), it’s more than just this plugin.

I also tried to download the latest version of Julia, depending on the installed libc:

```julia
if [["$MACHTYPE" == *musl]]
then
  curl -o julia-1.9.3.tar.gz https://julialang-s3.julialang.org/bin/musl/x64/1.9/julia-1.9.3-musl-x86_64.tar.gz
else
  curl -o julia-1.9.3.tar.gz https://julialang-s3.julialang.org/bin/linux/x64/1.9/julia-1.9.3-linux-x86_64.tar.gz
fi
tar -xvf julia-1.9.3.tar.gz
julia-1.9.3/bin/julia -e 'using InteractiveUtils; versioninfo()'

```

The glibc version fails because of the libc, [it seems](https://buildkite.com/julialang/yggdrasil/builds/4986#018a754a-fb13-4677-be29-39c23c2acb86):

```julia
[15:13:36] ---> julia-1.9.3/bin/julia -e 'using InteractiveUtils; versioninfo()'
[15:13:36] WARNING: failed to select UTF-8 encoding, using ASCII
[15:13:36] 
[15:13:36] [20864] signal (11.1): Segmentation fault
[15:13:36] in expression starting at none:0
[15:13:36] get_random_secret at /workspace/srcdir/musl-1.2.2/src/malloc/mallocng/glue.h:45 [inlined]
[15:13:36] __malloc_alloc_meta at /workspace/srcdir/musl-1.2.2/src/malloc/mallocng/malloc.c:50
[15:13:36] unknown function (ip: 0x1c4d20d)
[15:13:36] Allocations: 2997 (Pool: 2985; Big: 12); GC: 0
[15:13:36] /bin/bash: line 78: 20864 Segmentation fault julia-1.9.3/bin/julia -e 'using InteractiveUtils; versioninfo()'

```

Using the musl libc yields [an even stranger error](https://buildkite.com/julialang/yggdrasil/builds/4990#018a75a5-eed5-4bbd-87d1-c8831c0acd89):

```julia
[16:39:47] ---> julia-1.9.3/bin/julia -e 'using InteractiveUtils; versioninfo()'
[16:39:47] ERROR: Unable to dladdr(&jl_get_libdir)!
[16:39:47] ---> julia-1.9.3/bin/julia -e 'using InteractiveUtils; versioninfo()'
[16:39:47] ---> julia-1.9.3/bin/julia -e 'using InteractiveUtils; versioninfo()'
[16:39:47] Previous command exited with 1
[16:39:47] Child Process exited, exit code 1

```

---

<div class="post-metadata">

**Author:** ![odow](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/odow/32/28685_2.png) [@odow](https://discourse.julialang.org/u/odow)\
**Post date:** [September 10, 2023, 11:10pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/9 "2023-09-10T23:10:59Z")

</div>

I’d just do this in the `.jl` package. It’s a lot easier.

---

<div class="post-metadata">

**Author:** ![jmair](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jmair/32/35117_2.png) [@jmair](https://discourse.julialang.org/u/jmair)\
**Post date:** [September 11, 2023, 8:00am UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/10 "2023-09-11T08:00:45Z")

</div>

To add to @odow’s suggestion to do this in the wrapper - if you want to make sure that it will always use the correct `jll`, you can use tight constraints in the compat section of your wrapper (e.g. [6. Compatibility · Pkg.jl](https://pkgdocs.julialang.org/v1/compatibility/#Equality-specifier)).

When you update the version of the `jll` and the bindings in your wrapper, bump the version in the compat, and it all should stay closely coupled.

---

<div class="post-metadata">

**Author:** ![dourouc05](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dourouc05/32/2646_2.png) [@dourouc05](https://discourse.julialang.org/u/dourouc05)\
**Post date:** [September 22, 2023, 9:29pm UTC](https://discourse.julialang.org/t/yggdrasil-how-to-call-julia/103586/11 "2023-09-22T21:29:37Z")

</div>

To answer the original question: there is no way to call Julia from within Yggdrasil. That’s a bug no one is interested in solving (apparently, I’m the first one to have the problem). Therefore, do everything in the Julia package(s) you are building you JLL for, possibly throwing a new package in for good measure.