# Using Julia from the JVM (Clojure)

**URL:** <https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052>\
**Category:** Package Announcements\
**Created:** [December 1, 2020, 4:00pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052 "2020-12-01T16:00:18Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 15, 2020, 12:08am UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/21 "2020-12-15T00:08:08Z")

</div>

Release of [libjulia-clj 0.02](https://clojars.org/cnuernber/libjulia-clj) is up with some [documentation on the JVM and signals](https://cnuernber.github.io/libjulia-clj/signals.html).

If you intend to use libjulia-clj with Julia and enable Julia’s threading mechanism then you probably should read this document :-). @mkitti and myself just did quite a deep dive into how the JVM and Julia deal with signals and figured out a middle path that allows both to coexist peacefully.

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 16, 2020, 8:45pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/22 "2020-12-16T20:45:30Z")

</div>

@mkitti - Simple demos work for diffeq lib:

```nohighlight

user> (require '[libjulia-clj.julia :as julia])
nil
user> (julia/initialize! {:n-threads -1})
13:33:32.571 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Attempting to initialize Julia at /home/chrisn/dev/cnuernber/libjulia-clj/julia-1.5.3/lib/libjulia.so
13:33:32.624 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO tech.v3.jna.base - Library /home/chrisn/dev/cnuernber/libjulia-clj/julia-1.5.3/lib/libjulia.so found at [:system "/home/chrisn/dev/cnuernber/libjulia-clj/julia-1.5.3/lib/libjulia.so"]
13:33:32.640 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.jna - Julia startup options: n-threads -1, signals? true
:ok
user> (require '[libjulia-clj.modules.DifferentialEquations :as diffeq])
13:34:08.754 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Attempting to initialize Julia at /home/chrisn/dev/cnuernber/libjulia-clj/julia-1.5.3/lib/libjulia.so
nil
user> (def f (julia/eval-string "f(u,p,t) = 0.98u"))
13:36:11.268 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413C2A36C0
#'user/f
user> (def u0 1.0)
#'user/u0
user> (def tspan (julia/tuple 0.0 1.0))
13:36:58.599 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F41390CB310
#'user/tspan
user> tspan
(0.0, 1.0)
user> (def prob (diffeq/ODEProblem f u0 tspan))
13:37:40.532 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413D3A8A30
13:37:40.533 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413D3A8AC0
#'user/prob
user> prob
ODEProblem with uType Float64 and tType Float64. In-place: false
timespan: (0.0, 1.0)
u0: 1.0
user> (def sol (diffeq/solve prob))
13:38:20.848 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413D1EDF90
13:38:20.848 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413D1EE050
#'user/sol
user> sol
retcode: Success
Interpolation: automatic order switching interpolation
t: [0.0, 0.10042494449239292, 0.35218603951893646, 0.6934436028208104, 1.0]
u: [1.0, 1.1034222047865465, 1.4121908848175448, 1.9730384275622996, 2.664456142481451]

```

- [Diffeq docs](https://cnuernber.github.io/libjulia-clj/libjulia-clj.modules.DifferentialEquations.html)

There is more to do here; I haven’t setup a way to get field values from a thing only call it as a function but, after waiting quite a while for diffeq to compile, it appears to work.

Speaking of which:

- I should redirect logging to the normal logging pathways. Do you have an example of someone writing a custom logger? I had zero feedback during diffeq compilation and killed the process to try it from Julia then saw the log message.
- Does Julia have an object-oriented way to override stderr/stdout or do they have to be C-files? In python I could implement an object and set that as std err and stdout and this really helps debugging things. Perhaps for Julia simply setting up logging correctly is fine; I just do not know.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 17, 2020, 7:07am UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/23 "2020-12-17T07:07:36Z")

</div>

On redirecting stderr/stdout, see  
[https://docs.julialang.org/en/v1/base/io-network/#Base.redirect\_stdout](https://docs.julialang.org/en/v1/base/io-network/#Base.redirect_stdout)

For Logging see  
[https://docs.julialang.org/en/v1/stdlib/Logging/#Writing-log-events-to-a-file](https://docs.julialang.org/en/v1/stdlib/Logging/#Writing-log-events-to-a-file)

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 17, 2020, 8:52pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/24 "2020-12-17T20:52:10Z")

</div>

Stuck on general ccall wrapping:

```nohighlight
user> (def fn-wrapper (julia/eval-string "function(fn-ptr::Ptr{Nothing}) return function(a...) return ccall(fn-ptr, Any, (Any,), a) end end"))
13:45:05.497 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413C2A3C28
#'user/fn-wrapper
user> (def jl-fn (fn-wrapper jl-voidp))
Execution error at libjulia-clj.impl.base/check-last-error (base.clj:131).
Julia error:
MethodError: no method matching -(::Ptr{Nothing})
Closest candidates are:
  -(::Any, !Matched::Ptr{Nothing}) at none:1
  -(::Any, !Matched::Any) at none:1

```

Help would be appreciated :-).

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [December 17, 2020, 8:56pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/25 "2020-12-17T20:56:13Z")

</div>

The problem with your code is that `fn-ptr` is not a valid identifier in Julia and is instead parsed as `(-)(fn, ptr)`.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [December 17, 2020, 9:01pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/26 "2020-12-17T21:01:22Z")

</div>

> [@simeonschaub](#):
>
> `fn-ptr` is not a valid identifier in Julia

Julia is lisp-y, but not to the point of allowing dashes in identifiers 😜

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 17, 2020, 10:32pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/27 "2020-12-17T22:32:50Z")

</div>

Haha, thanks for the help :-).

```nohighlight

user> (def fn-wrapper (julia/eval-string "function(fn_ptr::Ptr{Nothing}) return function(a...) return ccall(fn_ptr, Any, (Any,), a) end end"))
15:30:01.547 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413C2A3CA0
#'user/fn-wrapper
user> (def jl-voidp (libjulia-clj.impl.base/fn->jl println))
15:30:06.034 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413CE0C950
#'user/jl-voidp
user> (def new-fn (fn-wrapper jl-voidp))
15:30:10.676 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413CE0D1E0
15:30:10.677 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413CE0D240
#'user/new-fn
user> (new-fn 1 2 3 "hey")
15:30:19.283 [nRepl-session-5d74e137-6c8c-4d00-bd4b-4ce1958fb7be] INFO libjulia-clj.impl.base - Rooting address 0x00007F413DCF33D0
1 2 3 hey
nil

```

That functionality survives a forced GC run of the JVM and julia so it is robust enough for now :-).

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 18, 2020, 8:33pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/28 "2020-12-18T20:33:52Z")

</div>

I found how pyjulia manages [stdout and stderr](https://github.com/JuliaPy/pyjulia/blob/2dea434fcbed01a838184dd0639750e77ab9e3f0/src/julia/pyjulia_helper.jl#L156). I have a question about references and GC. The python code maintains the new stdout, stderr pipes in global variables. But the code also closes over them like such:

```julia
    # At file scope
    const read_stdout = Ref{Base.PipeEndpoint}()

    # At local scope
    global readout_task
    read_stdout[], = redirect_stdout()
    readout_task = @async pipe_stream(read_stdout[], out_receiver)

```

I could see the readout task needing to be global else the GC could clean it up but I do not understand why the read\_stdout needs to be global. It is closed over in pipe\_stream.

Not a big issues as I can just copy the code and it was written by people with a lot more Julia knowledge than I have. Just trying to get a feel for some of the more subtle issues of Julia GC.

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 19, 2020, 7:44pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/29 "2020-12-19T19:44:22Z")

</div>

Nex issue: – tasks appear to be failing:

```nohighlight
user> (def sync-fn (julia/eval-string "function(receiver) receiver(\"heyyou\") end"))
#'user/sync-fn
user> (def async-fn (julia/eval-string "function(receiver) @async receiver(\"heyyou\") end"))
#'user/async-fn
user> (sync-fn (fn [data] (println "data was" data) data))
data was heyyou
"heyyou"
user> (def task (async-fn (fn [data] (println "data was" data) data)))
#'user/task
user> (def started? (julia/eval-string "istaskstarted"))
#'user/started?
user> (def done? (julia/eval-string "istaskdone"))
#'user/done?
user> (started? task)
false
user> (done? task)
false
user> (julia/eval-string "yield()")
nil
user> (started? task)
true
user> (done? task)
true
user> ;;??? Nothing printed
user> (def fetch (julia/eval-string "fetch"))
#'user/fetch
user> (fetch task)
nil

```

I would like to use async tasks to redirect IO to Clojure but so far I cannot get any task to output anything to clojure at all.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 20, 2020, 5:22am UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/30 "2020-12-20T05:22:31Z")

</div>

It might be useful to note that Julia uses libuv to implement asynchronous I/O:  
[https://docs.julialang.org/en/v1.1/devdocs/stdio/#Libuv-wrappers-for-stdio-1](https://docs.julialang.org/en/v1.1/devdocs/stdio/#Libuv-wrappers-for-stdio-1)

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 20, 2020, 5:40am UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/31 "2020-12-20T05:40:11Z")

</div>

> [@cnuernber](#):
>
> I could see the readout task needing to be global else the GC could clean it up but I do not understand why the read\_stdout needs to be global. It is closed over in pipe\_stream.

My guess is that this code was originally from IJulia which is the Julia kernel for Jupyter Notebooks.

> <https://github.com/JuliaLang/IJulia.jl/blob/9b10fa9b879574bbf720f5285029e07758e50a5e/src/init.jl#L28-L29>

One reason to expose a global is so that the variable is accessible. It’s like a public property of a class in Java.

The reason for the `const` `Ref` is actually a Julia idiom. globals can really slow things down in Julia because they mutate. A `const Ref` allows us to make sure that a global’s type stays consistent.

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 20, 2020, 5:31pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/32 "2020-12-20T17:31:02Z")

</div>

I think first I would like to figure out exactly what is going on with the task system. I can always read data out of the redirection pipe with readline or something like that. I tested this and it works.

But why the task never actually runs but reports that it did is something that I think is important. Perhaps it is a problem with my setup; not sure but I was hoping there was a clear answer or some global task error queue I could check.

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 20, 2020, 8:17pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/33 "2020-12-20T20:17:02Z")

</div>

@mkitti - I feel the task question deserves a new thread on discourse or on slack. What is the best way to elevate this question a bit? Or perhaps get a few more eyes on that specific question in this thread? I think it comes down to exactly how the @async macro is implemented as it feels to me like there is some interaction with ccall inside the macro. If the code is pure Julia code, everything works fine. As soon as I try to call back to clojure using a C fn ptr then async call fails.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 20, 2020, 8:38pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/34 "2020-12-20T20:38:36Z")

</div>

A new Discourse post would be useful in either the Embedding or Internals topic. We could then chat about it in Slack in the corresponding channels.

A clear title would be useful.

Is this behavior dependent on `JULIA_COPY_STACKS=1`?

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 20, 2020, 8:46pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/35 "2020-12-20T20:46:59Z")

</div>

Great question. I do not set that environment variable anywhere so it is left to be the default. I assume the ccall convention has to work but I can set the variable to zero and see.

---

<div class="post-metadata">

**Author:** ![mkitti](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mkitti/32/12459_2.png) [@mkitti](https://discourse.julialang.org/u/mkitti)\
**Post date:** [December 21, 2020, 9:28am UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/36 "2020-12-21T09:28:23Z")

</div>

This is a nice write-up about `Tasks` and `Threads` in Julia:

> **[Announcing composable multi-threaded parallelism in Julia](https://julialang.org/blog/2019/07/multithreading/)**
>
> Announcing composable multi-threaded parallelism in Julia | Software performance depends more and more on exploiting multiple processor cores....

The `ALWAYS_COPY_STACKS` variable eventually came to be controlled by the `JULIA_COPY_STACKS` environmental variable. What I have figured out is that the JVM does not like the the `Task` mode where where the stack is not copied.

Also if you are actually multithreading on the Julia side, then you might need to be very careful about which what JNIENV pointers you are using since you need a different one per thread. The problem is that Julia `Tasks` were purposely designed to abstract the relationship to `Threads`, so you don’t really know what `Thread` a `Task` might be running on.

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 21, 2020, 3:57pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/37 "2020-12-21T15:57:58Z")

</div>

Thanks, that is super helpful. Will have more test results soon.

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 21, 2020, 8:09pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/38 "2020-12-21T20:09:56Z")

</div>

JULIA\_COPY\_STACKS=1 allows tasks to at least run and callback into clojure. The IO redirection, however, is still failing.

---

<div class="post-metadata">

**Author:** ![cnuernber](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cnuernber/32/19882_2.png) [@cnuernber](https://discourse.julialang.org/u/cnuernber)\
**Post date:** [December 21, 2020, 8:27pm UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/39 "2020-12-21T20:27:46Z")

</div>

Hmm, perhaps the best way to do redirection is to call redirect\_xxx and then get the raw libuv handle from the pipe and install a read callback on it [using libuv](http://docs.libuv.org/en/v1.x/stream.html). This sidesteps the task system entirely.

---

<div class="post-metadata">

**Author:** ![jdwright007](https://avatars.discourse-cdn.com/v4/letter/j/eada6e/32.png) [@jdwright007](https://discourse.julialang.org/u/jdwright007)\
**Post date:** [September 15, 2021, 12:06am UTC](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052/40 "2021-09-15T00:06:12Z")

</div>

Trying to use your LibJulia library - I’m having trouble with first line of  
jna.clj as it utilizes julia\_clj.JLOptions

From leinegen repl I’m using -

(require '[libjulia-clj.julia :as julia])

What I’ve done -

1. Installed Julia 1.5.4, julia\_home is set to point to the install directory
2. Used git to download LibJulia project
3. Used from LibJulia directory

> lein repl

At user prompt used:

user\> (require '[libjulia-clj.julia :as julia])

I’m getting:

Syntax error (ClassNotFoundException) compiling at (libjulia\_clj\impl\jna.clj:1:1).  
julia\_clj.JLOptions

Note - I found that Julia 1.5.4 does have the Options.jl library. My path for  
julia\_home is pointing to Julia 1.5.4 install directory. I have not checked for  
later Julia install on my system. My julia\_home is set to point to Julia 1.5.4  
install directory.

[Previous page](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052.md?page=1)

[Next page](https://discourse.julialang.org/t/using-julia-from-the-jvm-clojure/51052.md?page=3)