`eval` scope problem and `println` problem for multithreading

Update 1:

Thanks for all the suggesions. I have modifed the code as follows:

using Sockets

# server
server = listen(2000)
errormonitor(
  @async begin
    sock = accept(server)
    ex = quote my() = println("aa") end
    errormonitor(@async invokelatest(Core.eval, Main, ex))
    println("bb")
    errormonitor(@async invokelatest(Core.eval, Main, Meta.parse("my()")))
    sleep(1)
  end
)

# client
socket = connect(2000)

If I run the code from a file julia blah.jl , then “aa” is not printed out.

However, if I run the code from the REPL julia> include("blah.jl") , then “aa” is printed. Can anyone help and explain what is wrong when running julia blah.jl?

original post:

Hello!

I am exploring eval and I encounter a problem. Here is my code:

using Sockets

# server
server = listen(2000)
@async begin
  sock = accept(server)
  ex = quote my() = println("aa") end
  Core.eval(Main, ex)
  println("bb")
  my()
end

# client
socket = connect(2000)

The code is supposed to execute the function my(), but it only prints “bb” and “aa” is not printed out.

If I enter the code in the REPL as follows, however, the function my() works:

ex = quote my() = println("aa") end
Core.eval(Main, ex)
println("bb")
my()

Can anyone tell me what is wrong with the first part of code when using eval with socket?

Thank you very much!

1 Like

If I recall correctly, @async creates an anonymous function that is then run on the asynchronous task. This means that the function doesn’t see the newly defined function from your eval because of Julia’s world age mechanism.

Basically, functions don’t see the result of an eval until the return back to global scope, which the block in your @async doesn’t do.

1 Like

Thaks for your reply. However, my() does not produce any error message, which indicates that there is a newly defined my() function in scope. Anyway, to address the issue of global scope, I change the code as follows:

using Sockets

# server
server = listen(2000)
@async begin
  sock = accept(server)
  ex = quote my() = println("aa") end
  Core.eval(Main, ex)
  println("bb")
  Core.eval(Main, Meta.parse("my()"))
end

# client
socket = connect(2000)

This way, I think there is no global scope concern any more. my() function is called in the global scope by eval. Still, my() is not executed.

1 Like

You don’t get an error because the Task spawned by @async captured it:

julia> t = @async begin
         sock = accept(server)
         ex = quote my() = println("aa") end
         Core.eval(Main, ex)
         println("bb")
         my()
       end

       # client
Task (runnable, started) @0x00007f960a3b5780

julia> socket = connect(2000)
bb
TCPSocket(RawFD(21) open, 0 bytes waiting)

julia> t
Task (failed) @0x00007f960a3b5780
MethodError: no method matching my()
The applicable method may be too new: running in world age 25655, while current world is 25656.

Closest candidates are:
  my() (method too new to be called from this world context.)
   @ Main REPL[3]:3

Stacktrace:
 [1] (::var"#1#2")()
   @ Main ./REPL[3]:6

With your modification, the message is printed for me. How are you running the example (from the REPL, from a file…)? Are you starting Julia with more than one thread, so that the async task has the opportunity to be scheduled?

3 Likes

You don’t get an error because the Task spawned by @async captured it

Thanks. As you pointed out, I can see the error message now. To show error messages, I have added errormonitor for @async and changed the code as follows:

using Sockets

# server
server = listen(2000)
errormonitor(
  @async begin
    sock = accept(server)
    ex = quote my() = println("aa") end
    Core.eval(Main, ex)
    println("bb")
    Core.eval(Main, Meta.parse("my()"))
  end
)

# client
socket = connect(2000)

If I run the above code from a .jl file, then only “bb” is printed out and there are no error messages. If I run the above code from the REPL, then both “bb” and “aa” are printed out. I started REPL using julia command, and the number of thread is

julia> Threads.nthreads()
1

I do not understand why results are different when using a file and the REPL, respectively. Can you help me figure this out? Thanks!

1 Like

Odd, there is a hard vs soft scoping rule change, but that only involves some local scopes reassigning names shared with the global scope, which you don’t do. Couple questions for clarification:

  1. when you say “run from a jl file” do you mean opening the REPL and running include("blah.jl")?
  2. Did you close and reopen the REPL before each time you ran the code?

I don’t have a definitive answer, but I guess it has to do with scheduling & yielding. Can you try running julia with more than one thread (e.g. julia -t 2) and see if that fixes things?

You are probably looking for invokelatest, e g. replacing my() with invokelatest(my). Without invokelatest only methods that live in the world age of the containing functions are accessible, so not ones that are newly defined using eval. This is also why this happened to work when omitting the @async

1 Like

Thanks for your reply.

  1. when you say “run from a jl file” do you mean opening the REPL and running include("blah.jl")?

I run the file with the command julia blah.jl and it does not work as expected, i.e. “aa” is not printed. If I open the REPL and run with include("blah.jl"), then it works and “aa” is printed out.

  1. Did you close and reopen the REPL before each time you ran the code?

Yes. I did close and repoen the REPL each time I ran the code.

1 Like

I looked into yield before, but I haven’t figured out what is wrong. Running the file with julia -t 2 blah.jl does not fix the problem.

Thanks for your reply. I have used invokelatest and changed the code as follows:

using Sockets

# server
server = listen(2000)
errormonitor(
  @async begin
    sock = accept(server)
    ex = quote my() = println("aa") end
    errormonitor(@async invokelatest(Core.eval, Main, ex))
    println("bb")
    errormonitor(@async invokelatest(Core.eval, Main, Meta.parse("my()")))
    sleep(1)
  end
)

# client
socket = connect(2000)

As pointed out in other posts, if I run the code from a file julia blah.jl, then “aa” is not printed out.
However, if I run the code from the REPL julia> include("blah.jl"), then “aa” is printed. There is something wrong when running from a file.

1 Like

As pointed out by @Benny post #13, it’s just the script’s primary task ending and closing the process when the @async task yielded control midway. Therefore, we can use wait or @sync begin to avoid the problem.

Therefore, I modified the code as follows by adding wait:

using Sockets

# server
server = listen(2000)
t =
  @async begin
    sock = accept(server)
    ex = quote
      function my()
        println("aa")
      end
    end
    errormonitor(@async invokelatest(Core.eval, Main, ex))
    println("bb")
    errormonitor(@async invokelatest(Core.eval, Main, Meta.parse("my()")))
  end


# client
socket = connect(2000)

wait(t)

By using wait, I get the same result for both julia blah.jl and julia> include("blah.jl"), i.e., “bb” and “aa” are both printed out.

2 Likes

I don’t think it’s println’s nested locks causing “aa” not to be printed at all; locking would affect task printing order. It’s just your script’s primary task ending and closing the process when your @async task yielded control midway. That’s just something you always need to account for when you run a script from the command line. I think enclosing the tasks in @sync begin ... end would work as well as wait, and it’s more convenient to do once over multiple tasks.

2 Likes

Thanks for your reply. Yes, your comment makes sense.