How to bind modules between tasks?

General Usage
1

I am trying to communicate to the repl remotely, but I am having problems in binding the Main module used by the repl and the Main module used by the server and I don’t know how to fix it.

I prepared a couple of scripts to reproduce the issue.

Setup

Server side

Create a test_module.jl with the following content:

module MyModule

using Sockets

# -------------------------------
# Server configuration
# -------------------------------
const _HOST = ip"127.0.0.1"
const _PORT = 8765
const _server_running = Ref(true)

# -------------------------------
# Handlers
# -------------------------------
function get_repl_variables(conn, id, params)
  println("active_module server: ", Base.active_module())
  println("Main objectid server: ", objectid(Main))
  println(names(Main, all=true))
end

# -------------------------------
# Client handler
# -------------------------------
function handle_client(conn::TCPSocket, addr)
  println("Client connected from $addr")
  try
    msg = read(conn, 1)
    get_repl_variables(conn, nothing, nothing)
  catch e
    println("Client error: $e")
  finally
    close(conn)
  end
end

# -------------------------------
# Server loop
# -------------------------------
function start_server()
  server = listen(_HOST,_PORT)
  println("Julia TCP server running on $_HOST:$_PORT")

  while _server_running[]
    try
      conn = accept(server)
      handle_client(conn,getpeername(conn))
    catch e
      println("Accept error: $e")
    end
  end

  println("Server stopped")
end

# -------------------------------
# Run server in background
# -------------------------------
@async start_server()

# End of module
end

The module is quite easy: it create a server that is supposed to interact with the repl.
When a message is received, then the server calls the function get_repl_variables() that shall print some information about the Main module.

Client side

Next, we should send some message to the server and see what happens in the repl.
You can create a send_request.sh script with the following content (the script use zsh, but I think you can just change the shebang to use it with bash or similar).

#!/usr/bin/env zsh

HOST="127.0.0.1"
PORT=8765

# Send just a few bytes to trigger the server
echo -n "ping" | nc $HOST $PORT

Make the above script executable with chmod +x send_request.sh.

Basic test

At this point, run julia -i test_module.jl. This should start the repl and create the server.
Then, open another terminal windows and run ./send_request.sh.

In the repl, you should see the following:

active_module server: Main
Main objectid server: 8699491554302981441
[Symbol("##meta#60"), :Base, :Core, :Main, :MyModule, :eval, :include]

Now, from the repl, run println("active_module repl: ", Base.active_module()) followed by println("Main object id repl: ", objectid(Main)) and you should see the following

active_module repl: Main
Main object id repl: 8699491554302981441

Finally, from the repl run println(names(Main, all=true)) and you should see the following:

[Symbol("##meta#60"), :Base, :Core, :Main, :MyModule, :eval, :include]

So far so good.

Here comes the issue

Now, from the repl, type A = 10, followed by println(names(Main, all=true)). You should see the following:

[Symbol("##meta#60"), :A, :Base, :Core, :Main, :MyModule, :eval, :include]

As you see, A is in the Main scope.

Next, run ./send_request.sh. You get the following:

[Symbol("##meta#60"), :Base, :Core, :Main, :MyModule, :eval, :include]

BOOM! The variable A is not included.

It seems that either the subprocess has its own Main that does not update when the repl Main is updated - which is strange given that both the Main:s have the same object_id and in both cases is the active module - or that the Main used in the server side is somehow frozen or it reads from some cached values.

Interestingly enough, by running MyModule.get_repl_variables(1,2,3) from the repl, you get:

active_module server: Main
Main objectid server: 8699491554302981441
[Symbol("##meta#60"), :A, :Base, :Core, :Main, :MyModule, :eval, :include]

I have no idea why it does not work when you go through the server, and TBH I don’t know if this is a bug or not. Perhaps the repl namespace has some dedicated area that I fail to see and from where I should pick up info instead of using names(Main, all=true) from the server side?
Perhaps from the server I should query something like names(REPLSecretPlace, all=true) to get all the variables defined in the current interactive repl?

Or perhaps I should bind the repl Main with the server Main in some way? If so, I have no idea how to do that and any suggestion would be highly appreciated.

1 Like
2

Just to surface the obvious question: Why REPL for remote rather than Pluto? (I’m sure you have your reasons—just hoping to get it settled in any one else’s head.)

3

Well, for the sake of generality. :slight_smile:

I think you can potentially create any simple frontend to connect to a Julia backend through very simple protocols while enjoying an interactive experience at the same time.

You would not be constrained to use off-the-shelf Pluto, Jupyter, and so on, but you can create your own.

For the record, I created a frontend using Vim9 and the only missing part is accessing variables defined in the REPL space, as discussed in the thread.

But regardless of the specific usage, in a more general setting: not being able to access the REPL namespace from a subprocess is just… strange.

1 Like
4

FYI: This seems to work:

function get_repl_variables(conn, id, params)
  println(@eval Main names(Main, all=true))
end

Is should be because I am using @eval I am forcing runtime evaluation whereas using just names(Main, all=true) happens at compile time.

1 Like