I just stumbled over Running commands over ssh - #8 by jw3126, which had me thinking:
Could I use
Distributed to launch a Julia process on a Raspberry Pi, and then use
Distributed.remotecall to run arbitrary commands there, without incurring TTFX? And have the function return values directly available in a Julia process on the host computer? Specifically, I would want to run Python commands (current done via PyCall.jl) that make analog measurements, or set analog outputs. This is done using some Harware Attached on Top (HAT), for which Python bindings are provided.
So far, I have been using an SSH connection to access a shell on the Pi, from which I make measurements and save (on the Pi). I have then used FileZilla/
sftp to send the files. Where I want a live oscilloscope, I have limited myself to Sixel-based terminal plotting, as I suspected that running fancy web-applications and/or something like Makie on the Pi would be painfully slow. But if I can directly trigger measurements on the Pi and seamlessly send them to the host computer, via the SSH functionality of
Distributed.jl, I could immediately perform arbitrary analysis and visualization, without worrying about hardware requirements. That would be freaking awesome!
So I was wondering if anyone with any experience with Distributed/Raspberry Pi/SSH could evaluate if this was possible, or perhaps if there are simpler ways to do what I want. If this is in fact possible, I have never done anything like it, so some example code to create a remote process and run some function would be sweet - esentially a “hello world” of SSH-based distributed Julia.
If all goes well, perhaps some of this could be packaged up and put inside the
JuliaBerry org, as I do believe that many people using the Pi would love to have the IDE and hardware capabilites of their computers, with full access to running arbitrary code on the Pi and get the return values.
remotecall still potentially has to compile on the worker, since the CPU architecture might be different. I think the way it works is that the function definition (uncompiled) is sent to the worker via
@everywhere, which then compiles & calls the function locally. Sending compiled code to a remote worker is not supported, since julia doesn’t support cross compilation.
Raspberry PI are pretty powerful. I once lookedat the specs and they heavilyoutpower the $20000 Sun workstation I used to use.
Have you looed at remoterepl
For the file transfer you should set up a shared filesystem between your Windows laptop/workstation and the Pi.
RemoteREPL.jl seems like a more direct way to do this than Distributed.jl, but essentially the same for my proposes - thanks!! It seems focused on REPL usage, but the docs state that running
connect_remote() should " allow you to use
@remote without the REPL mode". I just may believe that RemoteREPL is perfect for what I want, and that it should probably be advertised in JuliaBerry. If I get a chance to test things between/after exams, I will report back here and possibly add references on JuliaBerry.
I am having some problems with making RemoteREPL.jl work, due to an error about the premissions of
If there was something like it for Python, that would also work. In fact, it would alleviate the need to a) install julia on the Pi, and b) have the same Julia version on the client side (due to compatibility of Serialization.jl between Julia versions).
Does anyone know of tools to run arbitrary python code remotely, via SSH, where the return value can be accessed directly in a python process on the client?
What are the errors regarding permissions ?
We are currently debugging it over at [this issue](https://github.com/c42f/RemoteREPL.jl/issues/47). But internally, RemoteREPL runs the command
ssh_cmd = `/home/username/.julia/artifacts/cf570328554ecb898e625367a9c9fb2c01f55267/bin/ssh -o ExitOnForwardFailure=yes -o ServerAliveInterval=60 -N -L 127.0.0.1:46163:localhost:27754 xyz`
ssh binary can not be run due to permissions problems with
/home/username/.julia/artifacts/cf570328554ecb898e625367a9c9fb2c01f55267/bin/ssh: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory
Edit: The problem I described was not it at all. I had to remove write premissions from “others” and “groups” to “~/.ssh/config”, and it worked - the permissions problem is fixed.
I am now running into another error, namely “cannot pickle ‘module’ object”. This occurs when I use RemoteREPL to call
pyimport from PyCall.jl. I opened an issue here, but I suspect that this is an issue somewhere between RemoteREPL, PyCall and the python module “piplates.DAQC2plate”, and I am not sure anyone will be able to pinpoint what goes wrong and why. Help would be much appreciated.
Perhaps this would not be a problem if the python module was imported into a remote python proces, reviving my search for a way to do what RemoteREPL.jl does, just from python to python.