# Why running remote ssh cmd in threads is still blocked?

**URL:** <https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262>\
**Category:** General Usage\
**Tags:** question, package\
**Created:** [June 13, 2023, 2:28am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262 "2023-06-13T02:28:07Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![peng](https://avatars.discourse-cdn.com/v4/letter/p/c2a13f/32.png) [@peng](https://discourse.julialang.org/u/peng)\
**Post date:** [June 13, 2023, 2:28am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/1 "2023-06-13T02:28:07Z")

</div>

Hi there, I want to run some commands on remote machine. I use julia to manage my commands. To avoid blocking in main threads, I run the commands in a spawned thread.For example:

```julia
cmd = """
sleep 3
echo Hello
"""
Threads.@spawn begin
    run(`ssh user@remoteip "$cmd"`)
end

```

However, it seems that the ssh still blocked the main thread. I can do nothing in the REPL. But if I run those commands locally, the main thread won’t be blocked:

```julia
Threads.@spawn begin
    run(`bash -c "$cmd")
end

```

In my julia REPL,`Threads.nthreads()` equals 16. And I’m sure the code above almost runs in different thread everytime. Is the `ssh`’s internet io communication blocked the main thread? Or have I missed any thing?  
I know, we can use Distributed’s functions. Could we just use Threads package to realize my needs? Thanks.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 13, 2023, 5:03am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/2 "2023-06-13T05:03:42Z")

</div>

How many threads did you start julia with? What’s the output of `versioninfo()`? Does connecting to your remote require entering a password or similar?

---

<div class="post-metadata">

**Author:** ![peng](https://avatars.discourse-cdn.com/v4/letter/p/c2a13f/32.png) [@peng](https://discourse.julialang.org/u/peng)\
**Post date:** [June 13, 2023, 5:06am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/3 "2023-06-13T05:06:03Z")

</div>

Hi Sukera,thanks for your reply. I started julia: `julia -t auto`. There are 16 threads. versioninfo is as follow:

```julia
versioninfo()
Julia Version 1.9.0
Commit 8e630552924 (2023-05-07 11:25 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 16 × Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-14.0.6 (ORCJIT, skylake)
  Threads: 16 on 16 virtual cores
Environment:
  LD_LIBRARY_PATH = /usr/local/nvidia/lib:/usr/local/nvidia/lib64
  JULIA_EDITOR = code
  JULIA_NUM_THREADS = 

```

I use key file for ssh authorization. No passwd needed.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 13, 2023, 5:56am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/4 "2023-06-13T05:56:59Z")

</div>

Then I can’t reproduce the issue - this works for me:

```julia
julia> cmd = """
       sleep 3
       echo Hello
       """

julia> Threads.@spawn run(`ssh user@host $cmd`); println("test")
test

julia> Hello

```

Note that the `"` are not needed - julia does escaping of interpolated values in commands automatically:

```julia
julia> `ssh user@host $cmd`
`ssh user@host 'sleep 3
echo Hello
'`

```

See also [this juliacon talk](https://www.youtube.com/watch?v=1XHSQ3925dc) for more information.

Does the connection without `Threads.@spawn` work, i.e. just this:

```julia
julia> run(`ssh user@host "$cmd"`)
Hello
Process(`ssh user@host 'sleep 3
echo Hello
'`, ProcessExited(0))

```

?

---

<div class="post-metadata">

**Author:** ![peng](https://avatars.discourse-cdn.com/v4/letter/p/c2a13f/32.png) [@peng](https://discourse.julialang.org/u/peng)\
**Post date:** [June 13, 2023, 6:09am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/5 "2023-06-13T06:09:50Z")

</div>

Thanks, I confirmed again. Your code works for me too. The `test` string outputs normally. But have you found that when you run the code, the REPL stuck? I think I must misunderstand the phenomenon, so, it’s not the main thread is blocked, it’s just the REPL blocked. Sorry for the inappropriate description.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 13, 2023, 6:32am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/6 "2023-06-13T06:32:48Z")

</div>

No, the REPL isn’t stuck - I can execute commands as usual. Is it actually blocked, i.e. entering things doesn’t show up? Can you run commands? Do you maybe need the `-n` flag to `ssh`, so that `ssh` doesn’t try to highjack your `stdin`? On my machine I have this:

```julia
-n Redirects stdin from /dev/null (actually, prevents reading from stdin). This must be used when ssh
        is run in the background. A common trick is to use this to run X11 programs on a remote machine.
        For example, ssh -n shadows.cs.hut.fi emacs & will start an emacs on shadows.cs.hut.fi, and the X11
        connection will be automatically forwarded over an encrypted channel. The ssh program will be put
        in the background. (This does not work if ssh needs to ask for a password or passphrase; see also
        the -f option.) Refer to the description of StdinNull in ssh_config(5) for details.

```

If you want to capture the output of that `ssh` command in julia, you can use `read(`ssh ...`)` by the way.

---

<div class="post-metadata">

**Author:** ![peng](https://avatars.discourse-cdn.com/v4/letter/p/c2a13f/32.png) [@peng](https://discourse.julialang.org/u/peng)\
**Post date:** [June 13, 2023, 6:49am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/7 "2023-06-13T06:49:00Z")

</div>

Oh god, it’s true. I tried with `-n` option, the REPL isn’t stuck anymore. Thanks a lot. You really solved my problem.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [June 13, 2023, 7:00am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/8 "2023-06-13T07:00:03Z")

</div>

You’re welcome!

I’d also recommend looking at `pipeline` as well as the `wait` keyword to `run`, to disentangle your parent process IO with the child process.

---

<div class="post-metadata">

**Author:** ![peng](https://avatars.discourse-cdn.com/v4/letter/p/c2a13f/32.png) [@peng](https://discourse.julialang.org/u/peng)\
**Post date:** [June 13, 2023, 7:04am UTC](https://discourse.julialang.org/t/why-running-remote-ssh-cmd-in-threads-is-still-blocked/100262/9 "2023-06-13T07:04:48Z")

</div>

Thanks. Yes, I’ve used pipeline for my ordinary commands execution and result retrival.