Why can't I use julia addprocs?(ECONNREFUSED)

GalloLuigi · December 2, 2021, 5:15pm

I’m writing a program that should connect to remote virtual machines and then do distributed execution.

I have this warning: “Warning: Identity file /home/luigal/.ssh/id_rsa not accessible: No such file or directory.”.(Even if the file exists and the path is correct)

The program crashes when I try to do :

worker=[(“myuser@ip:port”,1)]

addprocs(worker;sshflags=“-i /home/luigal/.ssh/id_rsa”,exename=“/usr/local/bin/julia”, tunnel=true,dir=“/home/luigal/HGEpidemics-main/”)

This is the project I am working on: GitHub - GalloLuigi/HGEpidemics: Ottimizzazione di HGEpidemics.jl, un framework che consente di effettuare simulazioni di diffusione epidemica basate ad agenti.

I need to do this work on: https://github.com/GalloLuigi/HGEpidemics-main/blob/main/src/experiments/spreading/spreading_experiment_d.jl

github.com

GalloLuigi/HGEpidemics/blob/main/src/experiments/spreading_parallel/spreading_experiment_d.jl

using Pkg
Pkg.activate(".")




include("D:\\Tesi\\Julia\\Seconda_Parte\\HGEpidemics-main\\src\\Parallel\\HGEpidemics_p.jl")
using .HGEpidemics_p
using Distributed



@everywhere using Distributed, Pkg
@everywhere Pkg.activate(".")

using NPZ

using CSV
using DataFrames
using Dates

This file has been truncated. show original

Someone can help me?

The console says:

lawless-m · December 3, 2021, 1:04pm

It looks like you are running

addprocs(worker;sshflags="-i /home/luigal/.ssh/id_rsa",exename="/usr/local/bin/julia", tunnel=true,dir="/home/luigal/HGEpidemics-main/")

On a Windows machine, and that machine cannot access “/home/luigal/.ssh/id_rsa” which is on the machine you are trying to log into

it should be something like C:\\Users\\Tesi\\.ssh\\id_rsa pointing to wherever it is on the Windows machine

GalloLuigi · December 3, 2021, 3:27pm

Hi, thanks for the clarification.

I have this message: “load pubkey “C:\Users\luigi/.ssh/id_rsa”: invalid format”.

I have my local machine (Windows) and some remote virtual machines (Linux).
Should I create the shh keys on the remote machines and then move them to the window machine( as sshflags input) ?

lawless-m · December 3, 2021, 3:29pm

in that case try C:\Users\luigi\.ssh\id_rsa.pub

lawless-m · December 3, 2021, 3:31pm

it matters not where you generate the keys, just so long as they are distributed

You’ll need id_rsa.pub to be included in /home/luigal/.ssh/authorized_keys

try to get it working in a standalone ssh client

ssh -i C:\Users\luigi.ssh\id_rsa.pub REMOTE_HOST

GalloLuigi · December 3, 2021, 4:33pm

I downloaded id_rsa and id_rsa.pub from virtual machines and put them in "C:\Users\luigi.ssh".
I have create a folder /home/luigal/.ssh/authorized_keys and then i copied id_rsa.pub into it.

Finally I tried to connect with ssh but this is the result:

lawless-m · December 4, 2021, 9:15am

https://unix.stackexchange.com/questions/340844/how-to-enable-diffie-hellman-group1-sha1-key-exchange-on-debian-8-0

GalloLuigi · December 4, 2021, 10:47am

I definitely think it’s a key format problem.
I named id_rsa/id_rsa.pub the pair of keys generated by windows (local machine)and id_rsa_l/id_rsa_l.pub the pair of keys generated by linux(remote machine).

When i try to use ssh on the terminal this happens:

The same format error happens when I use Julia code:

Warning: Identity file C:\Users\luigi.ssh\id_rsa.pub not accessible: No such file or directory.
load pubkey “C:\Users\luigi/.ssh/id_rsa”: invalid format

lawless-m · December 4, 2021, 12:12pm

Oh, I thought you had got over that. I gave you bad advice yesterday

it should be ssh -i c:\users\luigi.ssh\id_rsa (without the .pub)

In fact, you can probably skip all the options and just go with

ssh 193… or ssh luigal@193… - hiding the domain name wasn’t so successful

and in addprocs

addprocs(worker; exename=“/usr/local/bin/julia”, tunnel=true,dir=“/home/luigal/HGEpidemics-main/”)

GalloLuigi · December 4, 2021, 6:01pm

The program now does not give me any errors on the key but still the “ECONNREFUSED” error.

The ssh connection gives me this:

newerror

(The credentials are correct)

lawless-m · December 4, 2021, 8:13pm

You can output diagnostic messages in ssh by adding -v -vv & -vvv

ssh -vvv -oKex…(all the other stuff)

GalloLuigi · December 5, 2021, 9:17am

Here is the result:

C:\Users\luigi>ssh -vvv -oKexAlgorithms=+diffie-hellman-group1-sha1 193.205.161.5

OpenSSH_for_Windows_8.1p1, LibreSSL 3.0.2

debug1: Reading configuration data C:\Users\luigi/.ssh/config

debug3: Failed to open file:C:/ProgramData/ssh/ssh_config error:2

debug2: resolve_canonicalize: hostname 193.205.161.5 is address

debug2: ssh_connect_direct

debug1: Connecting to 193.205.161.5 [193.205.161.5] port 22.

debug1: Connection established.

debug3: Failed to open file:C:/Users/luigi/.ssh/id_rsa error:2

debug1: identity file C:\Users\luigi/.ssh/id_rsa type 0

debug3: Failed to open file:C:/Users/luigi/.ssh/id_rsa-cert error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_rsa-cert.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_rsa-cert type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_dsa error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_dsa.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_dsa type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_dsa-cert error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_dsa-cert.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_dsa-cert type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ecdsa error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ecdsa.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_ecdsa type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ecdsa-cert error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ecdsa-cert.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_ecdsa-cert type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ed25519 error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ed25519.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_ed25519 type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ed25519-cert error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_ed25519-cert.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_ed25519-cert type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_xmss error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_xmss.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_xmss type -1

debug3: Failed to open file:C:/Users/luigi/.ssh/id_xmss-cert error:2

debug3: Failed to open file:C:/Users/luigi/.ssh/id_xmss-cert.pub error:2

debug1: identity file C:\Users\luigi/.ssh/id_xmss-cert type -1

debug1: Local version string SSH-2.0-OpenSSH_for_Windows_8.1

debug1: Remote protocol version 2.0, remote software version dropbear_0.52

debug1: no match: dropbear_0.52

debug2: fd 3 setting O_NONBLOCK

debug1: Authenticating to 193.205.161.5:22 as ‘luigi’

debug3: hostkeys_foreach: reading file “C:\Users\luigi/.ssh/known_hosts”

debug3: record_hostkey: found key type RSA in file C:\Users\luigi/.ssh/known_hosts:4

debug3: load_hostkeys: loaded 1 keys from 193.205.161.5

debug3: Failed to open file:C:/Users/luigi/.ssh/known_hosts2 error:2

debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2

debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2

debug3: order_hostkeyalgs: prefer hostkeyalgs: rsa-sha2-512-cert-v01 @openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa

debug3: send packet: type 20

debug1: SSH2_MSG_KEXINIT sent

debug3: receive packet: type 20

debug1: SSH2_MSG_KEXINIT received

debug2: local client KEXINIT proposal

debug2: KEX algorithms: curve25519-sha256,curve25519-sha256 @libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c

debug2: host key algorithms: rsa-sha2-512-cert-v01 @openssh.com,rsa-sha2-256-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ssh-ed25519-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519

debug2: ciphers ctos: chacha20-poly1305 @openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

debug2: ciphers stoc: chacha20-poly1305 @openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com

debug2: MACs ctos: umac-64-etm @openssh.com,umac-128-etm @openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1

debug2: MACs stoc: umac-64-etm @openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1

debug2: compression ctos: none,zlib @openssh.com,zlib

debug2: compression stoc: none,zlib @openssh.com,zlib

debug2: languages ctos:

debug2: languages stoc:

debug2: first_kex_follows 0

debug2: reserved 0

debug2: peer server KEXINIT proposal

debug2: KEX algorithms: diffie-hellman-group1-sha1

debug2: host key algorithms: ssh-rsa,ssh-dss

debug2: ciphers ctos: aes128-ctr,3des-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes256-cbc,twofish256-cbc,twofish-cbc,twofish128-cbc,blowfish-cbc

debug2: ciphers stoc: aes128-ctr,3des-ctr,aes256-ctr,aes128-cbc,3des-cbc,aes256-cbc,twofish256-cbc,twofish-cbc,twofish128-cbc,blowfish-cbc

debug2: MACs ctos: hmac-sha1-96,hmac-sha1,hmac-md5

debug2: MACs stoc: hmac-sha1-96,hmac-sha1,hmac-md5

debug2: compression ctos: zlib,zlib @ openssh .com,none

debug2: compression stoc: zlib,zlib @openssh.com,none

debug2: languages ctos:

debug2: languages stoc:

debug2: first_kex_follows 0

debug2: reserved 0

debug1: kex: algorithm: diffie-hellman-group1-sha1

debug1: kex: host key algorithm: ssh-rsa

debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha1 compression: none

debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha1 compression: none

debug2: bits set: 530/1024

debug3: send packet: type 30

debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

debug3: receive packet: type 31

debug1: Server host key: ssh-rsa SHA256:y1/7G8kP+ao4JsOQuIPNOMJk2TEWk+Q/RpGgZwPzG88

debug3: hostkeys_foreach: reading file “C:\Users\luigi/.ssh/known_hosts”

debug3: record_hostkey: found key type RSA in file C:\Users\luigi/.ssh/known_hosts:4

debug3: load_hostkeys: loaded 1 keys from 193.205.161.5

debug3: Failed to open file:C:/Users/luigi/.ssh/known_hosts2 error:2

debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts error:2

debug3: Failed to open file:C:/ProgramData/ssh/ssh_known_hosts2 error:2

debug1: Host ‘193.205.161.5’ is known and matches the RSA host key.

debug1: Found key in C:\Users\luigi/.ssh/known_hosts:4

debug2: bits set: 516/1024

debug3: send packet: type 21

debug2: set_newkeys: mode 1

debug1: rekey out after 4294967296 blocks

debug1: SSH2_MSG_NEWKEYS sent

debug1: expecting SSH2_MSG_NEWKEYS

debug3: receive packet: type 21

debug1: SSH2_MSG_NEWKEYS received

debug2: set_newkeys: mode 0

debug1: rekey in after 4294967296 blocks

debug3: unable to connect to pipe \\.\pipe\openssh-ssh-agent, error: 2

debug1: pubkey_prepare: ssh_get_authentication_socket: No such file or directory

debug1: Will attempt key: C:\Users\luigi/.ssh/id_rsa RSA SHA256:FJyfIFwNR7GRNwp0jMR/8ekgEogxgLa79EdtIKnGRwQ

debug1: Will attempt key: C:\Users\luigi/.ssh/id_dsa

debug1: Will attempt key: C:\Users\luigi/.ssh/id_ecdsa

debug1: Will attempt key: C:\Users\luigi/.ssh/id_ed25519

debug1: Will attempt key: C:\Users\luigi/.ssh/id_xmss

debug2: pubkey_prepare: done

debug3: send packet: type 5

debug3: receive packet: type 6

debug2: service_accept: ssh-userauth

debug1: SSH2_MSG_SERVICE_ACCEPT received

debug3: send packet: type 50

debug3: receive packet: type 53

debug3: input_userauth_banner

Release: 06/14/11 (SVN revision: 17201M)

debug3: receive packet: type 51

debug1: Authentications that can continue: publickey,password

debug3: start over, passed a different list publickey,password

debug3: preferred publickey,keyboard-interactive,password

debug3: authmethod_lookup publickey

debug3: remaining preferred: keyboard-interactive,password

debug3: authmethod_is_enabled publickey

debug1: Next authentication method: publickey

debug1: Offering public key: C:\Users\luigi/.ssh/id_rsa RSA SHA256:FJyfIFwNR7GRNwp0jMR/8ekgEogxgLa79EdtIKnGRwQ

debug3: send packet: type 50

debug2: we sent a publickey packet, wait for reply

debug3: receive packet: type 51

debug1: Authentications that can continue: publickey,password

debug1: Trying private key: C:\Users\luigi/.ssh/id_dsa

debug3: no such identity: C:\Users\luigi/.ssh/id_dsa: No such file or directory

debug1: Trying private key: C:\Users\luigi/.ssh/id_ecdsa

debug3: no such identity: C:\Users\luigi/.ssh/id_ecdsa: No such file or directory

debug1: Trying private key: C:\Users\luigi/.ssh/id_ed25519

debug3: no such identity: C:\Users\luigi/.ssh/id_ed25519: No such file or directory

debug1: Trying private key: C:\Users\luigi/.ssh/id_xmss

debug3: no such identity: C:\Users\luigi/.ssh/id_xmss: No such file or directory

debug2: we did not send a packet, disable method

debug3: authmethod_lookup password

debug3: remaining preferred: ,password

debug3: authmethod_is_enabled password

debug1: Next authentication method: password

debug3: failed to open file:C:/dev/tty error:3

debug1: read_passphrase: can’t open /dev/tty: No such file or directory

Should I also send the -v -vv -vvv version?
(they look the same)

lawless-m · December 5, 2021, 3:07pm

Is it correct that you are trying to log into your DD-WRT ?

Or are you trying to connect to a machine on the other side of the firewall ?

If so, I suspect you don’t have NAT port forwarding enabled to forward it to the machine inside the firewall.

If you’re not familiar with that, here are some articles.

https://forum.dd-wrt.com/wiki/index.php/Port_Forwarding

GalloLuigi · December 5, 2021, 4:36pm

I am trying to connect to the machines on the other firewall side.

The situation is strange, I can connect with ssh to the machines via the visual studio extension but I cannot via julia programs or windows terminal.

This is my VSCode extension:

lawless-m · December 5, 2021, 4:57pm

Do you get a terminal window in VSCode

Do you control the machine (i.e. have root) on the ssh host?

It could be that the University only gives you scp/sftp access and not a terminal account.

GalloLuigi · December 5, 2021, 6:51pm

I have a non-root account.
I created the ssh keys with this account.
I can run julia programs, create files etc

lawless-m · December 5, 2021, 7:00pm

hmm, it is quite the mystery

I too have struggled with getting a Windows 10 machine to connect over the public internet with no answers

I have PMd you a message with a shell account you can try

GalloLuigi · December 5, 2021, 7:58pm

With your machine I can connect via terminal in ssh without problems.
The problem probably is my virtual machines.
In these days I will be doing further tests with Julia.
How can I upload my project from github?
Thank you very much for everything ,you have been very kind!

lawless-m · December 5, 2021, 8:37pm

Dec 5 20:35:26 pox sshd[2619]: Unable to negotiate with YOUR-IP port 59046: no matching key exchange method found. Their offer: diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1 [preauth]

No need for -o KEx…

lawless-m · December 5, 2021, 8:51pm

If your files are laid out in the correct way in Git you can add direct from Julia package prompt, like this one of mine

julia> ]
(@v1.7) pkg> add https://github.com/lawless-m/DFUtil.jl

Making packages is a lot easier than it sounds

https://pkgdocs.julialang.org/v1/creating-packages/

Or you can just copy the files into the file system and use “include”

julia> include("/path/to/julia/file.jl")

or tell julia where to look for module files

push!(LOAD_PATH,"/path/to/packages/")

which you can put in ~/.julia/config/startup.jl

along with anything else you might want at startup

such as some useful packages for develeopment using Revise and using OhMyREPL