Julia wrapper for libssh?

Specific Domains Data
1

Is there a julia wrapper for libssh somewhere? I would like to be able to transfer a part of a large file located on a remote server using sftp (the file itself could be 100G, but I might be interested in a small subset of the data contained in that file). I realise this could be solved by splitting up the data into smaller files, but I have quite a number of these large files generated by a different process and I’d like to avoid having to split them up.

It would be really cool if something like this worked:

using SSH
data = open(SSH.new_session("username","password")) do session
   data = SSH.sftp_open(session, "remotefile") do ff
    SSH.sftp.seek(session, ff, pos)
    data = SSH.sftp.read(session, ff, bytes)
 end
end
2

I started working on RemoteFiles that will hopefully end up solving my problem.

3

No idea whether anyone wrote a wrapper around libssh already, but note that we have OpenSSH_jll which provide OpenSSH on all platforms. Note also a package called RemoteFiles.jl already exists.

4

Thanks for pointing out the name clash, I will rename my package then. Also, thanks for pointing out OpenSSH_jll, I’ll definitely have a look at that.

5

Just to post a tiny update to this, I renamed my repo SecureRemoteFiles to avoid the name clash. Adding the Secure- prefix seemed appropriate because the package is built on top of sftp. As such, it requires that you have ssh access to the sever hosting the remote file you are interested in.
You can use standard file handling methods, such as open and read, which will then be forwarded over sftp. The following is a short (silly) example

using SecureRemoteFiles

# create a file to read from
open("/tmp/testfile.txt","w") do f
   write(f, "This is a test")
end

# read the file contents back over sftp
data = open(sftp"localhost:/tmp/testfile.txt",0) do ff
    read(ff, 14)
end
txt = unsafe_string(pointer(data), length(data))

It is still very much a work in progress, but I’ve successfully incorporated this package into a visualisation tool that allows me to look at snapshots of data from electrophysiology experiments without having to download the whole file to my laptop. Once I’ve polished that up a bit, I hope to be able to put it out there for anyone interested in taking a look.

4 Likes