My main machine is a MacBook Pro, where I edit Julia.
I also have a PC running Windows that has an 8-core CPU and a GTX-1070ti.
I can connect them via local LAN.
Is there some good way that I could send Julia that I edit on a notebook on the mac to be executed on the PC? I’m thinking of some ML tasks that could use the GPU.
Configure SSH and launch the notebook on your desktop machine. If you port-forward the notebook port correctly through the SSH tunnel you can open the notebook in a browser on your laptop. I have never tried this on Windows though, but I assume it should be doable.
Oh, I could have thought of that (but didn’t). I’ll just need to install an SSH server on the PC. I’m pretty sure the client version of Windows doesn’t have one.
After thinking some more, I probably should install some linux on the PC (dual boot). I’ve run open source software on windows some years ago, but it wasn’t a particularly nice experience. Basic tools run nicely, but then you want a compiler and specific libraries, etc. Cygwin is pretty ok, but still not that great. I see that CUDA is available for several distros, so that’s a good start.
If you use VSCode you can install the remote editting extension. This lets you use your IDE on your main computer but comfortably edit code with all the goodies of VSCode.
Take a look at how the most high-performance computing facilites handle this. Usually, they run on a Linux environment and access to the machine is achieved by a SSH, as @fredrikekre already mentioned. There are numerous providers for windows, mac and linux that are building the bridge to the remote computer. Submission of jobs is done by batch scripts, for know simplified commands are available (e.g. with so called slurm syntax).
Well, cloud storage would be an easy way to go. More old school would be an NFS shared directory.
Maybe I should use docker… I should be able to install docker desktop to the windows machine, and submit Julia+CUDA to it using the mac’s command line tools, and of course the mac would also have docker desktop for easy development. There’s already an Nvidia image for CUDA, just put Julia on top.
https://hub.docker.com/r/nvidia/cuda/
This way I wouldn’t need to install any linux on the PC, saving some work.
As a meta-thought, my original problem was that I mislabeled the problem, and now I’ve relabelled it a few times. (I just finished books by Daniel Kahneman and Chris Voss.)
You don’t need SSH, just configure the notebook server on your Windows PC properly and open the right port in the Windows firewall. This should be very easy, you don’t need to mess with certificates and router port forwarding as long as you are inside the LAN.
I am assuming you mean Jupyter, but the same idea works with Pluto: Running a notebook server — Jupyter Notebook 6.4.12 documentation
@jebej You’re right in a way, SSH is not needed for having access to the notebook. SSH is useful for remotely running things on the PC from the Mac.
Using SSH does not need certificates, just the old-fashioned public key based auth is enough. You can use certificates, apparently, if you want: https://medium.com/@berndbausch/ssh-certificates-a45bdcdfac39
Port forwarding to access the notebook interface would be useful in a datacenter setting if access to those ports has been firewalled away from outside computers like your laptop.
I was talking about the SSL certificates mentioned in the jupyter docs.