Like @liuyxpp, I also recommend the “Remote - SSH” extension of VS Code. I started using it recently, and I am satisfied with it.
In my case, though, I cannot use the REPL built in VS Code when using Remote - SSH, because it connects to a login node of the Linux cluster. Starting the built-in REPL starts the REPL on the connected login node, which is not for heavy computation. For computation, I request an access to a compute node. This is typically done by obtaining an interactive shell of a compute node from the shell of the login node. Then, I start REPL from the obtained interactive shell.
This is fine for usual computation, but visualization is a challenge, because I cannot draw plots like I do in the REPL built in VS Code. I tried to do X11 forwarding, but it failed because compute nodes usually don’t have GPU for visualization. (I use GLMakie. Maybe other plotting backends work.)
My current solution is to use the Remote - SSH to access files and develop code using the VS Code editor, but to run REPL on a terminal with SIXEL support. Then, I can plot directly in REPL using SixelTerm.jl; see more details here. There is an effort to support SIXEL in VS Code’s built-in terminal, which will eliminate the need to run a separate terminal program.