This seems like an issue with your virus scanner to me – I’m very sure that the Juno installation can’t delete the Julia executable.
I can install it in a different path but then it does not work.
What doesn’t work then? You can adjust the path to the julia executable in Atom (Settings > Packages > julia-client > Julia Path).