Contrary to some of the languages traditionally used in research computing environments (I’m thinking of C/C++ and Fortran), Julia actually comes with a decent package manager, so using versions of packages different from those provided by the environment is even more common.
I don’t think you want to touch JULIA_LOAD_PATH at all, but only the depot. For that,
export JULIA_DEPOT_PATH="${JULIA_DEPOT_PATH}:/foo/bar"
should achieve the desired effect.