Yeah, this is only per process (like any regular variable). Restarting julia will reset LOAD_PATH.
But a more direct way is:
LOAD_PATH_BACKUP = copy(LOAD_PATH)
# modify and use modified LOAD_PATH here
# now reset
empty!(LOAD_PATH)
for x in LOAD_PATH_BACKUP
push!(LOAD_PATH, x)
end