To change the cache path run Base.LOAD_CACHE_PATH[1] = "/path/to/cache"
Note that by default the cache path is ~/.julia/lib/version. If you want to set it automatically add the command to either ~/.juliarc.jl (for just you) or julia_install_dir/etc/julia/juliarc.jl (for all users). The following command will set it to the default folder in the users home directory.
Base.LOAD_CACHE_PATH[1] = string(ENV["HOME"],"/.julia/lib/v",VERSION.major,".",VERSION.minor)