I must be missing why using the official binaries is not an option then. You don’t have to compile anything.
Also, cross-compiling Julia isn’t really feasible at the moment: you can actually cross-compile the executable, but to build the sysimage (which is by far the bottleneck of the process) you have to run it.