Probably it’s using a different number of threads?
You can call LinearAlgebra.BLAS.set_num_threads for OpenBLAS in Julia. I think with MKL you have to set the MKL_NUM_THREADS environment variable (before launching Julia)?
PS. If you are solving a least-square or similar non-invertible system, in most cases you should use pivoted QR rather than a pseudo-inverse, via A \ b … or equivalently by QR = qr(A, Val{true}()) followed by QR \ b (if you want to re-use the factorization for multiple right-hand sides). See also this post. (If you have an underdetermined solve and want the minimum-norm solution, you can use the LQ factorization instead.)
Switching languages will not help you here. Everyone is using the same dense linear-algebra libraries (LAPACK with some backend, typically either OpenBLAS or MKL). The only way to improve your performance if you are limited by LAPACK is by changing your algorithm (e.g. to use QR rather than SVD, or doing something even more clever).
Julia helps when you need to write your own performance-critical inner-loop code.