1 * X' * X is not symmetric

Those are different BLAS routines. If not mistaken syrk (symmetric rank-k update) takes advantage of crossproducts while gemm (General Matrix Multiply) is general.