Other than the StructArray trick, you could also try permute! to reduce allocations (or even Base.permute!!, as you don’t need to preserve the result of sortperm).
For example
sort_by!(f, v) = Base.permute!!(v, sortperm(f.(v)))
should reduce allocations. I’m not sure if it is slower or faster than simply v[sortperm(f.(v))] though.