Theres a lot of information that I can’t glean from your post but that could have performance implications.
How large are the SVectors? Beyond a certain size they perform less efficiently then Array.
Also how type stable is the code? If you put SVectors into containers or use them as feilds in structs it can be easy to end up in a type unstable situation, for example:
# This is an array type which's eltype is not concrete, and so requires dispatch when you try to do things with elements from it
Array{SVector{3,Float64}}
# This should be used instead
Array{SArray{Tuple{3}, Float64, 1, 3}}
# A different way to do the above which may be easier.
const Float64x3 = typeof(SVector(0,0,0))
Array{Float64x3}