At first glance, it seems your code performs a lot of unneeded allocations. For Julia to be fast, you want to avoid allocating new memory when you can reuse it instead.
See the performance tips for more details, especially the sections about
In many cases it will be as easy as replacing
for i in 1:n
x = y - z
end
with
x = # init
for i in 1:n
x .= y .- z
end
whenever you deal with arrays