I tried these two functions to get the l2 regularization term for a NN model:
function ℓ₂(a) s = zero(Float32) for i in a s += sum(i.^2) end s end
function l₂(a) s = zero(Float32) Lₐ = length(a) for i in 1:Lₐ @inbounds Lᵢ = length(a[i]) for j in 1:Lᵢ @inbounds s += a[i][j]^2 end end s end
But I get different result, for the first function I get 469.8826f0 and for the second I get 469.87442f0.
Why this happens?
Moreover the second function is much slower than the first one. Why?
What is the best way to compute the sum of params of a model in julia?
Sorry for my english.