Accumulation in the for loop

My problem can be simply understand by the following case:
when I write the following program, I hope that c=a[1:5]+b[1:3]

a=[1 2 3 4 5]
b=[1 2 3]
c = 0
for i in 1:5, j in 1:3
    c += a[i] + b[j]  
end

but that is equal to

a = [1 2 3 4 5]
b = [1 2 3]
c = 0
for i in 1:5
    for j in 1:3
        c += a[i] + b[j]
    end
end

how can I write c=a[1:5]+b[1:3] in one formula with for loop instead of array or sum function, because
for my real program, sum or array is too slow compare with for loop

I think you want something along the lines of

for v in [a, b] # iterate among the various "containers"
    for i in eachindex(v) # iterate along the indices of each container
        c += v[i]
    end
end

That being said, if you have a list of arrays, e.g. vs = [rand(100) for _ in 1:100], you could use the sum(f, v) method (apply f to each entry and sum the results) and just do

sum(sum, vs) # add together the sum of each array

I think this should be pretty fast (and not allocate memory), so it’s probably good to benchmark to see if it really is the bottleneck in your program.

1 Like

What exactly are you expecting? Here you are summing a vector of length 5 with another vector of length 3, it is not clear to me what you want as a result.

My fault, I just wnat to add each element of array a and arry b, here the example is too simple, actually, I also need to make some complex calculation for each element of array a and array b and then sum them togher with for loop

julia> some_complex_calculation(x) = x + sin(x)
some_complex_calculation (generic function with 1 method)

julia> a = [1,2,3,4,5]; b = [6,7,8,9,10]; # same length (and comas, if these are column-vectors)

julia> c = some_complex_calculation.(a) .+ some_complex_calculation.(b) # note the dots
5-element Vector{Float64}:
  7.562055486608971
 10.566284025544471
 12.13047825468325
 12.655315989933829
 13.497054614447492

Yeah, seems work, I will try, if I have more questions, will ask you again, thanks

My initial plan is doing this, but this will take on a lot of memory allocation, so I plan to use for loop to expand the equations, and add them with for loop and it works better, but now, I have to add two formulas with different array length together(add each element of each formulas together), it’s easy to realize it with dot, but not easy with for loop since their length is not the same, that’s why I ask this question, it’s also hard to explain clearly my question, thanks anyway

There one would only be allocating the output array c, which could be also preallocated and overwritten with c .= . And that would be equivalent to the loop, roughly.

But anyway, I do not get exactly what you want, I’m happy others did. Good luck :slightly_smiling_face:

1 Like

Maybe this example is more close to my question

a=rand(5)
b=rand(5)
c=rand(5)
d=rand(3)
e=rand(3)
y=dot(a,b.*exp(c.+2))+dot(d,e)  # the initial formula which will cause a lot of memory allocation

I just want to realize the same purpose with for loop, but a,b,c, is longer than d,e array, and I cannot use for loop to combine them together, initially there is no dot(d,e), I can write it like

for  i in 1:length(a)
    y+=a[i]*b[i]*exp(c[i]+2)
end

but now with dot(d,e), how can I realize the same aim as this y=dot(a,b.*exp(c.+2))+dot(d,e) with for loop

Compute the dot(d,e) before the loop and add it as a scalar in the loop to each element.

Ps. The allocations in your code are from the b.*exp(c.+2) expression, which generates a new array. It is fine to use a loop to solve that, and one-liners also are possible.

You can just have y = dot(d,e) before the loop, or y += dot(d,e) after. You want to iterate over these two things one after the other, that’s what the code with two dots does. Your first post takes the product of the iteration spaces, you want the union.

This (edited to make capitals!) could be

mapreduce(+, A, B, C) do a,b,c
  a' * b * exp(c+2)
end

but unfortunately that’ll be no faster at the moment, as it just calls map which materialises the array before summing.

I cannot do that in real programs, since here just one equation, in reality, there are hundreds of equations

and both the first part and the second part will cause allocation, remember here is a simple example, in reality both parts are very long equations with temperory allocation, but without array calculation ( like .+, .-), it will be faster, and that’s why I want for loop

I think there are some misconceptions there about what allocates and what does not, but it is hard to tell without a more realistic example. Equations being long or short, being many or a few, do not really relate to the issues.

1 Like

Yeah, maybe I am wrong, I already learned a lot from your answer, I will try them later, thanks

Or maybe you can see this equation, this is part of my program. Let’s see dy[1], it includes three dot calculation, the first one is for 5 element array, the second and the third dot calculation is for 3 element array. My initial formula doesn’t have the second and the third dot, so I just write it like this

and it do decrease the allocation from ~900k to 41
Now with the second and third one, I want to do the same thing with for loop, but the array size is different

Or maybe I can use the for loop twice

I am stupid, I think using the for loop twice will be good enough, thanks for every reply, I learn a lot and julia is fantastic

Hi, if you have time, could you please help me see another question about distributed problems, here is the link

Distributed solving ODEs problem