Help fo working with Iters

for this I got:

DimensionMismatch("arrays could not be broadcast to a common size")

Just think about it for a sec. Eg imagine that series has 10 elements and slen is 2.

for this also,I got the same:

y = map((a, b) -> (b - a) / 2, series[1:12], series[12+1:end])
DimensionMismatch("dimensions must match")

Of course, it’s the same problem.

Maybe:

function initial_trend(series, slen)
   sum = 0.0
   for i in 1:slen
       sum += series[i+slen]-series[i]
   end
   return sum/slen
end

I worked from the python and removed what I suspect is a redundant /slen (it is being done both in the loop and outside the loop).

P.S. I’m assuming that series is a vector of floats.

Thanks, actually the other slen is required, not extra

     sum += (series[i+slen]-series[i]) / slen

Your code with returning the slen gave me the correct output, so now how can I covert this Julia code to functional format using Iter

So, how can we enforce it to work based on the shortest array, so for 10 and 2, it will take only the first 2 elements from the array of 10

zip does that. did you read my very first reply? nope it does not work that way. I would just calculate indexes to match in length, like @dmolina suggests, and maybe use @view.

In that case, I would suggest:

y = map((a, b) -> (b - a) / 2, series[1:slen-1], series[slen+1:2*slen])

Be careful with indexes, julia series[1:n] includes n index, while Python does not.

Edit: Sorry, it was missing a parenthesis.

3 Likes

I got:

DimensionMismatch("dimensions must match")

Iter b is bigger than iter a so it fails, how can I ask it to take the first x elements in iter b where x is the length(a)

My code was working inspired in Python, which does not explore to the end, so until 2*slen. Please, do not submit versions if other language if it is not right.

If you want to avoid problem you should use zip as @Tamas_Papp suggested:

function initial_trend(series, slen)
   sum = 0.0

   for (a, b) in zip(series[1:slen], series[slen+1:end])
        sum += (b-a)/slen
   end
   return sum/slen
end

This code should give the same than Rust version.

The zip(series[1:slen], series[slen+1:end]) is fine, and your code gave correct result,
Can I replace:

for (a,b) in .. 
     sum+=(b-a)/slen
end

with map() function?

I do not think so, zips stop when one of them finish before the other one, but for some reason this check is only done in a for loop, not inside a map or the collect function.

1 Like

Two comments then:

  1. If you need to divide by slen twice, shouldn’t that still happen outside the loop? (i.e. return sum/slen^2 gives me the correct answer).
  2. I don’t understand what you are trying to do or need. I frankly get lost with all the zips and maps when the simple for loop is perfectly readable and produces the correct answer.

Thanks to all the inputs and feedbacks given, I solved it as:

initial_trend(series, slen) = sum(
    map(i -> (last(i) - first(i)) / slen, 
        zip(series[1:slen], series[slen+1:2*slen])
        )
    ) / slen

I was having 2 issues:

  1. As @dmolina mentioed zipping is not stopping in mapping when one of the iters finish before the other one

  2. Tuple in mapping is not handled as map((a,b) -> ... , x), it is handled as map(i -> ... , x), then can be handled as first(i) and so on.

You are correct, but I’m trying to keep the coding formula matching with the mathematical formulas

I would like to do it the functional programming way. thanks a lot for your feedbacks

I used @btime from BenchmarkTools to compare the loop version to the sum/map/zip version. I’m not sure if you care about speed or if this is an academic exercise but the simple loop runs 5x faster with less memory usage.

julia> @btime initial_trend(series, 12)
  131.302 ns (7 allocations: 624 bytes)
-0.7847222222222222

julia> @btime initial_trend_for_loop(series, 12)
  24.727 ns (1 allocation: 16 bytes)
-0.7847222222222222
2 Likes

If you want a “functional” approach, you can also do

initial_trend(series, slen) =
    mapreduce(+, zip(series[1:slen], series[slen+1:2*slen])) do (a, b)
        (b - a) / slen
    end / slen

If you care performance a bit then,

function initial_trend(series, slen)
    x = @view series[1:slen]
    y = @view series[slen+1:2*slen]
    n = min(length(x), length(y))
    mapreduce(+, (@view x[1:n]), (@view y[1:n])) do a, b
        (b - a) / slen
    end / slen
end

The latter in principle should be almost as fast as the raw loop and has better accuracy.

1 Like

Those are both slower than the for loop by quite a bit, and the second one is much slower than the first.

julia> @btime initial_trend_for_loop(series, 12)
  24.727 ns (1 allocation: 16 bytes)
-0.7847222222222222

julia> @btime initial_trend_tkf1(series, 12)
  96.970 ns (5 allocations: 416 bytes)
-0.7847222222222222

julia> @btime initial_trend_tkf2(series, 12)
  778.307 ns (13 allocations: 480 bytes)
-0.7847222222222222
1 Like

Ah, so it seems that mapreduce for multiple arrays is not as optimized as I hoped:

https://github.com/JuliaLang/julia/blob/a68237f9c9805d492fc2d483bf63b8ecba647e8f/base/reducedim.jl#L307-L308

Fusing it would need something like https://github.com/JuliaLang/julia/pull/31020