Understanding DimensionMismatch in list comprehension

New to Julia
1

Hello!

I am a fairly new Julia user and have been dabbling with the language on and off during the last year. I am currently working on my first bigger project and have run into issues with list comprehension.

Here’s a minimum working example that will produce the error:

using DSP

window_func(n) = sin.(pi/n*(0:n-1))

function fails()
    # just random data
    data = rand(Complex{Float64}, 46080)
    # I'm suspecting the return type Periodograms.Spectrograms to cause issues
    spec = Periodograms.spectrogram(data, 512, 384, fs=375, window=window_func)

    @show typeof(spec.power)

    cols = 1:2:162
    rows = [rand(1:10, 20), rand(1:10, 20)]

    @show typeof(cols)
    @show typeof(rows)

    # throws error
    [[spec.power[r, c] for (r, c) in zip(row, cols)] for row in rows]
end

function works()
    array = rand(1.0:100.0, 10, 10)

    @show typeof(array)

    cols = collect(1:10)
    rows = [rand(1:10, 10), rand(1:10, 10)]

    @show typeof(cols)
    @show typeof(rows)

    [[array[r, c] for (r, c) in zip(row, cols)] for row in rows]
end

Running it produces the following outputs:

julia> include("forum.jl")
works (generic function with 1 method)

julia> fails()
typeof(spec.power) = Array{Float64,2}
typeof(cols) = StepRange{Int64,Int64}
typeof(rows) = Array{Array{Int64,1},1}
ERROR: DimensionMismatch("dimensions must match")
Stacktrace:
 [1] collect(::Base.Generator{Base.Iterators.Zip{Tuple{Array{Int64,1},StepRange{Int64,Int64}}},getfield(Main, Symbol("##20#22")){DSP.Periodograms.Spectrogram{Float64,Frequencies}}}) at ./indices.jl:154
 [2] collect(::Base.Generator{Array{Array{Int64,1},1},getfield(Main, Symbol("##19#21")){DSP.Periodograms.Spectrogram{Float64,Frequencies},StepRange{Int64,Int64}}}) at ./none:0
 [3] fails() at /home/galvanix/Projects/julia/wspr/forum.jl:21
 [4] top-level scope at none:0

julia> works()
typeof(array) = Array{Float64,2}
typeof(cols) = Array{Int64,1}
typeof(rows) = Array{Array{Int64,1},1}
2-element Array{Array{Float64,1},1}:
 [61.0, 51.0, 76.0, 26.0, 83.0, 1.0, 25.0, 16.0, 10.0, 10.0]
 [49.0, 70.0, 64.0, 46.0, 31.0, 47.0, 2.0, 33.0, 83.0, 78.0]

I am at a loss here as to why one works and the other doesn’t. Looking at the types of the arrays involved everything seems to be identical?

Can anyone explain this behaviour?

1 Like
2

Your row entries are not the same length as cols in the first example, and you can’t zip together two collections of different length:

julia> collect(zip(rand(1:10, 20), 1:2:162))
ERROR: DimensionMismatch("dimensions must match")
Stacktrace:
 [1] promote_shape at ./indices.jl:154 [inlined]
 [2] _promote_shape at ./iterators.jl:317 [inlined]
 [3] axes at ./iterators.jl:316 [inlined]
 [4] _similar_for at ./array.jl:519 [inlined]
 [5] _collect at ./array.jl:550 [inlined]
 [6] collect(::Base.Iterators.Zip{Tuple{Array{Int64,1},StepRange{Int64,Int64}}}) at ./array.jl:544
 [7] top-level scope at none:0
3

Thank you for your quick reply. The documentation for the zip led me to believe that the iteration will stop as soon as the first collection is exhausted.

In fact the following produces the expected result:

julia> for i in zip(rand(1:10, 10),1:2:162)
       @show i
       end
i = (6, 1)
i = (1, 3)
i = (3, 5)
i = (3, 7)
i = (5, 9)
i = (10, 11)
i = (7, 13)
i = (9, 15)
i = (8, 17)
i = (3, 19)

However this does not:

julia> collect(zip(rand(1:10, 10),1:2:162))
ERROR: DimensionMismatch("dimensions must match")
Stacktrace:
 [1] collect(::Base.Iterators.Zip{Tuple{Array{Int64,1},StepRange{Int64,Int64}}}) at ./indices.jl:154
 [2] top-level scope at none:0

So it is possible to zip together collections of different lengths but you can’t run collect on the resulting iterator?

4

See this post for a discussion around this, and a workaround if you do need to collect zip:

function zip_collect(itr)
    itrsize = Base.IteratorSize(itr)
    itrsize isa Base.HasShape && (itrsize = Base.HasLength())
    Base._collect(1:1, itr, Base.IteratorEltype(itr), itrsize)
end

Which works like this:

julia> collect(zip(rand(1:10, 10), 1:2:162))
ERROR: DimensionMismatch("dimensions must match")

julia> zip_collect(zip(rand(1:10, 10), 1:2:162))
10-element Array{Tuple{Int64,Int64},1}:
 (10, 1)
 (3, 3)
 (2, 5)
 (6, 7)
 (1, 9)
 (6, 11)
 (2, 13)
 (3, 15)
 (6, 17)
 (3, 19)

Warning: The above code uses the internal Base._collect method which might not be a great idea in professional projects, since it could disappear or change behavior in a future version of Julia. It might be better to try to solve your problem in another way until the PR linked in the other post has been fixed.