What is Julia's im2col?

General Usage
4

Yes, that looks like roughly the same thing, except theirs doesn’t support different boundary conditions or asymmetric windows. Also, ours is fast either for passing a multidimensional array (as in nlfilter) or a column (as in colfilt).

Also, note that “easier vectorization” isn’t particularly relevant for Julia, it’s only important for slow languages. But I agree the cache-efficiency might be important in some applications, so I can see why you’d want this.

Fortunately, it comes for free via composition (note: example on julia 0.6, there seems to be some issue with 1.0, I’ll look into it):

julia> using ImageFiltering, StaticArrays

julia> A = collect(reshape(linspace(0, 1, 15), 3, 5))
3×5 Array{Float64,2}:
 0.0        0.214286  0.428571  0.642857  0.857143
 0.0714286  0.285714  0.5       0.714286  0.928571
 0.142857   0.357143  0.571429  0.785714  1.0     

julia> out = similar(A, SVector{4,eltype(A)})
3×5 Array{StaticArrays.SArray{Tuple{4},Float64,1,4},2}:
 [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]  [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]  …  [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]  [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]
 [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]  [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]     [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]  [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]
 [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]  [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]     [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]  [6.91108e-310, 6.91108e-310, 6.91108e-310, 6.91108e-310]

julia> mapwindow!(identity, out, A, (0:1, 0:1))
3×5 Array{StaticArrays.SArray{Tuple{4},Float64,1,4},2}:
 [0.0, 0.0714286, 0.214286, 0.285714]       [0.214286, 0.285714, 0.428571, 0.5]       [0.428571, 0.5, 0.642857, 0.714286]       [0.642857, 0.714286, 0.857143, 0.928571]  [0.857143, 0.928571, 0.857143, 0.928571]
 [0.0714286, 0.142857, 0.285714, 0.357143]  [0.285714, 0.357143, 0.5, 0.571429]       [0.5, 0.571429, 0.714286, 0.785714]       [0.714286, 0.785714, 0.928571, 1.0]       [0.928571, 1.0, 0.928571, 1.0]          
 [0.142857, 0.142857, 0.357143, 0.357143]   [0.357143, 0.357143, 0.571429, 0.571429]  [0.571429, 0.571429, 0.785714, 0.785714]  [0.785714, 0.785714, 1.0, 1.0]            [1.0, 1.0, 1.0, 1.0]

If you like the plain-old im2col output it’s just

julia> i2c = reinterpret(Float64, out, (4, 15))
4×15 Array{Float64,2}:
 0.0        0.0714286  0.142857  0.214286  0.285714  0.357143  0.428571  0.5       0.571429  0.642857  0.714286  0.785714  0.857143  0.928571  1.0
 0.0714286  0.142857   0.142857  0.285714  0.357143  0.357143  0.5       0.571429  0.571429  0.714286  0.785714  0.785714  0.928571  1.0       1.0
 0.214286   0.285714   0.357143  0.428571  0.5       0.571429  0.642857  0.714286  0.785714  0.857143  0.928571  1.0       0.857143  0.928571  1.0
 0.285714   0.357143   0.357143  0.5       0.571429  0.571429  0.714286  0.785714  0.785714  0.928571  1.0       1.0       0.928571  1.0       1.0

Note we get one column per input pixel. If you want something like Matlab’s version which only returns columns corresponding to interior windows, use Inner() for the border:

julia> out = similar(A, SVector{4,eltype(A)}, (2,4))
2×4 Array{StaticArrays.SArray{Tuple{4},Float64,1,4},2}:
 [0.0, 0.0, 0.0, 0.0]  [0.0, 0.0, 0.0, 0.0]  [0.0, 0.0, 0.0, 0.0]  [0.0, 0.0, 0.0, 0.0]
 [0.0, 0.0, 0.0, 0.0]  [0.0, 0.0, 0.0, 0.0]  [0.0, 0.0, 0.0, 0.0]  [0.0, 0.0, 0.0, 0.0]

julia> mapwindow!(identity, out, A, (0:1, 0:1), Inner())
2×4 Array{StaticArrays.SArray{Tuple{4},Float64,1,4},2}:
 [0.0, 0.0714286, 0.214286, 0.285714]       [0.214286, 0.285714, 0.428571, 0.5]  [0.428571, 0.5, 0.642857, 0.714286]  [0.642857, 0.714286, 0.857143, 0.928571]
 [0.0714286, 0.142857, 0.285714, 0.357143]  [0.285714, 0.357143, 0.5, 0.571429]  [0.5, 0.571429, 0.714286, 0.785714]  [0.714286, 0.785714, 0.928571, 1.0]     

julia> i2c = reinterpret(Float64, out, (4, 8))
4×8 Array{Float64,2}:
 0.0        0.0714286  0.214286  0.285714  0.428571  0.5       0.642857  0.714286
 0.0714286  0.142857   0.285714  0.357143  0.5       0.571429  0.714286  0.785714
 0.214286   0.285714   0.428571  0.5       0.642857  0.714286  0.857143  0.928571
 0.285714   0.357143   0.5       0.571429  0.714286  0.785714  0.928571  1.0

If this is something you do a lot, then of course you should wrap it in a function.

As is hopefully clear, the Julia approach has advantages in terms of flexibility:

  • you can have as many output “windows” as there are pixels in the original image (or not, it’s up to you). This can make alignment of the input and the result of some computation considerably easier.
  • you can have the output keep each column as an inner vector (or matrix, if you used SMatrix{2,2} instead of SVector{4}) and preserve the shape of the original image, or you can “flatten” into a matrix representation.
  • mapwindow is also more general than Matlab’s sliding vs distinct by allowing you to specify the output indices you want (see the imginds argument)
3 Likes