I defined a reducible for my Kernel type, which works fine, and I can materialize pipelines with collect, etc.
I now want to be able to write a function that does kernel |> Map(func) |> # ... materialize into existing vector storage.
Basically I want to do what Base.map! does to eagerly construct a collection from a transducer pipeline, but storing the result in a destination collection without an intermediary allocation. I see there is a map! in Transducers.jl, but it seems to want 3 arguments, and being new to Transducers.jl, I really have no idea how to work with that, or if it’s even what I’m looking for.
Is your Kernel <: AbstractArray? If so, map!(Map(func), dest, kernel) should work fine, no? Otherwise, sizehint!ing an empty dest array and using append!(Map(func), dest, kernel) or append!!(dest, kernel |> Map(func)) should work, I think.
The latter does work, and I was able to do this on my own previously, but obviously, instead of re-using existing storage, it grows the vector. Sure I could sizehint! it to prevent allocation, but the goal is to just do a memory copy and retain the length of the destination vector, or at least start writing from the beginning rather than the end.
Now that I read that again: Shouldn’t it just be map!(func, dest, kernel)?
Side note: I have never used Transducers before, so it’s unlikely anything useful is going to come out of me blindly suggesting things without access to the original codebase
No that won’t do, as a transducer is not a function. I finally figured something out though that is type-stable, doesn’t allocate, and is extremely fast (thanks to @jakobnissen for the starting point):
function Base.map!(f::F, dest, kernel::Kernel) where {F}
foldl(kernel |> Map(f) |> Enumerate(); init=dest) do v, (i, e)
i > length(dest) && resize!(dest, i)
v[i] = e
v
end
end
If anyone sees any room for improvement, I’d like to know, being relatively new to Julia, and extremely new to Transducers.jl myself. Thank you!
