Transducers, paralelism and feedback loop

[Janis_Erdmanis]

@tkf

Having said that, it may be useful if you want to fuse pre- and post- processings with learning. It would be great if you can explore this area because I’ve never tried. But I don’t want to blindly encourage this direction as there is a chance that it may not be as fruitful as you would have hoped.

I just finalised the design of the TaskMaster.jl package which takes inputs and puts outputs in the channels. I thought that it would be trivial to couple them also with transducers, but it just halts. The relevant code:

using Distributed
addprocs(2)
using TaskMaster
using Transducers

@everywhere f(x) = x^2

master = ProcMaster(f)
learner = IgnorantLearner(1:10)
loop = Loop(master,learner)

@sync begin
    inch = Channel(Map(identity),1:10)
    outch = Channel(1)
    @async evaluate(loop,inch,outch)

    for o in outch
        @show o
    end
    ### Gets stuck at the first element
    collect(Map(identity),outch)
end

To see how I use evaluate(loop,inch,outch) see evaluate.jl file on the GitHub.

It would also be aesthetically pleasing if I could couple Loop with transducers with |>. So, for example, I could do:

collect(Map(identity) |> Loop(master,learner) |> Map(identity),1:10)

Also, it seems easily possible to initiate Master with transducer instead of a function with channels. Not sure how that would be useful. Maybe it would allow modelling electrical circuits.