# Some questions on Transducers.jl

**URL:** <https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622>\
**Category:** General Usage\
**Created:** [February 19, 2021, 3:00pm UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622 "2021-02-19T15:00:24Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [February 19, 2021, 3:00pm UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622/1 "2021-02-19T15:00:24Z")

</div>

I’m recently interested in Transducers.jl.  
However, I cannot understand some methods provided in the package.

For example, anybody can explain `foldxl` [example](https://juliafolds.github.io/Transducers.jl/dev/reference/manual/#Transducers.foldxl) in detail? I cannot figure out how the answer comes.

Also, is there any way to incorporate multiple inputs?  
For example, I’d like to use a function `f(x, y, z) = x+y+z` like `xs |> Scan(f, y0, z0) |> collect` to compute functions recursively where `xs = (x1, x2, ..., xn)`.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 19, 2021, 5:46pm UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622/2 "2021-02-19T17:46:05Z")

</div>

> [@iHany](#):
>
> I’d like to use a function `f(x, y, z) = x+y+z` like `xs |> Scan(f, y0, z0) |> collect`

[`Transducers.Scan`](https://juliafolds.github.io/Transducers.jl/dev/reference/manual/#Transducers.Scan) expects a 2-arg function as the first argument \[1\]. So, I’m not sure how you want to use `f`. Maybe it’s helpful to write down what you want to do in plain code (ok to be non-efficient or lengthy), so that people can know exactly what you want to compute.

* * *

1. Transducers.jl APIs typically expect 2-arg (or sometimes 1-arg) function.

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [February 19, 2021, 11:51pm UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622/3 "2021-02-19T23:51:19Z")

</div>

Oh, you’re right. I should’ve attached an example.

I’m trying to write a code for a numerical simulator in FP style (for practice).

The following code is a simple Euler integration.  
Since the differential equation is in general given as multi-input functions (like `dyn(y, t)`),  
I need a multi-argument way to use Transducers.jl (or, `...` functionality).  
To reuse codes, multi-input support is necessary.

+) I’m interested in Transducers but I often feel it’s awkward and makes me hard to code fast. It would be delightful if you share your experience when using Transducers.jl and coding in FP style in aspects of 1) coding speed, 2) performance of codes, and 3) advantages.

```julia
julia> function euler(func)
           _euler(y, t, dt) = y + func(y, t) * dt
       end
euler (generic function with 1 method)

julia> dyn(y, t) = -y
dyn (generic function with 1 method)

julia> method = euler(dyn)
(::var"#_euler#2"{typeof(dyn)}) (generic function with 1 method)

julia> ts = 0:0.01:1
0.0:0.01:1.0

julia> ys = zeros(size(ts))
101-element Array{Float64,1}:
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 0.0
 ⋮

julia> ys[1] = 2
2

julia> for i in 1:length(ts)-1
           ys[i+1] = method(ys[i], ts[i], ts[i+1]-ts[i])
       end

julia> ys
101-element Array{Float64,1}:
 2.0
 1.98
 1.9602
 1.940598
 1.9211920200000001
 1.9019800998
 1.882960298802
 1.86413069581398
 1.8454893888558404
 1.8270344949672819
 1.808764150017609
 1.790676508517433
 1.7727697434322587
 ⋮

```

EDIT: wrapping code makes it work as follows.

```julia
function integrator(func, y0, ts, method, args...; kwargs...)
    _ts = ts[1:end-1]
    Δts = diff(ts)
    propagate = method(func)
    _propagate(y, t_Δt) = propagate(y, t_Δt..., args...; kwargs...)
    ys = zip(_ts, Δts) |> Scan(_propagate, y0) |> collect
    return ys
end

```

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 20, 2021, 1:05am UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622/4 "2021-02-20T01:05:14Z")

</div>

Before anything, I think you’d want to check out [DifferentialEquations.jl](https://github.com/SciML/DifferentialEquations.jl) universe, if you haven’t.

A short answer for your question is that Tnrasducers.jl is not very beneficial for this type of computation, unless you need to do something very specific.

Having said that, it is certainly possible to use Transducers.jl for this

```julia
euler(dyn, y0, dt) = Scan(y0) do y, i
    y + dyn(y, i * dt) * dt
end
# e.g., 1:10 |> euler(dyn, 1.0, 0.01) |> collect

```

or, if you want the input to be the sequence of times:

```julia
euler2(dyn, y0, t0) = ScanEmit((y0, t0)) do (y, t_prev), t
    dt = t - t_prev
    y_next = y + dyn(y, t) * dt
    y_next, (y_next, t)
end
# e.g., 0.001:0.01:0.1 |> euler2(dyn, 1.0, 0.0) |> collect

```

(Side note: you can think of [`ScanEmit`](https://juliafolds.github.io/Transducers.jl/dev/reference/manual/#Transducers.ScanEmit) as a “discrete dynamical system executor”. So, if you can express your simulator as a discrete dynamical system, you can always use `ScanEmit`.)

However, I don’t think it is always a good idea to implement this kind of code with Trasudcers.jl, unless you have some specific technical requirements. For example:

1. Fuse post-processing of the output of the system you are simulating; e.g., you are interested in a subset of the state variables or you want to filter out some outputs.
2. Fuse event detection predicate functions in a very efficient manner.
3. Simulating a system that is driven by input data (as in x(t) of dy/dt = f(y(t), x(t), t)) and the input data is stored in a complex data structure (e.g., nested array)

> [@iHany](#):
>
> 1. coding speed

I’m probably the worst person to answer this question, since I designed the API for how I want to code.

But there are things very hard to express without transducers. For example, doing something like [`Transducers.Consecutive`](https://juliafolds.github.io/Transducers.jl/dev/reference/manual/#Transducers.Consecutive) in parallel without transducers is very tricky (= very slow coding speed).

> [@iHany](#):
>
> 1. performance of codes

Transducers.jl APIs are fast sometimes and slow sometimes. I’ve been trying to reduce the slow cases but there are always possible improvements.

> [@iHany](#):
>
> 1. advantages

I think parallelism is the strongest selling point. Check out [[ANN] Folds.jl: threaded, distributed, and GPU-based high-level data-parallel interface for Julia](https://discourse.julialang.org/t/ann-folds-jl-threaded-distributed-and-gpu-based-high-level-data-parallel-interface-for-julia/54701)

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [February 20, 2021, 1:48am UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622/5 "2021-02-20T01:48:22Z")

</div>

Thanks a lot 🙂

I’ve seen [DifferentialEquations.jl](https://github.com/SciML/DifferentialEquations.jl) but I sometimes felt it’s not suitable for my work including reinforcement learning and optimal control in continuous time with complex nested dynamical systems.

(I have no idea how to cite a certain part of your answer but) the examples you shared look nice.  
It would be much easier to write codes with Transducers.jl if I follow your examples.

And I agree with that some functionality of Transducers.jl (especially `ScanEmit`) seems identical to discrete dynamical system and numerical simulation can be expressed as discrete dynamical system in general so I’ve been interested in numerical simulation using Transducers.jl.

Actually, I think that convenient post-processing and event detection are the most expected points when using Transducers.jl for numerical simulation. Thanks for clarifying it.

I’m also interested in `Folds.jl`.  
I will practice and try to be familiar with Transducers universe.  
Thank you again!

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 22, 2021, 8:05pm UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622/6 "2021-02-22T20:05:26Z")

</div>

> [@tkf](#):
>
> I think parallelism is the strongest selling point.

I should’ve mentioned that stateful transducers like the `euler` example I wrote is not parallelizable (in the sense you can’t do `... |> euler(dyn, y0, t) |> tcollect` etc.). (Well, unless when the system is linear, but then you don’t need to simulate it.)

---

<div class="post-metadata">

**Author:** ![iHany](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ihany/32/18151_2.png) [@iHany](https://discourse.julialang.org/u/iHany)\
**Post date:** [February 22, 2021, 10:43pm UTC](https://discourse.julialang.org/t/some-questions-on-transducers-jl/55622/7 "2021-02-22T22:43:22Z")

</div>

I agree with you.  
It’s probably impossible in principle to apply parallelism to a single episode simulation for a causal dynamical system.  
Instead, parallelism can be applied (very easily) to simulation of multiple episodes (for example, various initial conditions).

It’s actually already on progress to develop a package for transuder-based dynamical system simulator and I think that it provides flexible and convenient interface. For more information, see [LazyFym.jl](https://github.com/JinraeKim/LazyFym.jl) and [a question about parallelism](https://discourse.julialang.org/t/parallel-implementation-of-transducers-jl/55767/2).