# Writing better / faster / more julia-ish code

**URL:** <https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269>\
**Category:** General Usage\
**Tags:** macros, reactiveprogramming\
**Created:** [April 18, 2017, 4:57pm UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269 "2017-04-18T16:57:43Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![pyrex41](https://avatars.discourse-cdn.com/v4/letter/p/4bbf92/32.png) [@pyrex41](https://discourse.julialang.org/u/pyrex41)\
**Post date:** [April 18, 2017, 4:57pm UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269/1 "2017-04-18T16:57:44Z")

</div>

Using ReactiveBasics, I am using Signals to model time series events (extension of discussion [here](https://discourse.julialang.org/t/using-gensym-properly/3257/6)). I want to build a signal that gives the median value of the last 5 inputs to another signal. I have developed 2 approaches, and I suspect both are much slower than they need to be. How can I improve?

Approach 1:

```julia
ff = Signal(0.0)

macro medWin(sig,win = 5)
    arr = []
    quote
        flatmap($(esc(sig))) do iv
            unshift!($arr, iv)
            length($arr) > $(esc(win)) ? pop!($arr) : false
            median($arr) |> Signal
        end
    end
end

function testmacro()
    @time begin
        f = Signal(0.0)
        g = @medWin(f)
        for i=1:10^5
            push!(f, i)
        end
        f,g
    end
end

testmacro()

```

Output:

```julia
1.243613 seconds (898.99 k allocations: 32.028 MB, 0.70% gc time)

```

I was concerned that using an array within the global scope to hold values would be expensive, so I tried to find another approach that would use the functions in ReactiveSignal more directly:

```julia
function lastX(u::Signal, n ::Int)
    sigs = Array{Signal}(1,n+1)
    sigs[1] = u
    for i=1:n
        sigs[i+1] = previous(sigs[i])
    end
    return zip(sigs[2:end]...)
end

function testfunction()
    @time begin
        f = Signal(0.0)
        g = lastX(f, 5)
        h = flatmap(g) do tup
            v = collect(tup)
            return median(v) |> Signal
        end
        for i=1:10^5
            push!(f, i)
        end
        f,h
    end
end

```

Output:

```julia
 1.755510 seconds (13.00 M allocations: 633.236 MB, 16.01% gc time)

```

I suspect the answer lies in the ReactiveBasics implementations of `previous`, `subscribe!`, and `flatmap`; would have to look closer. Any insight into why this uses so much memory would be helpful to me learning.

Is my first approach as good as it gets? Or are there more improvements that can be achieved?

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [April 18, 2017, 5:31pm UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269/2 "2017-04-18T17:31:03Z")

</div>

```julia
    arr = []
    quote
        ...
    end

```

I’m pretty much sure it’s not what you actually want because `arr` exists only during macro expand time and it’s starting value - `Any[]` - is inserted everywhere instead of it’s name. Try running `macroexpand(:(@medWin Signal(0)))` in Julia REPL to see what is actually generated.

Also running your code on Julia 0.6 gives an error that `lastX` is not defined.

---

<div class="post-metadata">

**Author:** ![pyrex41](https://avatars.discourse-cdn.com/v4/letter/p/4bbf92/32.png) [@pyrex41](https://discourse.julialang.org/u/pyrex41)\
**Post date:** [April 18, 2017, 5:42pm UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269/3 "2017-04-18T17:42:53Z")

</div>

I did not expect the macro approach to work like this, either; however, I tried this based on [this thread](https://discourse.julialang.org/t/using-gensym-properly/3257/5). To demonstrate:

```julia
ff = Signal(0.0)
gg = @medWin(ff,3)

for i=1:10
    push!(ff, i)
    println("ff is ", ff.value)
    println("gg is ", gg.value)
    println("-----")
end

```

Gives desired output for gg (median of last 3 values of f):

```julia
ff is 1.0
gg is 0.5
-----
ff is 2.0
gg is 1.0
-----
ff is 3.0
gg is 2.0
-----
ff is 4.0
gg is 3.0
-----
ff is 5.0
gg is 4.0
-----
ff is 6.0
gg is 5.0
-----
ff is 7.0
gg is 6.0
-----
ff is 8.0
gg is 7.0
-----
ff is 9.0
gg is 8.0
-----
ff is 10.0
gg is 9.0
-----

```

And, i forgot to include my lastX function:

```julia
function lastX(u::Signal, n ::Int)
    sigs = Array{Signal}(1,n+1)
    sigs[1] = u
    for i=1:n
        sigs[i+1] = previous(sigs[i])
    end
    return zip(sigs[2:end]...)
end

```

---

<div class="post-metadata">

**Author:** ![yuyichao](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/yuyichao/32/20_2.png) [@yuyichao](https://discourse.julialang.org/u/yuyichao)\
**Post date:** [April 18, 2017, 6:46pm UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269/4 "2017-04-18T18:46:38Z")

</div>

> [@pyrex41](#):
>
> I did not expect the macro approach to work like this, either; however, I tried this based on this thread. To demonstrate:
> 
> ff = Signal(0.0)  
> gg = @medWin(ff,3)

I’m not sure what you are expecting but [the difference between the two is that](https://discourse.julialang.org/t/using-gensym-properly/3257/4) by splicing in the value, you have

> a compile time generated (non-threadsafe and non-reentrant) array that are shared through multiple runtime execution of the same macro expansion

Which doesn’t seems to be what you want unless you want multiple invocation at **runtime** to affect each other or you will never use this macro in a function or a local scope.

Your domonstration doesn’t show the difference between generating the array at compile time and runtime. You should only do this if you really know what it is doing. Trying things and observing something that seems to work without understanding/checking the document is one of the major reason why people write code/packages that breaks easily due to changes in base. (It’s unlikely that this particular behavior will break but it’s a bad habit to do so.)

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [April 18, 2017, 7:58pm UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269/5 "2017-04-18T19:58:28Z")

</div>

I’m afraid it’s still not runnable since `previous` is undefined. Can you provide a complete runnable example?

---

<div class="post-metadata">

**Author:** ![pyrex41](https://avatars.discourse-cdn.com/v4/letter/p/4bbf92/32.png) [@pyrex41](https://discourse.julialang.org/u/pyrex41)\
**Post date:** [April 19, 2017, 11:56am UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269/6 "2017-04-19T11:56:53Z")

</div>

It appears that not all of the functions [here on github](https://github.com/tshort/ReactiveBasics.jl/blob/master/src/ReactiveBasics.jl) have been pushed to the version found in Pkg.add(“ReactiveBasics”). For completeness, I have included all the functions applicable below:

```julia
type Signal{T}
   value::T
   callbacks::Vector{Function}   
end

Signal(val) = Signal(val, Function[])

value(u::Signal) = u.value

function subscribe!(f, u::Signal)
    push!(u.callbacks, f)
    u
end

function flatmap(f, input::Signal)
    signal = Signal(f(input.value).value)
    subscribe!(input) do u
        innersig = f(u)
        push!(signal, innersig.value)
        subscribe!(innersig) do v
            push!(signal, v)
        end
    end      
    signal
end

function Base.map(f, u::Signal)
    signal = Signal(f(u.value))
    subscribe!(x -> push!(signal, f(x)), u)
    signal
end
function Base.map(f, u::Signal, v::Signal)
    signal = Signal(f(u.value, v.value))
    subscribe!(x -> push!(signal, f(x, v.value)), u)
    subscribe!(x -> push!(signal, f(u.value, x)), v)
    signal
end
function Base.map(f, u::Signal, v::Signal, w::Signal)
    signal = Signal(f(u.value, v.value, w.value))
    subscribe!(x -> push!(signal, f(x, v.value, w.value)), u)
    subscribe!(x -> push!(signal, f(u.value, x, w.value)), v)
    subscribe!(x -> push!(signal, f(u.value, v.value, x)), w)
    signal
end
function Base.map(f, u::Signal, v::Signal, w::Signal, xs::Signal...)
    us = (u,v,w,xs...)
    signal = Signal(f((u.value for u in us)...))
    for (i,u) in enumerate(us)
        subscribe!(u) do x
            vals = f(((i == j ? x : us[j].value for j in 1:length(us))...)...)
            push!(signal, vals)
        end
    end
    signal
end

function Base.push!(u::Signal, val)
    u.value = val
    foreach(f -> f(val), u.callbacks)
end

function Base.zip(u::Signal, us::Signal...)
    map((args...) -> (args...), u, us...)
end

function previous(input::Signal, default=value(input))
    past = Ref(default)
    map(input) do u
        res = past[]
        past[] = u
        res
    end
end

```

And my code (note that I have moved “arr” within the quote of the macro, based on comments above):

```julia
function lastX(u::Signal, n ::Int)
    sigs = Vector{Signal}(n+1)
    sigs[1] = u
    for i=1:n
        sigs[i+1] = previous(sigs[i])
    end
    return zip(sigs[2:end]...)
end

function testfunction()
    @time begin
        f = Signal(0.0)
        g = lastX(f, 5)
        h = flatmap(g) do tup
            v = collect(tup)
            return median(v) |> Signal
        end
        for i=1:10^5
            push!(f, i)
        end
        f,h
    end
end

macro medWin(sig,win = 5)
    quote
        arr = []
        flatmap($(esc(sig))) do iv
            unshift!(arr, iv)
            length(arr) > $(esc(win)) ? pop!(arr) : false
            median(arr) |> Signal
        end
    end
end

function testmacro()
    @time begin
        f = Signal(0.0)
        g = @medWin(f)
        for i=1:10^5
            push!(f, i)
        end
        f,g
    end
end

testfunction()

testmacro()

```

---

<div class="post-metadata">

**Author:** ![dfdx](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dfdx/32/120_2.png) [@dfdx](https://discourse.julialang.org/u/dfdx)\
**Post date:** [April 19, 2017, 9:47pm UTC](https://discourse.julialang.org/t/writing-better-faster-more-julia-ish-code/3269/7 "2017-04-19T21:47:44Z")

</div>

A bit of benchmarking (using `BenchmarkTools.@benchmark`), profiling (using `@profile` macro) and `@time` on subexpressions shows that by far the most expensive part of the code is:

```julia
for i=1:10^5
    push!(f, i)
end

```

Which assigns a new value to a signal (fast) and calls its single callback (slow). Without learning the library and diving deep into your code, I don’t know what this (anonymous) callback actually is, but this is exactly your bottleneck. Optimize this function and you’ll optimize the whole code.