# Codegen woes

**URL:** <https://discourse.julialang.org/t/codegen-woes/27098>\
**Category:** Performance\
**Created:** [August 2, 2019, 1:34am UTC](https://discourse.julialang.org/t/codegen-woes/27098 "2019-08-02T01:34:23Z")\
**Posts on this page:** 11\
**Page:** 2

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [August 2, 2019, 4:30pm UTC](https://discourse.julialang.org/t/codegen-woes/27098/21 "2019-08-02T16:30:17Z")

</div>

> [@kristoffer.carlsson](#):
>
> You can kinda use FunctionWrappers.jl for this as well

Nice, I hadn’t seen FunctionWrappers before. A quick google search leads to [this package](https://github.com/yuyichao/FunctionWrappers.jl), last updated around a year ago. Is this the right place to get it?

Any comparisons with @ChrisRackauckas’s PR in terms of use cases, predictable performance, etc?

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [August 2, 2019, 6:21pm UTC](https://discourse.julialang.org/t/codegen-woes/27098/22 "2019-08-02T18:21:11Z")

</div>

> [@cscherrer](#):
>
> Is this the right place to get it?

Yep.

> [@cscherrer](#):
>
> Any comparisons with @ChrisRackauckas’s PR in terms of use cases, predictable performance, etc?

They both kinda do the same “trick” with `cfunction` and `ccall`. I haven’t done any real comparisons so can’t comment on that.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [August 2, 2019, 8:08pm UTC](https://discourse.julialang.org/t/codegen-woes/27098/23 "2019-08-02T20:08:10Z")

</div>

> [@mohamed82008](#):
>
> I think in @cscherrer’s case, the body of the function `eval` ed can have arbitrary user code including code from modules not visible inside the module defining the function which calls `eval`

This has been a difficulty anyway. Two things have helped work around this:

- I’m only using `eval` to build a function, so there’s some encapsulation of the scope anyway
- Any user dependencies are required to be passed in as arguments. Unfortunately, there’s currently no possibility of closures. This is a little bit of an annoyance, but makes reasoning about things statically much easier.

I have sometimes used a trick of passing around a `ctx` (for “context”) mapping, usually implemented as a Dict or NamedTuple. This lets you fake global scope without having to do anything too fancy. I might at some point make this accessible to users, though I’m not sure yet what that interface would look like.

---

<div class="post-metadata">

**Author:** ![chakravala](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chakravala/32/6832_2.png) [@chakravala](https://discourse.julialang.org/u/chakravala)\
**Post date:** [August 3, 2019, 12:02am UTC](https://discourse.julialang.org/t/codegen-woes/27098/24 "2019-08-03T00:02:05Z")

</div>

@ChrisRackauckas told me at JuliaCon that he doesn’t like `FunctionWrapper`, but it wasn’t clear to me what his issue with it was. Please elaborate, since I am going to implement something similar for [Leibniz.jl](https://github.com/chakravala/Leibniz.jl) this is an issue that affects me also.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [August 3, 2019, 12:27am UTC](https://discourse.julialang.org/t/codegen-woes/27098/25 "2019-08-03T00:27:57Z")

</div>

> [@chakravala](#):
>
> @ChrisRackauckas told me at JuliaCon that he doesn’t like `FunctionWrapper` , but it wasn’t clear to me what his issue with it was.

FunctionWrappers require that you put a type wrapper for the inputs and outputs. They are fine but a bit difficult to use in some cases, and they do have a noticeable performance overhead vs a standard call (though this is much less in 1.0+ than it was in v0.6). It think it’s easier to just invoke in a way where you say what the return is than to wrap every function in a type. I wonder if there is an overhead difference.

---

<div class="post-metadata">

**Author:** ![cortner](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cortner/32/204_2.png) [@cortner](https://discourse.julialang.org/u/cortner)\
**Post date:** [August 3, 2019, 5:52am UTC](https://discourse.julialang.org/t/codegen-woes/27098/26 "2019-08-03T05:52:07Z")

</div>

I’d be very interested in a benchmark on this.

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [August 27, 2019, 1:11am UTC](https://discourse.julialang.org/t/codegen-woes/27098/28 "2019-08-27T01:11:56Z")

</div>

I think @thautwarm is very close to solving this problem once and for all with [GG.jl](https://github.com/thautwarm/GG.jl).

Here’s an example in Soss which uses GG:

```julia
julia> m = @model begin
       μ ~ Normal()
       x ~ Normal(μ,1)
       end
@model begin
        μ ~ Normal()
        x ~ Normal(μ, 1)
    end

julia> @btime rand(m)
  24.240 ns (1 allocation: 32 bytes)
(μ = 1.5394913244089232, x = 0.53374410189993)

```

It’s even faster if the model is `const` (still figuring out how to avoid the need for this):

```julia
julia> const m2 = m
@model begin
        μ ~ Normal()
        x ~ Normal(μ, 1)
    end

julia> @btime rand(m2)
  10.610 ns (0 allocations: 0 bytes)
(μ = 2.0368121003101187, x = 0.06231985426467612)

```

Here’s the hand-written version for comparison:

```julia
julia> function f()
       μ = rand(Normal())
       x = rand(Normal(μ,1.0))
       (μ=μ,x=x)
       end
f (generic function with 1 method)

julia> @btime f()
  10.439 ns (0 allocations: 0 bytes)
(μ = -0.5998600215661466, x = -1.0367247568633182)

```

After some setup, the implementation can be very short:

```julia
export rand
@generated function rand(m::Model{T} where T) 
    type2model(m) |> sourceRand
end

```

where `sourceRand` generates what you’d expect,

```julia
julia> sourceRand(m)
quote
    μ = rand(Normal())
    x = rand(Normal(μ, 1))
    (μ = μ, x = x)
end

```

---

<div class="post-metadata">

**Author:** ![dawbarton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dawbarton/32/215461_2.png) [@dawbarton](https://discourse.julialang.org/u/dawbarton)\
**Post date:** [August 27, 2019, 9:12am UTC](https://discourse.julialang.org/t/codegen-woes/27098/29 "2019-08-27T09:12:56Z")

</div>

I’d guess that the `const` issue is just a benchmarking problem - does

```julia
@btime rand($m)

```

give you the faster timings?

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [August 27, 2019, 1:38pm UTC](https://discourse.julialang.org/t/codegen-woes/27098/30 "2019-08-27T13:38:29Z")

</div>

> [@dawbarton](#):
>
> I’d guess that the `const` issue is just a benchmarking problem

You may be right. I’m in the middle of other makes some otrher changes now, but IIRC that does fix it. But having a model defined at the global level is a common use case, and generated functions like this will be in inner loops, so I need to be sure I understand what’s going on and that there won’t be a real performance hit.

---

<div class="post-metadata">

**Author:** ![dawbarton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dawbarton/32/215461_2.png) [@dawbarton](https://discourse.julialang.org/u/dawbarton)\
**Post date:** [August 27, 2019, 2:14pm UTC](https://discourse.julialang.org/t/codegen-woes/27098/31 "2019-08-27T14:14:03Z")

</div>

I don’t think you need to worry about things like that; you take a hit on the first function call because of the dynamic dispatch (the compiler doesn’t know what type the global variable is) but after that it doesn’t matter. Moreover, the dynamic dispatch is (largely?) a constant overhead and so, though your example has a x2 slowdown it’s actually a constant overhead rather than a multiplier. This can have a big impact on benchmarking small functions but that’s largely it.

A quick example

```julia
julia> a = 0.5
0.5

julia> g(a) = cos(a)
g (generic function with 1 method)

julia> h(a) = (cos(a), cos(2a)) # twice the work
h (generic function with 1 method)

julia> @btime g(a)
  23.247 ns (1 allocation: 16 bytes)
0.8775825618903728

julia> @btime g($a)
  7.216 ns (0 allocations: 0 bytes)
0.8775825618903728

julia> @btime h(a) # twice the work but not twice the time; more like time of g(a) + g($a)
  31.720 ns (1 allocation: 32 bytes)
(0.8775825618903728, 0.5403023058681398)

```

---

<div class="post-metadata">

**Author:** ![cscherrer](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cscherrer/32/7631_2.png) [@cscherrer](https://discourse.julialang.org/u/cscherrer)\
**Post date:** [August 27, 2019, 2:16pm UTC](https://discourse.julialang.org/t/codegen-woes/27098/32 "2019-08-27T14:16:16Z")

</div>

This is a great example, and really helpful. Thank you!

[Previous page](https://discourse.julialang.org/t/codegen-woes/27098.md?page=1)