# Possible anonymous function bug in Flux

**URL:** <https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389>\
**Category:** Machine Learning\
**Tags:** flux\
**Created:** [July 15, 2019, 3:27pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389 "2019-07-15T15:27:44Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![stu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stu/32/8772_2.png) [@stu](https://discourse.julialang.org/u/stu)\
**Post date:** [July 15, 2019, 3:27pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/1 "2019-07-15T15:27:44Z")

</div>

Has anyone noticed that

```julia
m = Chain(Dense(5, 2), Dense(2, 2) Dense(2, 5))

```

produces an entirely result than  
`m = Chain(x -> Dense(5, 2)(x), Dense(2, 2) Dense(2, 5))`  
are anonymous functions somehow not supported for Chain?

Also, I do not mean different as in like they randomly generated value when initialized are different, I mean like when you seed the random number generator and try to train one versus the other, the anonymous function version simply not only fails to produce the same output but even after more training just does not accomplish any kind of meaningful fitting of the data.

For example, try replacing in any model a Dense layer with a layer written as x → Dense(…)(x) and see if your model works as it did before.

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [July 15, 2019, 3:31pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/2 "2019-07-15T15:31:15Z")

</div>

`params` can’t find the parameters inside the dense layer used in the anonymous function. You need to manually add those parameters to the set of parameters being trained

---

<div class="post-metadata">

**Author:** ![stu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stu/32/8772_2.png) [@stu](https://discourse.julialang.org/u/stu)\
**Post date:** [July 15, 2019, 3:41pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/3 "2019-07-15T15:41:08Z")

</div>

How exactly would you do that? Are you saying I need to store params(layer1) and have that as an input to the train! function if I were to use an anonymous function?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [July 15, 2019, 3:50pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/4 "2019-07-15T15:50:14Z")

</div>

You call `params(m) ` right? You need to call params on the dense layer in the anonymous function as well and concatenate the twob

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [July 15, 2019, 3:51pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/5 "2019-07-15T15:51:50Z")

</div>

Any trainable parameters closed over by an anonymous function must be manually extracted and added to the params sent to train. If you don’t want to do this, define a callable struct and run `@treelike` on it, this is how Dense is defined

---

<div class="post-metadata">

**Author:** ![stu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stu/32/8772_2.png) [@stu](https://discourse.julialang.org/u/stu)\
**Post date:** [July 15, 2019, 3:51pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/6 "2019-07-15T15:51:50Z")

</div>

> [@baggepinnen](#):
>
> `params` can’t find the parameters inside the dense layer used in the anonymous function. You need to manually add those parameters to the set of parameters being trained

Awesome thank you. That is very helpful. Especially that I can just concatenate them.

---

<div class="post-metadata">

**Author:** ![stu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stu/32/8772_2.png) [@stu](https://discourse.julialang.org/u/stu)\
**Post date:** [July 15, 2019, 4:00pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/7 "2019-07-15T16:00:45Z")

</div>

One thing: it looks like that in order to concatenate the two, I would need to call collect() in order to access that Array as otherwise the return type of params() is Params(). Once I call collect, I can vcat() the arrays but I’m not sure how to add the Params type back, is there a way to do this?

---

<div class="post-metadata">

**Author:** ![baggepinnen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/baggepinnen/32/693_2.png) [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)\
**Post date:** [July 15, 2019, 4:26pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/8 "2019-07-15T16:26:25Z")

</div>

Try something like

```julia
params((m, layer))

```

---

<div class="post-metadata">

**Author:** ![stu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stu/32/8772_2.png) [@stu](https://discourse.julialang.org/u/stu)\
**Post date:** [July 15, 2019, 4:57pm UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/9 "2019-07-15T16:57:18Z")

</div>

that works!  
and is a lot cleaner than the  
`Params(vcat(collect(params(m)), collect(params(layer))))`  
I was using

---

<div class="post-metadata">

**Author:** ![jaynick](https://avatars.discourse-cdn.com/v4/letter/j/71c47a/32.png) [@jaynick](https://discourse.julialang.org/u/jaynick)\
**Post date:** [April 14, 2020, 5:49am UTC](https://discourse.julialang.org/t/possible-anonymous-function-bug-in-flux/26389/10 "2020-04-14T05:49:07Z")

</div>

how is `layer` written for the given example, i.e.

> [@stu](#):
>
> m = Chain(x → Dense(5, 2)(x), Dense(2, 2) Dense(2, 5))

Meaning, the layer that params() does not understand is the anonymous function, but I guess you do not substute the anonymous function for layer, like  
`params((m, x -> Dense(5, 2)(x))`  
!