# Flux: concatenate layers
**URL:** https://discourse.julialang.org/t/flux-concatenate-layers/18877
**Category:** Machine Learning
**Created:** [December 21, 2018, 4:02am UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877 "2018-12-21T04:02:11Z")
**Posts on this page:** 8
**Page:** 1
### Author:  [@zot](https://discourse.julialang.org/u/zot)
#### Post date: [December 21, 2018, 4:02am UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/1 "2018-12-21T04:02:11Z")
I’m looking for a way to concatenate layers in Flux like this:
Would this work for that? The input is an array of inputs to the layers and the output is the concatenation of the layers’ outputs. Creating it would be something like `Chain(Concat(Chain(A1, B1), Chain(A2, B2, C2)), D, E)`
```julia
using Flux
using Flux: @treelike
struct Concat
layers::Array
end
@treelike Concat
function (c::Concat)(inputs::Array)
output = []
for i in 1:length(c.layers)
append!(output, c.layers[i](inputs[i]))
end
output
end
```
---
### Author:  [@avikpal](https://discourse.julialang.org/u/avikpal)
#### Post date: [December 27, 2018, 4:10pm UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/2 "2018-12-27T16:10:10Z")
You can simply do
```julia
Chain(x -> cat(Chain(A1, B1)(x), Chain(A2, B2, C2)(x), dims=3), # Concatenating along channel dimension
D, E)
```
---
### Author:  [@zot](https://discourse.julialang.org/u/zot)
#### Post date: [January 6, 2019, 5:43pm UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/3 "2019-01-06T17:43:51Z")
Thanks!
---
### Author:  [@stu](https://discourse.julialang.org/u/stu)
#### Post date: [July 15, 2019, 3:32pm UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/4 "2019-07-15T15:32:25Z")
Can you help explain why you are concatenating along the channel dimension (dim=3)?
---
### Author:  [@Zach\_Christensen](https://discourse.julialang.org/u/Zach_Christensen)
#### Post date: [July 15, 2019, 6:00pm UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/5 "2019-07-15T18:00:37Z")
The channel dimension is typically synonymous with “features”. We typically assume that a network is learning features and combining multiple networks for further processing would be for combining features in another network.
---
### Author:  [@stu](https://discourse.julialang.org/u/stu)
#### Post date: [July 15, 2019, 6:24pm UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/6 "2019-07-15T18:24:59Z")
If this is the case, why is it that when I run the following code to attempt to reproduce the suggestion above:
```julia
using Flux
A1 = Dense(5, 5)
B1 = Dense(5, 5)
A2 = Dense(5, 5)
B2 = Dense(5, 5)
C2 = Dense(5, 5)
D = Dense(5, 5)
E = Dense(5, 5)
model = Chain(x -> cat(Chain(A1, B1)(x), Chain(A2, B2, C2)(x); dims=3), D, E)
model(rand(5))
ERROR: MethodError: no method matching *(::TrackedArray{…,Array{Float32,2}}, ::TrackedArray{…,Array{Float32,3}})
```
I get the above error when I try to run the model on any input? It does not seem that concatenating along the third dimension works.
---
### Author:  [@Sleort](https://discourse.julialang.org/u/Sleort)
#### Post date: [July 17, 2019, 1:00am UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/7 "2019-07-17T01:00:24Z")
That’s because the previous respondents for some reason assumed that the layers where [convolutional](https://fluxml.ai/Flux.jl/stable/models/layers/#Flux.Conv) and the input two-dimensional (images). `Dense`, [on the other hand](https://fluxml.ai/Flux.jl/stable/models/layers/#Flux.Dense), takes in and returns a vector (or a batch of vectors, stored as a matrix). This will work:
```julia
using Flux
A1 = Dense(5, 5)
B1 = Dense(5, 5)
A2 = Dense(5, 5)
B2 = Dense(5, 5)
C2 = Dense(5, 5)
D = Dense(10, 5)
E = Dense(5, 5)
model = Chain(x -> cat(Chain(A1, B1)(x), Chain(A2, B2, C2)(x); dims=1), D, E)
model(rand(5))
```
or, equivalently,
```julia
model = Chain(x -> vcat(Chain(A1, B1)(x), Chain(A2, B2, C2)(x)), D, E)
```
(Note that we have to write `D = Dense(10, 5)` instead of `D = Dense(5, 5)`.)
---
### Author:  [@tgrassi](https://discourse.julialang.org/u/tgrassi)
#### Post date: [September 18, 2020, 11:13am UTC](https://discourse.julialang.org/t/flux-concatenate-layers/18877/8 "2020-09-18T11:13:45Z")
As @tomerarnon suggested in [the answer to this question](https://discourse.julialang.org/t/flux-params-does-not-recognize-parameters-with-x-layer-x-syntax/46832/5), the solution proposed here doesn’t work properly. A workaround is to use the `Concat` struct.