# Flux. Pooling followed by Dense
**URL:** https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394
**Category:** Machine Learning
**Created:** [March 4, 2022, 10:05am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394 "2022-03-04T10:05:13Z")
**Posts on this page:** 11
**Page:** 1
### Author:  [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)
#### Post date: [March 4, 2022, 10:05am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/1 "2022-03-04T10:05:13Z")
I am at a loss about the following. I am building a small 2D network where the last 2 layers are a `MaxPool` followed by a `Dense`. The parameters to create `MaxPool` do not care about the size of the input. But the `Dense` just afterwards do.
I have tried many ways to extract this work but without success. (A `Flux.flatten` is in between) . Hoping for guidance elsewhere, I have gone through the source code of Flux and NNLib. Every single example hard-codes the input size of `Dense` or _cheats_ with keeping the stride of the `MaxPool` at 1.
I am sure I am miising something stupid.
---
### Author:  [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)
#### Post date: [March 4, 2022, 10:37am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/2 "2022-03-04T10:37:24Z")
What exactly is the question?
---
### Author:  [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)
#### Post date: [March 4, 2022, 10:39am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/3 "2022-03-04T10:39:47Z")
How to append a `Dense` layer after a `Pooling` layer when I don’ t know what the size coming out of `MaxPool` is and `Dense` requires to know it.
---
### Author:  [@albheim](https://discourse.julialang.org/u/albheim)
#### Post date: [March 4, 2022, 10:45am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/4 "2022-03-04T10:45:23Z")
You can do something like this if you don’t want to calculate it manually. Otherwise it shouldn’t be too hard to figure out an expression for the output shape based on the stride and padding of the maxpool in combination with the size of the output from the convolution (which might also need some calculation).
```julia
julia> xs = rand(Float32, 100, 100, 3, 50);
julia> layer1 = Conv((5,5), 3 => 7, relu; bias = false)
Conv((5, 5), 3 => 7, relu, bias=false) # 525 parameters
julia> layer2 = MaxPool((5, 5), pad=SamePad())
MaxPool((5, 5), pad=2)
julia> layer3 = flatten
flatten (generic function with 1 method)
julia> tmp = Chain(layer1, layer2, layer3)
Chain(
Conv((5, 5), 3 => 7, relu, bias=false), # 525 parameters
MaxPool((5, 5), pad=2),
Flux.flatten,
)
julia> size(tmp(xs))
(2800, 50)
julia> layer4 = Dense(size(tmp(xs), 1), 3)
Dense(2800, 3) # 8_403 parameters
julia> model = Chain(layer1, layer2, layer3, layer4)
Chain(
Conv((5, 5), 3 => 7, relu, bias=false), # 525 parameters
MaxPool((5, 5), pad=2),
Flux.flatten,
Dense(2800, 3), # 8_403 parameters
) # Total: 3 arrays, 8_928 parameters, 35.414 KiB.
```
---
### Author:  [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)
#### Post date: [March 4, 2022, 11:51am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/5 "2022-03-04T11:51:56Z")
Thanks for putting the time in this answer.
I have tried all of that.
I did define a generic struct where you stop in the constructor to extract the size… Dosn’t work because no information about the input at that point of the definition.
I then did this within a function being the “forwarding” function. It creates a model fine. But applying to an actual input always bombs out. (Including Zygote trying to differentiate the size.)
The forwarding function is where it has to happen. But this is where I cannot find a way to extract the size of the output from `MaxPool`.
I have a helper function to generate size and iterators of convolutions. It works everywhere in the code, but there. Never gives me a consistent correct result.
I might end up creating a replicate of MaxPool or getting rid of it. But this is not really a satisfying answer.
P.S.: Don’t get me started of Zygote complaining about mutating arrays. That has been biting on and on.
---
### Author:  [@albheim](https://discourse.julialang.org/u/albheim)
#### Post date: [March 4, 2022, 1:10pm UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/6 "2022-03-04T13:10:47Z")
Hmm, I’m not sure I understand.
> [@Emmanuel-R8](#):
>
> I did define a generic struct where you stop in the constructor to extract the size… Dosn’t work because no information about the input at that point of the definition.
But what do you know about your input? You know it will be RGB images of the same dimensions?
> [@Emmanuel-R8](#):
>
> I then did this within a function being the “forwarding” function. It creates a model fine. But applying to an actual input always bombs out. (Including Zygote trying to differentiate the size.)
So you create a function that does the forward pass, and halfway through you try to check what the current size of the data is and generate a dense layer corresponding to that size? That seems like it would be problematic in many ways.
> [@Emmanuel-R8](#):
>
> I have a helper function to generate size and iterators of convolutions. It works everywhere in the code, but there. Never gives me a consistent correct result.
This is what I was thinking would be the nicer solution. What is the problem, can you share the function and a case where it gives the wrong answer?
---
### Author:  [@CarloLucibello](https://discourse.julialang.org/u/CarloLucibello)
#### Post date: [March 4, 2022, 1:27pm UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/7 "2022-03-04T13:27:08Z")
@Emmanuel-R8 maybe you are looking for `Flux.outputsize`? It is still not clear to me what you are trying to achieve, a code example would help
---
### Author:  [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [March 4, 2022, 5:09pm UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/8 "2022-03-04T17:09:47Z")
Let me flip the question around and ask: how would you do this with Python libraries? Because if you can articulate that, then the Flux solution will be a pretty direct translation.
For example, with TF/PyTorch your options are:
1. Fix the input size and derive the pre-dense output size from that. If you want help from something like tf.Keras’ shape inference, check out `Flux.outputsize` as @CarloLucibello mentioned.
2. Use adaptive or global pooling. Unlike normal pooling layers, these generate a fixed size output for variable-sized inputs. Global pooling in particular is bread-and-butter for most vision models, but Flux has layers for both types.
---
### Author:  [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)
#### Post date: [March 5, 2022, 6:15am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/9 "2022-03-05T06:15:32Z")
Thanks. I went with the adaptative layer option.
I uploaded the code to [https://github.com/Alba-Intelligence/SharpenedCorrelation](https://github.com/Alba-Intelligence/SharpenedCorrelation).
The idea is to implement a variant of the Sharpened Cosine Similarity. See [https://www.rpisoni.dev/posts/cossim-convolution/](https://www.rpisoni.dev/posts/cossim-convolution/) for a description, and [https://e2eml.school/scs.html](https://e2eml.school/scs.html) for various implementations.
Now, I reached the dreaded Zygote complaints about mutating arrays.
---
### Author:  [@ToucheSir](https://discourse.julialang.org/u/ToucheSir)
#### Post date: [March 8, 2022, 5:10pm UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/10 "2022-03-08T17:10:00Z")
> [@Emmanuel-R8](#):
>
> Now, I reached the dreaded Zygote complaints about mutating arrays.
As always, this is where at bare minimum a full stacktrace would be necessary, if not a MWE as well.
---
### Author:  [@Emmanuel-R8](https://discourse.julialang.org/u/Emmanuel-R8)
#### Post date: [March 9, 2022, 2:44am UTC](https://discourse.julialang.org/t/flux-pooling-followed-by-dense/77394/11 "2022-03-09T02:44:28Z")
Of course. I wrote that with a lot of negative feelings dreading the incoming hair pulling.
(any questions about it would warrant a separate thread anyway)