# Convolution with a multi-layer kernel

I have a 2D input that I would like to run a convolutional model on. However, instead of using a matrix as a convolution kernel, I would like to use something like `Chain(Dense(3^2; 3^2, tanh), ...)` as a kernel. Since `conv2d` in NNlib uses a gemm routine internally, I don’t think I can use it for my purposes. Are there some useful functions that I overlooked to implement this or do I need to roll my own loop-based implementation?

1 Like

Marten I am a little confused by your interest here.

So you would like the output of a chain to be convolved as a filter in a convolution operation?

I’ve never tried this but I don’t see why you couldn’t. Have you tried something like the following?

``````a = Chain(Dense(3^2; 3^2, tanh), ...)
b = Conv2d(...)

b.weight = a(x)
b(z)
``````

?

I was wondering whether the input to this Chain was supposed to be a view of 3x3 pixels, scanned over the array. But what happens to its output? What is `...`, what object does this Chain return?

1 Like

I just tried and it does not work because `Conv.weight` needs to be of type `Array{Float32,4}`.

In a normal convolutional layer, I map each element with it’s surrounding let’s say 3x3 block to a new value with the filter matrix. I would like to do the same thing but instead of using a matrix, I would like to apply a general neural network at each step.

You can emulate “convolution with a chain of dense layers” by using 1x1 conv layers.
Convolution with `Chain( Dense(3*3, c1), Dense(c1, c2), Dense(c2, c3))` is equivalent to `Chain(Conv((3,3), 1 => c1), Conv((1,1), c1 => c2), Conv((1,1), c2 => c3))`

3 Likes

There is a way to solve that problem. I believe `reshape`, or `permutedims` will do the trick.

Googling found this, maybe it will help you:julia - How do I add a dimension to an array? (opposite of `squeeze`) - Stack Overflow

I like that solution. I only need dense layers in my convolution anyway and this way I can reuse existing layers.