I have a 2D input that I would like to run a convolutional model on. However, instead of using a matrix as a convolution kernel, I would like to use something like Chain(Dense(3^2; 3^2, tanh), ...) as a kernel. Since conv2d in NNlib uses a gemm routine internally, I don’t think I can use it for my purposes. Are there some useful functions that I overlooked to implement this or do I need to roll my own loop-based implementation?
1 Like
Marten I am a little confused by your interest here.
So you would like the output of a chain to be convolved as a filter in a convolution operation?
I’ve never tried this but I don’t see why you couldn’t. Have you tried something like the following?
a = Chain(Dense(3^2; 3^2, tanh), ...)
b = Conv2d(...)
b.weight = a(x)
b(z)
?
I was wondering whether the input to this Chain was supposed to be a view of 3x3 pixels, scanned over the array. But what happens to its output? What is ..., what object does this Chain return?
1 Like
I just tried and it does not work because Conv.weight needs to be of type Array{Float32,4}.
In a normal convolutional layer, I map each element with it’s surrounding let’s say 3x3 block to a new value with the filter matrix. I would like to do the same thing but instead of using a matrix, I would like to apply a general neural network at each step.
You can emulate “convolution with a chain of dense layers” by using 1x1 conv layers.
Convolution with Chain( Dense(3*3, c1), Dense(c1, c2), Dense(c2, c3)) is equivalent to Chain(Conv((3,3), 1 => c1), Conv((1,1), c1 => c2), Conv((1,1), c2 => c3))
3 Likes
There is a way to solve that problem. I believe reshape, or permutedims will do the trick.
Googling found this, maybe it will help you:julia - How do I add a dimension to an array? (opposite of `squeeze`) - Stack Overflow
I like that solution. I only need dense layers in my convolution anyway and this way I can reuse existing layers.