# Autodiff on image convolution
**URL:** https://discourse.julialang.org/t/autodiff-on-image-convolution/25574
**Category:** Machine Learning
**Tags:** question
**Created:** [June 23, 2019, 3:58pm UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574 "2019-06-23T15:58:11Z")
**Posts on this page:** 7
**Page:** 1
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [June 23, 2019, 3:58pm UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574/1 "2019-06-23T15:58:11Z")
This post was temporarily hidden by the community for possibly being off-topic, unfocused, inappropriate, or spammy.
---
### Author:  [@baggepinnen](https://discourse.julialang.org/u/baggepinnen)
#### Post date: [June 23, 2019, 6:04pm UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574/2 "2019-06-23T18:04:52Z")
See [https://github.com/FluxML/NNlib.jl/blob/master/src/conv.jl](https://github.com/FluxML/NNlib.jl/blob/master/src/conv.jl)
Nnlib is what’s used by Flux.
---
### Author:  [@RoyiAvital](https://discourse.julialang.org/u/RoyiAvital)
#### Post date: [June 24, 2019, 5:57am UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574/3 "2019-06-24T05:57:35Z")
I think in this case it is best to use the explicit form of the gradient instead of using AD.
Why won’t you use that?
---
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [June 24, 2019, 7:56am UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574/4 "2019-06-24T07:56:29Z")
@baggepinnen Thank you, i think this will solve my issues.
@RoyiAvital That is also a good idea to solve the problem, but my model will probably get a lot more complicated, thus i’d rather exploit AD than writing the gradient myself (for the simple problem at hand it is actually very easy to compute it, but if my model becomes more involved, a manual computation of the gradient is going to be a bottleneck in productivity)
---
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [June 24, 2019, 12:22pm UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574/5 "2019-06-24T12:22:46Z")
New issue, i cannot manage to make it train the model…
MWE:
```julia
using Flux,LinearAlgebra
function MWE()
η=0.01
model=Conv((64, 64), 1=>1, relu,stride=30)
loss(x,y) = Flux.mse(model(x),y)
dataset=[(rand(100,100,1,1),rand(2,2,1,1))]
@show model(dataset[1][1])
@show loss(dataset[1]...)
for i in 1:10
Flux.train!(loss,params(model), dataset, ADAM(η))
@show i, loss(dataset[1]...)
end
end
```
```
MWE (generic function with 1 method)
```
```julia
MWE()
```
```
model((dataset[1])[1]) = [2.74843 1.46427; 0.0 1.95925] (tracked)
loss(dataset[1]...) = 2.7275042760204453 (tracked)
(i, loss(dataset[1]...)) = (1, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (2, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (3, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (4, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (5, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (6, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (7, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (8, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (9, 0.2400080973923382 (tracked))
(i, loss(dataset[1]...)) = (10, 0.2400080973923382 (tracked))
```
```julia
```
---
### Author:  [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)
#### Post date: [June 24, 2019, 12:38pm UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574/6 "2019-06-24T12:38:39Z")
> [@francesco.alemanno](#):
>
> my model will probably get a lot more complicated, thus i’d rather exploit AD than writing the gradient myself
You may be able to just encapsulate the particular calculation in a function, calculate the derivative, and integrate it into the AD system you wish to use. I have done this for simple functions in `ForwardDiff`, for a case which worked OK but was much faster with the [adjoint method](https://math.mit.edu/~stevenj/18.336/adjoint.pdf).
> [@francesco.alemanno](#):
>
> every AD system that i tried fails on this for different reasons
If you want to get help on this, it may help to provide an MWE for the particular AD system you want to use. You did not provide the dimensions, but I imagine that for typical sizes, forward mode would make sense.
---
### Author:  [@francesco.alemanno](https://discourse.julialang.org/u/francesco.alemanno)
#### Post date: [June 24, 2019, 1:55pm UTC](https://discourse.julialang.org/t/autodiff-on-image-convolution/25574/7 "2019-06-24T13:55:15Z")
I would already be happy to have it work with Flux predefined objects, anyway here is what happened with “imfilter” approach
```julia
using Images
```
```julia
K=rand(11,11)
img=randn(100,100)
img2=randn(100,100)
m(x,ker)=imfilter(x,centered(ker))
loss(x,y,ker)=norm(y.-m(x,ker))
```
```
loss (generic function with 1 method)
```
```julia
loss(img,img2,K)
```
```
759.0782976571238
```
```julia
import AutoGrad
AutoGrad.grad(x->loss(img,img2,x),1)(K)
```
```
MethodError: no method matching centered(::Param{Array{Float64,2}})
Closest candidates are:
centered(!Matched::AxisArray) at /home/aizen/.julia/packages/ImageFiltering/8fmJ4/src/ImageFiltering.jl:95
centered(!Matched::ImageMeta) at /home/aizen/.julia/packages/ImageFiltering/8fmJ4/src/ImageFiltering.jl:98
centered(!Matched::AbstractArray) at /home/aizen/.julia/packages/ImageFiltering/8fmJ4/src/utils.jl:15
```
```julia
import ForwardDiff
ForwardDiff.gradient(x->loss(img,img2,x),K)
```
```
MethodError: no method matching svd!(::Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(Main, Symbol("##14#15")),Float64},Float64,11},2}; full=false)
Closest candidates are:
svd!(!Matched::LinearAlgebra.AbstractTriangular) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/triangular.jl:2471 got unsupported keyword argument "full"
svd!(!Matched::Union{DenseArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2}, ReinterpretArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray}, ReshapedArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray}, SubArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,I,L} where L where I<:Tuple{Vararg{Union{Int64, AbstractRange{Int64}, AbstractCartesianIndex},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, DenseArray}}; full) where T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64} at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/svd.jl:59
svd!(!Matched::Union{DenseArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2}, ReinterpretArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray}, ReshapedArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray}, SubArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,I,L} where L where I<:Tuple{Vararg{Union{Int64, AbstractRange{Int64}, AbstractCartesianIndex},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, DenseArray}}, !Matched::Union{DenseArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2}, ReinterpretArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray}, ReshapedArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray}, SubArray{T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64},2,A,I,L} where L where I<:Tuple{Vararg{Union{Int64, AbstractRange{Int64}, AbstractCartesianIndex},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, ReshapedArray{T,N,A,MI} where MI<:Tuple{Vararg{SignedMultiplicativeInverse{Int64},N} where N} where A<:Union{ReinterpretArray{T,N,S,A} where S where A<:Union{SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, SubArray{T,N,A,I,true} where I<:Union{Tuple{Vararg{Real,N} where N}, Tuple{AbstractUnitRange,Vararg{Any,N} where N}} where A<:DenseArray where N where T, DenseArray} where N where T, DenseArray}}) where T<:Union{Complex{Float32}, Complex{Float64}, Float32, Float64} at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.1/LinearAlgebra/src/svd.jl:274 got unsupported keyword argument "full"
```
```julia
import Zygote
Zygote.gradient(x->loss(img,img2,x),K)
```
```
Compiling Tuple{typeof(imfilter!),Array{Float64,2},Array{Float64,2},Tuple{OffsetArrays.OffsetArray{Int64,2,Array{Int64,2}},OffsetArrays.OffsetArray{Float64,2,Array{Float64,2}}},Pad{0},ImageFiltering.Algorithm.FFT}: try/catch is not supported.
```
```julia
import Flux
Flux.Tracker.gradient(x->loss(img,img2,x),K)
```
```
MethodError: no method matching Float64(::Tracker.TrackedReal{Float64})
Closest candidates are:
Float64(::Real, !Matched::RoundingMode) where T<:AbstractFloat at rounding.jl:194
Float64(::T<:Number) where T<:Number at boot.jl:741
Float64(!Matched::Int8) at float.jl:60
...
```