# Gradient of a normal (NOT vector) function that has vector arguments

**URL:** <https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621>\
**Category:** General Usage\
**Created:** [July 22, 2019, 7:38am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621 "2019-07-22T07:38:02Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ujku\_KU](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ujku_ku/32/13159_2.png) [@Ujku\_KU](https://discourse.julialang.org/u/Ujku_KU)\
**Post date:** [July 22, 2019, 7:38am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/1 "2019-07-22T07:38:02Z")

</div>

Hi guys,

I am trying to implement a code in Julia (it’s my first in Julia) and I am having problems in computing the gradient of a function with vector arguments.

I would like to compute the derivative in relation with only one of those vector arguments of my function.

If you need further details of my code (or pseudocode) please let me know and I will be happy to provide them for you.

I have to let you know also that I have already tried with ForwardDiff but without success.

I thank you in advance for your time and support.

Cheers.

Ergnoor

---

<div class="post-metadata">

**Author:** ![nilshg](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/nilshg/32/2283_2.png) [@nilshg](https://discourse.julialang.org/u/nilshg)\
**Post date:** [July 22, 2019, 7:56am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/2 "2019-07-22T07:56:58Z")

</div>

Please refer to [this topic](https://discourse.julialang.org/t/psa-make-it-easier-to-help-you/14757) and post an MWE - I don’t think your question can be answered at present without additional information.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [July 22, 2019, 10:57am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/3 "2019-07-22T10:57:38Z")

</div>

Just use [`ForwardDiff.gradient`](http://www.juliadiff.org/ForwardDiff.jl/stable/user/api.html#Gradients-of-f(x::AbstractArray)::Real-1):

```julia
julia> using ForwardDiff

julia> f(x) = sum(x)
f (generic function with 1 method)

julia> ForwardDiff.gradient(f, [1., 2.])
2-element Array{Float64,1}:
 1.0
 1.0

```

Edit: hadn’t noticed

> [@Ujku\_KU](#):
>
> in relation with only one of those vector arguments of my function

You can use a closure:

```julia
julia> using ForwardDiff

julia> f(x, y) = sum(x .* y)
f (generic function with 1 method)

julia> ForwardDiff.gradient(x -> f(x, [3, 4]), [1., 2.])
2-element Array{Float64,1}:
 1.0
 1.0

```

---

<div class="post-metadata">

**Author:** ![Ujku\_KU](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ujku_ku/32/13159_2.png) [@Ujku\_KU](https://discourse.julialang.org/u/Ujku_KU)\
**Post date:** [July 23, 2019, 7:55am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/4 "2019-07-23T07:55:28Z")

</div>

@nilshg Hi nilshg, and thank you for your advice. I was trying to create a minimal working code that represent generally what I would like to realize with my code, but in the mean time “tkoolen” has answered and he is in line with what I was experimenting with. Hence I will carry on where he has left.

---

<div class="post-metadata">

**Author:** ![Ujku\_KU](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ujku_ku/32/13159_2.png) [@Ujku\_KU](https://discourse.julialang.org/u/Ujku_KU)\
**Post date:** [July 23, 2019, 8:09am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/5 "2019-07-23T08:09:21Z")

</div>

@tkoolen Hi tkoolen, and thank you very much for your answer. This part of your answer:

```julia
ForwardDiff.gradient(x -> f(x, [3, 4]), [1, 2])

```

ia what I was very much interested, but when I tried to implement it I got the same error as the one when I implemented it myself as follows:

```julia
julia> using ForwardDiff
julia> f(x, y) = sum(x .* y)
julia> f2(x) = f(x, [3, 4])
julia> ForwardDiff.gradient(f2, [1, 2])

```

It seems to me that is an error that has to do with value of ‘x’ which comes out to be:

```julia

::Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")

```

The “PreImageProblems” is a module of my project.

Cheers.

Ergnoor

---

<div class="post-metadata">

**Author:** ![simeonschaub](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/simeonschaub/32/216566_2.png) [@simeonschaub](https://discourse.julialang.org/u/simeonschaub)\
**Post date:** [July 23, 2019, 8:49am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/6 "2019-07-23T08:49:11Z")

</div>

It will be a lot easier to help you, if you posted the exact error message. Everything works fine on my machine, so I would first try `]up ForwardDiff` to make sure, you are running the latest version. Also make sure you are running a fresh Julia session. If that doesn’t work, the output of `versioninfo()` and `]st ForwardDiff` would be helpful.

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [July 23, 2019, 10:37am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/7 "2019-07-23T10:37:19Z")

</div>

I agree with `simeonschaub`. This code runs fine in a new Julia session. The fact that the tag is `getfield(PreImageProblems, Symbol("##4#5")` means that the code you posted cannot be what produced the error, regardless of ForwardDiff version.

---

<div class="post-metadata">

**Author:** ![Ujku\_KU](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ujku_ku/32/13159_2.png) [@Ujku\_KU](https://discourse.julialang.org/u/Ujku_KU)\
**Post date:** [July 24, 2019, 2:13am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/8 "2019-07-24T02:13:00Z")

</div>

@simeonschaub and @tkoolen Hi guys and thank you very much for your help. Maybe I wasn’t clear enough and I am sorry for that. As you guys affirm the code that tkoolen provided and the code I provided work in a new Julia session. But I am saying that when I try to implement it on my code (when I try to incorporate it in my code) it generates the following error:

```julia

ERROR: LoadError: MethodError: no method matching (::PolynomialKernel{Int64,Int64})(::Array{Float64,1}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2},1})
Closest candidates are:
  PolynomialKernel(::Array{T₃<:Real,1}, ::Array{T₃<:Real,1}) where T₃<:Real at /home/x_user/.julia/dev/PreImageProblems/src/PreImageProblems.jl:28
  PolynomialKernel(::Union{AbstractArray{T₃<:Real,1}, AbstractArray{T₃<:Real,2}}, ::Union{AbstractArray{T₃<:Real,1}, AbstractArray{T₃<:Real,2}}) where T₃<:Real at /home/x_user/.julia/dev/PreImageProblems/src/PreImageProblems.jl:35
Stacktrace:
 [1] cost_function(::KernelPCA{Float64}, ::Type{Denoising}, ::Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2},1}) at /home/x_user/.julia/dev/PreImageProblems/src/pre_image_problem.jl:33
 [2] (::getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)})(::Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2},1}) at /home/x_user/.julia/dev/PreImageProblems/src/pre_image_problem.jl:87
 [3] vector_mode_gradient(::getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)}, ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2,Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2},1}}) at /home/x_user/.julia/packages/ForwardDiff/N0wMF/src/apiutils.jl:37
 [4] gradient(::Function, ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2,Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2},1}}, ::Val{true}) at /home/x_user/.julia/packages/ForwardDiff/N0wMF/src/gradient.jl:17
 [5] gradient(::Function, ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2,Array{ForwardDiff.Dual{ForwardDiff.Tag{getfield(PreImageProblems, Symbol("##4#5")){KernelPCA{Float64},DataType,typeof(PreImageProblems.cost_function)},Float64},Float64,2},1}}) at /home/x_user/.julia/packages/ForwardDiff/N0wMF/src/gradient.jl:15 (repeats 2 times)
 [6] #update_step#3(::Float64, ::Float64, ::Function, ::KernelPCA{Float64}, ::Type{WithRespectToData}, ::Type{Denoising}, ::Array{Float64,1}, ::Int64, ::typeof(PreImageProblems.cost_function)) at /home/x_user/.julia/dev/PreImageProblems/src/pre_image_problem.jl:87
 [7] update_step at /home/x_user/.julia/dev/PreImageProblems/src/pre_image_problem.jl:71 [inlined]
 [8] pre_image_non_neg_constraint(::Type{KernelPCA}, ::PolynomialKernel{Int64,Int64}, ::Type{Denoising}, ::Type{WithRespectToData}, ::Array{Float64,2}, ::Int64) at /home/x_user/.julia/dev/PreImageProblems/src/pre_image_problem.jl:50
 [9] top-level scope at none:0
in expression starting at /home/x_user/.julia/dev/PreImageProblems/src/polyDataDenoisExp.jl:76

```

I believe that the version of Julia and other packages is updated, but anyway please find following the result of version checking:

```julia

julia> versioninfo()
Julia Version 1.1.0
Commit 80516ca202 (2019-01-21 21:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_EDITOR = atom -a
  JULIA_NUM_THREADS = 6

```

If you guys have the nerve to investigate further this story I will be happy to provide you details in order to proceed further with a minimal working code.

Again I thank you very much for your support.

Cheers.

Ergnoor

---

<div class="post-metadata">

**Author:** ![tkoolen](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkoolen/32/1603_2.png) [@tkoolen](https://discourse.julialang.org/u/tkoolen)\
**Post date:** [July 24, 2019, 2:21am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/9 "2019-07-24T02:21:32Z")

</div>

It looks like you designed the `PolynomialKernel` constructor such that the element type of the first and second argument must be the same, namely `T₃`. But when you use `ForwardDiff`, the constructor is called with an array with element type `ForwardDiff.Dual` as the second argument. One way to fix this is to allow heterogeneous element types in the `PolynomialKernel` constructor (and possibly the type itself). Another is to promote the first argument to an array of the same element type as the second argument. Be sure to read [http://www.juliadiff.org/ForwardDiff.jl/stable/user/limitations.html](http://www.juliadiff.org/ForwardDiff.jl/stable/user/limitations.html); basically your code is currently not sufficiently generic.

---

<div class="post-metadata">

**Author:** ![Ujku\_KU](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ujku_ku/32/13159_2.png) [@Ujku\_KU](https://discourse.julialang.org/u/Ujku_KU)\
**Post date:** [July 24, 2019, 2:57am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/10 "2019-07-24T02:57:35Z")

</div>

@tkoolen Hi tkoolen and thank you very much for your help. I wasn’t aware of:

```julia

struct Partials{N,V} <: AbstractVector{V}
    values::NTuple{N,V}
end

```

and

```julia

struct Dual{T,V<:Real,N} <: Real
    value::V
    partials::Partials{N,V}
end

```

in `ForwardDiff`.

It seems I have to generalize the `PolynomialKernel` constructor in order to accept as arguments not only `T₃<: Real` but also `Dual`.

I will let you if I am successful in implementing it.

Cheers.

Ergnoor

---

<div class="post-metadata">

**Author:** ![Ujku\_KU](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ujku_ku/32/13159_2.png) [@Ujku\_KU](https://discourse.julialang.org/u/Ujku_KU)\
**Post date:** [July 30, 2019, 1:52am UTC](https://discourse.julialang.org/t/gradient-of-a-normal-not-vector-function-that-has-vector-arguments/26621/11 "2019-07-30T01:52:19Z")

</div>

@tkoolen

Hi tkoolen,

sorry for this delay in contacting you back, but just to confirm that your suggestion worked fine.

Once more thank you very much.

Cheers.

Ergnoor