# Fast Hessian size increase

**URL:** <https://discourse.julialang.org/t/fast-hessian-size-increase/57560>\
**Category:** Numerics\
**Tags:** differentiation, sparse\
**Created:** [March 19, 2021, 6:04pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560 "2021-03-19T18:04:37Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sunny](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Post date:** [March 19, 2021, 6:04pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560/1 "2021-03-19T18:04:37Z")

</div>

I’d like to compute the Hessian for array values and to get  
some array with dimension (2,2) but instead I got this

```julia
julia> hessian(x -> sum(x.^3), [1 2; 3 4]) # uses linear indexing of x
4×4 Array{$Int,2}:
 6 0 0 0
 0 18 0 0
 0 0 12 0
 0 0 0 24

```

If I use the 2 x 10 array then I already have 20 x 20 matrix! Can I put the result in the other dimension?  
Because my neural network will be very slow as I use hessian in the loss function

---

<div class="post-metadata">

**Author:** ![stevengj](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stevengj/32/71_2.png) [@stevengj](https://discourse.julialang.org/u/stevengj)\
**Post date:** [March 19, 2021, 6:14pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560/2 "2021-03-19T18:14:14Z")

</div>

> [@Sunny](#):
>
> If I use the 2 x 10 array then I already have 20 x 20 matrix!

You have 20 inputs, so the Hessian is 20x20 by definition. If you want a 2x2 matrix, you are thinking of something other than a Hessian.

---

<div class="post-metadata">

**Author:** ![lferrant](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lferrant/32/22751_2.png) [@lferrant](https://discourse.julialang.org/u/lferrant)\
**Post date:** [March 19, 2021, 6:14pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560/3 "2021-03-19T18:14:19Z")

</div>

The result you got is correct. `sum` computes the sum of all elements, so your function `x -> sum(x.^3)` is indeed defined as \mathbb{R}^4\rightarrow\mathbb{R} and hence the hessian is a 4\times 4 matrix. If you feed to your function a 2\times 10 matrix, the function is \mathbb{R}^{20}\rightarrow\mathbb{R} and hence the hessian will be 20\times 20. In general if your function is \mathbb{R}^n\rightarrow\mathbb{R}, the hessian will be a n\times n matrix.

---

<div class="post-metadata">

**Author:** ![Sunny](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Post date:** [March 19, 2021, 6:17pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560/4 "2021-03-19T18:17:11Z")

</div>

Yes, sorry.  
I mean that I’d like to have the result with three dimensions (2,2,2) for 2x2 array and  
the hessian with shape (2,2,10) for (2,10) array

---

<div class="post-metadata">

**Author:** ![lferrant](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lferrant/32/22751_2.png) [@lferrant](https://discourse.julialang.org/u/lferrant)\
**Post date:** [March 19, 2021, 6:23pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560/5 "2021-03-19T18:23:26Z")

</div>

The hessian is by definition a n\times n matrix. Are you sure you don’t mean something else? 🙂

> **[Hessian matrix](https://en.wikipedia.org/wiki/Hessian_matrix)**
>
> In mathematics, the Hessian matrix, Hessian or (less commonly) Hesse matrix is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field. It describes the local curvature of a function of many variables. The Hessian matrix was developed in the 19th century by the German mathematician Ludwig Otto Hesse and later named after him. Hesse originally used the term "functional determinants".
> Suppose 
>   
>     
>       
> f
> :
>         
>           
> ...

---

<div class="post-metadata">

**Author:** ![lferrant](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lferrant/32/22751_2.png) [@lferrant](https://discourse.julialang.org/u/lferrant)\
**Post date:** [March 19, 2021, 6:39pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560/6 "2021-03-19T18:39:18Z")

</div>

> [@Sunny](#):
>
> I mean that I’d like to have the result with three dimensions (2,2,2) for 2x2 array and  
> the hessian with shape (2,2,10) for (2,10) array

This makes me think you actually have a \mathbb{R}^2\rightarrow\mathbb{R} function and you want to evaluate the hessian at n distinct points (e.g if your array is (2,10) at 10 points) is this what you want to do?

A possible way I can think of is the following (there might be better ones)

```julia
julia> f(x) = sum(x.^3)
f (generic function with 1 methods)

julia> pnts = [[1,2], [3,4], [5,6], [7,8]]
4-element Vector{Vector{Int64}}:
 [1, 2]
 [3, 4]
 [5, 6]
 [7, 8]

julia> ForwardDiff.hessian.(f, pnts)
4-element Vector{Matrix{Int64}}:
 [6 0; 0 12]
 [18 0; 0 24]
 [30 0; 0 36]
 [42 0; 0 48]

```

---

<div class="post-metadata">

**Author:** ![Sunny](https://avatars.discourse-cdn.com/v4/letter/s/7c8e57/32.png) [@Sunny](https://discourse.julialang.org/u/Sunny)\
**Post date:** [March 19, 2021, 6:58pm UTC](https://discourse.julialang.org/t/fast-hessian-size-increase/57560/7 "2021-03-19T18:58:15Z")

</div>

Yes, that’s exactly what I wanted! Thanks