# Why is SUM always used in differentiation?
**URL:** https://discourse.julialang.org/t/why-is-sum-always-used-in-differentiation/57559
**Category:** Numerics
**Tags:** differentiation
**Created:** [March 19, 2021, 5:41pm UTC](https://discourse.julialang.org/t/why-is-sum-always-used-in-differentiation/57559 "2021-03-19T17:41:35Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Sunny](https://discourse.julialang.org/u/Sunny)
#### Post date: [March 19, 2021, 5:41pm UTC](https://discourse.julialang.org/t/why-is-sum-always-used-in-differentiation/57559/1 "2021-03-19T17:41:35Z")
I’ve seen a lot of examples where the output of a NN or any function during the differentiation is preprocessed with `sum` operator. For example this:
```julia
julia> hessian(x -> sum(x.^3), [1 2; 3 4]) # uses linear indexing of x
4×4 Array{$Int,2}:
6 0 0 0
0 18 0 0
0 0 12 0
0 0 0 24
```
Is this a julia thing or there some math behind it?
---
### Author:  [@mcabbott](https://discourse.julialang.org/u/mcabbott)
#### Post date: [March 19, 2021, 5:50pm UTC](https://discourse.julialang.org/t/why-is-sum-always-used-in-differentiation/57559/2 "2021-03-19T17:50:04Z")
It’s just a short way to make a function which returns a scalar. You will get an error with functions which return an array:
```julia
julia> gradient(x -> x.^3, [1 2; 3 4])
ERROR: output an array, so the gradient is not defined. Perhaps you wanted jacobian.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:33
julia> jacobian(x -> x.^3, [1 2; 3 4])[1]
4×4 Matrix{Int64}:
3 0 0 0
0 27 0 0
0 0 12 0
0 0 0 48
julia> gradient(x -> sum(x.^3), [1 2; 3 4])[1]
2×2 Matrix{Int64}:
3 12
27 48
```
---
### Author:  [@Sunny](https://discourse.julialang.org/u/Sunny)
#### Post date: [March 19, 2021, 6:09pm UTC](https://discourse.julialang.org/t/why-is-sum-always-used-in-differentiation/57559/3 "2021-03-19T18:09:23Z")
Great answer, thank you!
---
### Author:  [@apo383](https://discourse.julialang.org/u/apo383)
#### Post date: [March 19, 2021, 6:54pm UTC](https://discourse.julialang.org/t/why-is-sum-always-used-in-differentiation/57559/4 "2021-03-19T18:54:55Z")
To expand a bit, a neural network or other optimization usually needs to be trained to minimize a scalar objective function. So you usually want to define the problem with a scalar in the first place. In fact, the term `gradient` usually refers to the partial derivative of a scalar function with respect to one or more variables, and `jacobian` (in @mcabbott’s example) to the partial derivative of a vector function. There are plenty of uses for Jacobians, but gradients are more common in optimization, which is one of the most popular uses for AD.
I consider `sum` to be more than a way to get Julia not to error, but a key part of defining a sensible problem to be solved.