Best practices regarding Adjoint, when should it be used instead of Array, best way to 'style it out'

New to Julia
3

I don’t know your specific use case but I find that for most functions I wrote in the pre Adjoint days that took arrays as arguments (i.e. f(x::Array)) work just fine with adjoints. Just need to change f(x::Array) to f(x::AbstractArray) and things mostly just work. An important exception is if you’re passing the array outside of julia into some c or fortran linear algebra routine that requires a contiguous array. But I would argue that in this case you should hold onto the adjoint as long as possible and just convert to an Array before passing it outside of julia.

Depending on your use case, working with adjoints instead of converting them to new Arrays could save significant allocations. It’s not just about doing linear algebra rigorously. Here’s a simple example illustrating this

julia> using BenchmarkTools

julia> f1(x) = x'
f1 (generic function with 1 method)

julia> f2(x) = Array(x')
f2 (generic function with 1 method)

julia> a = rand(5000,5000);

julia> @benchmark f1($a)
BenchmarkTools.Trial: 
  memory estimate:  16 bytes
  allocs estimate:  1
  --------------
  minimum time:     4.055 ns (0.00% GC)
  median time:      8.516 ns (0.00% GC)
  mean time:        8.439 ns (7.66% GC)
  maximum time:     6.475 μs (99.86% GC)
  --------------
  samples:          10000
  evals/sample:     1000

julia> @benchmark f2($a)
BenchmarkTools.Trial: 
  memory estimate:  190.73 MiB
  allocs estimate:  3
  --------------
  minimum time:     212.697 ms (0.00% GC)
  median time:      215.247 ms (0.05% GC)
  mean time:        222.924 ms (3.94% GC)
  maximum time:     276.981 ms (22.93% GC)
  --------------
  samples:          23
  evals/sample:     1

The f1 function returns an Adjoint, which is just a wrapper around the original array. No need to copy the contents into a new array. The second version has to allocate a new array to store the transposed matrix.

2 Likes