I found a fix, consider the following MWE
using Symbolics, Flux
@variables x, y
function f(x,y)
A = transpose([x;y])*[x;y]
return A[1]
end
display(f(x,y))
This returns
x^2 + y^2
Then if I add
Flux.gradient(f,x,y)
it returns
(2x, 2y)
Which is correct. Seems I was using incorrect syntax for what I wanted.