Broadcasting power with integer literal issues

New to Julia
1

Hi, I am having some issues with custom broadcasting working with the power (^) operator when I am trying to use an integer literal as the exponent. My broadcast methods seem to work fine for other methods such as + or * or - with integer literals.

The exponent works with literal floats, or explicit integers, but when using an integer float it seems to be trying to broadcast with a Base.RefValue{Val{2}} instead of just an integer, which doesn’t happen with the other methods. I’m not sure why this is happening, I am sure there are workarounds I can do (such as just using float literals) but I’d like to be able to just do .^2 on my struct as expected.

I have uploaded a full reproducible file on gist, and below is a snippet of the relevant code and the stacktrace.

function Base.broadcasted(::VariableStyle, ::typeof(+), a::Variable, b::Union{AbstractArray,Number})
    Variable(a.data .+ b)
end
println(Variable([1 2 3]) .+ 5) # Works

function Base.broadcasted(::VariableStyle, ::typeof(^), a::Variable, b::Number)
    Variable(a.data .^ b)
end

println("1 ", Variable([1 2 3]) .^ 2.) # Works
println("2 ", Variable([1 2 3]) .^ convert(Int, 2)) # Works
two = 2
println("3 ", Variable([1 2 3]) .^ two) # Works
println("4 ", [1 2 3] .^ 2) # Works
println("5 ", Variable([1 2 3]) .^ 2) # Doesn't work

This prints up until print("5.... upon which it throws an error
ERROR: LoadError: MethodError: no method matching size(::VariableArray)

Closest candidates are:
  size(::Union{LinearAlgebra.Adjoint{T, var"#s972"}, LinearAlgebra.Transpose{T, var"#s972"}} where {T, var"#s972"<:(AbstractVector)})
   @ LinearAlgebra ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/LinearAlgebra/src/adjtrans.jl:296
  size(::Union{LinearAlgebra.Adjoint{T, var"#s972"}, LinearAlgebra.Transpose{T, var"#s972"}} where {T, var"#s972"<:(AbstractMatrix)})
   @ LinearAlgebra ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/LinearAlgebra/src/adjtrans.jl:297
  size(::Union{LinearAlgebra.QR, LinearAlgebra.QRCompactWY, LinearAlgebra.QRPivoted})
   @ LinearAlgebra ~/.julia/juliaup/julia-1.9.3+0.x64.linux.gnu/share/julia/stdlib/v1.9/LinearAlgebra/src/qr.jl:582
  ...

Stacktrace:
 [1] axes
   @ ./abstractarray.jl:98 [inlined]
 [2] combine_axes
   @ ./broadcast.jl:512 [inlined]
 [3] combine_axes
   @ ./broadcast.jl:511 [inlined]
 [4] instantiate
   @ ./broadcast.jl:294 [inlined]
 [5] materialize(bc::Base.Broadcast.Broadcasted{VariableStyle, Nothing, typeof(Base.literal_pow), Tuple{Base.RefValue{typeof(^)}, VariableArray, Base.RefValue{Val{2}}}})
   @ Base.Broadcast ./broadcast.jl:873
 [6] top-level scope
   @ ~/code/Bijoulia/reproducable.jl:52
in expression starting at /home/ybbat/code/Bijoulia/reproducable.jl:52

(I am aware that this error is caused by not having a size method, which I did not include in the reproducible file, but it is stacktrace [5] I am concerned about because this should not be what is being called)

Throwing a random error within the + broadcast method reveals a much different stacktrace, wherein broadcasted is called as expected with the function, VariableArray and Int64.

2

Welcome to the forum :slight_smile:

Cthulhu.jl is a great for this type of debugging.

Even trying this is helpful:

julia> Cthulhu.@descend Variable([1 2 3]) .^ two
ERROR: dot expressions are not lowered to a single function call, so @descend_code_typed cannot analyze them. You may want to use Meta.@lower to identify which function call to target.

where it points one to compare

julia> Meta.@lower Variable([1 2 3]) .^ two
:($(Expr(:thunk, CodeInfo(
    @ none within `top-level scope`
1 ─ %1 = ^
│   %2 = Base.hcat(1, 2, 3)
│   %3 = Variable(%2)
│   %4 = Base.broadcasted(%1, %3, two)
│   %5 = Base.materialize(%4)
└──      return %5
))))

to

julia> Meta.@lower Variable([1 2 3]) .^ 2
:($(Expr(:thunk, CodeInfo(
    @ none within `top-level scope`
1 ─ %1 = ^
│   %2 = Base.hcat(1, 2, 3)
│   %3 = Variable(%2)
│   %4 = Core.apply_type(Base.Val, 2)
│   %5 = (%4)()
│   %6 = Base.broadcasted(Base.literal_pow, %1, %3, %5)
│   %7 = Base.materialize(%6)
└──      return %7
))))

where you can see that Base.literal_pow figures in the call you’re asking about.

To see this, a hacky way of using the different code path is doing

Variable([1 2 3]) .^ (0 + 2)

but I don’t have a good handle on the necessary method definition you need to make that’s consistent with your type. Something like

Base.broadcasted(::typeof(Base.literal_pow), ^, x::Variable, ::Val{p}) where {p} = Base.broadcasted(VariableStyle(), ^, x, p)
2 Likes
3

Thanks for the advice and the welcome! I should have thought of looking at the lowered form of the expression, great suggestion.

I figured out a similar solution before I saw your latest edit:

function Base.broadcasted(::VariableStyle, ::typeof(Base.literal_pow), op::typeof(^), a::Variable, b::Base.RefValue)
    @show result = broadcast(Base.literal_pow, op, a.data, b)
    Variable(result)
end

Though this broadcasts with literal_pow over the array instead of “de-valueing” the exponent; I’m not sure which approach is “correct” or better, I don’t know if there are any significant differences in a case like this. Also just want to note for anyone else who has a similar issue that due to broadcasting the exponent is a a RefValue{Val} rather than just a Val.

I used a small function deval(::Base.RefValue{Base.Val{n}}) where n = n to get the real value back in case I needed it for something else (I do).

Thanks again for the help! This problem was really bugging me despite the minor end impact.