Should Rationals be explicitly converted?

General Usage
, , ,
1

The question is basically, which is “better” in terms of intermediate type-stability and/or performance:

function half_promote(x :: T) :: T where T <: AbstractFloat
    half = 1//2
    return half * x
end

or

function half_convert(x :: T) :: T where T <: AbstractFloat
    half = oftype(x, 1//2)
    return half * x
end

?

For type-stability I look at the @code_warntype (for which the man-page is good but doesn’t seem to answer my question) below, but I can’t seem to extract a meaningful difference other than the explicit conversion in the latter.

julia> @code_warntype half_promote(2.0)
MethodInstance for half_promote(::Float64)
  from half_promote(x::T) where T<:AbstractFloat @ Main REPL[6]:1
Static Parameters
  T = Float64
Arguments
  #self#::Core.Const(Main.half_promote)
  x::Float64
Locals
  half::Rational{Int64}
  @_4::Float64
Body::Float64
1 ─ %1  = $(Expr(:static_parameter, 1))::Core.Const(Float64)
│         (half = 1 // 2)
│   %3  = half::Core.Const(1//2)
│   %4  = (%3 * x)::Float64
│         (@_4 = %4)
│   %6  = @_4::Float64
│   %7  = (%6 isa %1)::Core.Const(true)
└──       goto #3 if not %7
2 ─       goto #4
3 ─       Core.Const(:(@_4))
│         Core.Const(:(Base.convert(%1, %10)))
└──       Core.Const(:(@_4 = Core.typeassert(%11, %1)))
4 ┄ %13 = @_4::Float64
└──       return %13

and

julia> @code_warntype half_convert(2.0)
MethodInstance for half_convert(::Float64)
  from half_convert(x::T) where T<:AbstractFloat @ Main REPL[7]:1
Static Parameters
  T = Float64
Arguments
  #self#::Core.Const(Main.half_convert)
  x::Float64
Locals
  half::Float64
  @_4::Float64
Body::Float64
1 ─ %1  = $(Expr(:static_parameter, 1))::Core.Const(Float64)
│   %2  = Main.oftype::Core.Const(oftype)
│   %3  = (1 // 2)::Core.Const(1//2)
│         (half = (%2)(x, %3))
│   %5  = half::Core.Const(0.5)
│   %6  = (%5 * x)::Float64
│         (@_4 = %6)
│   %8  = @_4::Float64
│   %9  = (%8 isa %1)::Core.Const(true)
└──       goto #3 if not %9
2 ─       goto #4
3 ─       Core.Const(:(@_4))
│         Core.Const(:(Base.convert(%1, %12)))
└──       Core.Const(:(@_4 = Core.typeassert(%13, %1)))
4 ┄ %15 = @_4::Float64
└──       return %15

Thankfully, the LLVM lowering is a little more concise and provides nearly identical results with the only difference being in the comments.

julia> @code_llvm half_promote(2.0)
; Function Signature: half_promote(Float64)
;  @ REPL[6]:1 within `half_promote`
define double @julia_half_promote_10502(double %"x::Float64") #0 {
top:
;  @ REPL[6]:3 within `half_promote`
; ┌ @ promotion.jl:430 within `*` @ float.jl:493
   %0 = fmul double %"x::Float64", 5.000000e-01
; └
  ret double %0
}

and

julia> @code_llvm half_convert(2.0)
; Function Signature: half_convert(Float64)
;  @ REPL[7]:1 within `half_convert`
define double @julia_half_convert_10519(double %"x::Float64") #0 {
top:
;  @ REPL[7]:3 within `half_convert`
; ┌ @ float.jl:493 within `*`
   %0 = fmul double %"x::Float64", 5.000000e-01
; └
  ret double %0
}

So does the identical IR actually mean there is literally no difference between these functions?

2

There should be no difference in terms of type stability or in terms of performance between the two snippets. Don’t see anything special about Rational here, either.

Some suggestions:

  • Don’t do convert or oftype unnecessarily, let promotion (promote) take care of things.

  • Don’t use method return type annotations.

  • Don’t use unnecessary method static parameters. In your example, instead of (x :: T) :: T where T <: AbstractFloat, you could’ve just written (x::AbstractFloat)

1 Like
3

Could you elaborate? On any of your points?

I use Rational because I want to be able to make sure that my numbers stay the same type they start with because that’s the type that’s going to be returned anyway, so any promotion is effectively wasted. e.g. I don’t want to use 0.5 because if x is Float32, the result would first be promoted to a Float64 before being “demoted” to Float32 as instructed by the method type annotation. The terminal behavior doesn’t result in type-instability sure, but again, there would otherwise be needless promotion and demotion.

4

This is such a small toy example it’s hard to say what’s “best”. But generally, the simplest, most obvious computation is the right one. In this case, that’s almost surely x/2.

It’s really hard to extrapolate out to what your real world use-case might be.

3 Likes
5

I wouldn’t convert rationals in general, unless you know what you’re doing and want more performance. It’s easier to see with half → third:

function third_promote(x :: T) :: T where T <: AbstractFloat
    third = 1//3
    return third * x
end

function third_convert(x :: T) :: T where T <: AbstractFloat
    third = oftype(x, 1//3)
    return third * x
end

julia> Float32(1//3) # less accurate, even less with bfloat or FP8 or FP4
0.33333334f0

julia> Float64(1//3)
0.3333333333333333

If you convert eagerly you’re locked into the lower accuracy, and higher performance. Here it doesn’t matter since there’s nothing in between, but if you e.g. do third * some_other_var_also_rational, then you keep in the perfectly accurate rational slower domain for longer.

0.5 can promote yes (and it will never demote e.g. not from BigFloat, I checked), but that needs not be a bad thing. Yes 0.5f0 is as accurate on its own (even using it, demoting temporary may be ok, if you know what you’re doing), but if I’m not incorrect, it can help accuracy promoting temporary. On CPUs I believe Float64 is as fast, that would not be the case on GPUs…

Yes, when the denominator is a power of 2 integer, you get what you want, converted to a multiply, not the slower division (@code_native did NOT confirm that, since I believe in global scope). Note, it doesn’t happen with e.g.x/3.

1 Like
6

Bad news here, you can only do this much. * for Float16, Float32, Float64 forwards to the intrinsic Base.mul_float, and that starts deviating from your specified type down to the hardware.

julia> @code_llvm half_promote(Float16(3.5))
; Function Signature: half_promote(Float16)
;  @ REPL[5]:1 within `half_promote`
; Function Attrs: uwtable
define half @julia_half_promote_4499(half %"x::Float16") #0 {
top:
;  @ REPL[5]:3 within `half_promote`
; ┌ @ promotion.jl:430 within `*` @ float.jl:493
   %0 = fpext half %"x::Float16" to float
   %1 = fmul float %0, 5.000000e-01
   %2 = fptrunc float %1 to half
; └
  ret half %2
}

julia> @code_llvm half_convert(Float16(3.5))
; Function Signature: half_convert(Float16)
;  @ REPL[14]:1 within `half_convert`
; Function Attrs: uwtable
define half @julia_half_convert_4503(half %"x::Float16") #0 {
top:
;  @ REPL[14]:3 within `half_convert`
; ┌ @ float.jl:493 within `*`
   %0 = fpext half %"x::Float16" to float
   %1 = fmul float %0, 5.000000e-01
   %2 = fptrunc float %1 to half
; └
  ret half %2
}

So whether you convert strictly to the input type or give promotion a chance to do it, you end up converting Float16 (half) to Float32 (float) for the operation, then truncating back to Float16. If you’re not too bothered by the compiler doing as it pleases, then I’d prefer the semantics of explicit conversion; there’s no guarantee that promotion will favor an arbitrary AbstractFloat over Rational{Int}.

An intermediate operation in a higher precision is a widespread technique to reduce approximation errors, if that’s of interest.

1 Like
7

For Float64 input there is no difference. Any conversion in the promotion of *, and oftype is inlined by the compiler, and results in the same code being generated, and none of these operations are actually performed at runtime. I think this will be so for all the built in floats. So, no performance difference. And no type stability issues.

The problem with generic programming is what happens with floats which are not yet defined. E.g. if someone decides to create a float which behaves differently. There is no guarantee that * will promote to the highest precision of the arguments for not yet existing floats.

And, also, what happens in more complicated examples? One thing is to return the same type as the input, but there is also a question of at what precision should intermediate calculations be performed. This depends on your application. You might want to do all calculations in the input precision, or you might want to do them in the highest native precision (typically Float64 or Float32), and convert to the input precision upon return (which is what your declaration function (...)::T does).

In general, arithmetic seldom leads to type instability.

1 Like
8

Thank you all for your input. There definitely seems to be some good details when considering how to manage intermediate precision! The reason why I ask at all is, as @Palli guessed, the actual implementation is used in a GPU kernel (CUDA.jl), so keeping things as Float32s is preferable in addition to keeping track of allocations.

2 Likes
9

TBH, none of this discussion is relevant to any of that. I’ll echo the suggestions by Matt Bauman from above:

What I’m saying is: you gave us an example that was reduced so far that we couldn’t give you any relevant input on your actual issues behind the example.

1 Like
10

Disregard the earlier @code_llvm reports then check @device_code_llvm because a compiler for the GPU (which includes overriding some function calls for CUDA C intrinsics) won’t behave like a compiler for the CPU. I think CUDA supports direct Float16 (half-precision) operations but I’m not certain.

1 Like
11

The whole point of promotion is to enable generic code with heterogeneous types. Promotion will always be type stable (unless someone is doing something wild). But, yes, it might lead to a different type.

The problem with the micro example here is that we can’t see what generic behaviors you might want. In fact, it seems you don’t want generic behaviors at all. You just want to support Float32; that’s fine. And you can depend upon ::Float32 / ::Int giving you back a Float32. That’s how it works.

Then the only question left is if you want to divide by an integer or multiply by a reciprocal. This isn’t really a Julia question as it is about the numerics themselves. LLVM will automatically do the latter if it’s a bitwise exact interchange — it can be much faster. But if the reciprocal isn’t exactly represented, then it’s less accurate. Compilers have a flag (LLVM’s is arcp) that enables this less-accurate-but-faster tradeoff (and it’s part of fastmath). Which is “best” again depends on what you want the thing to do.

julia> code_llvm(x->x/2, (Float32,))
; Function Signature: var"#69"(Float32)
;  @ REPL[98]:1 within `#69`
define float @"julia_#69_10590"(float %"x::Float32") #0 {
top:
; ┌ @ promotion.jl:432 within `/` @ float.jl:494
   %0 = fmul float %"x::Float32", 5.000000e-01
   ret float %0
; └
}

julia> code_llvm(x->x/3, (Float32,))
; Function Signature: var"#71"(Float32)
;  @ REPL[99]:1 within `#71`
define float @"julia_#71_10593"(float %"x::Float32") #0 {
top:
; ┌ @ promotion.jl:432 within `/` @ float.jl:494
   %0 = fdiv float %"x::Float32", 3.000000e+00
   ret float %0
; └
}

julia> code_llvm(x->@fastmath(x/3), (Float32,))
; Function Signature: var"#73"(Float32)
;  @ REPL[100]:1 within `#73`
define float @"julia_#73_10596"(float %"x::Float32") #0 {
top:
; ┌ @ fastmath.jl:266 within `div_fast` @ fastmath.jl:166
   %0 = fmul fast float %"x::Float32", 0x3FD5555560000000
   ret float %0
; └
}
2 Likes
12

Yes that was the point. It wasn’t my intention to get a solution to a specific real-world problem, but rather to have a discussion on general ideas and solicit expert insights and experiences into managing Rationals and intermediate type-conversion; I apologize if that wasn’t clear. The admittedly extreme-toy example I provided was meant merely to make my thought clearer.