Switching off implicit conversion

General Usage
1

Let’s work with an example. Say I have two different units of length.

abstract type Length end

struct Meters <: Length
  value::Float64
end

struct CentiMeters <: Length
  value::Float64
end

I want to use the existing Float64 operations to do my calculations, so for convenience, I define the following conversion rules. Note that we do all calculations at the centimeter scale:

Base.convert(::Type{Float64}, x::Meters) = x.value * 100 # when in Float64-land, always represent length in centimeters
Base.convert(::Type{Float64}, x::CentiMeters) = x.value
Base.convert(::Type{Meters}, x::CentiMeters) = Meters(x.value / 100)
Base.convert(::Type{CentiMeters}, x::Meters) = CentiMeters(x.value * 100)

I want to be able to add two lengths of any unit. So:

Base.:+(a::Length, b::Length) = CentiMeters(convert(Float64, a) + convert(Float64, b))

Calculations are now convenient.

x = CentiMeters(100) + Meters(2) # defaults to CentiMeters
y::Meters = CentiMeters(100) + Meters(2) # but I can get Meters if I want (via implicit conversion and type assertion).
@assert x isa CentiMeters && x.value == 300
@assert y isa Meters && y.value == 3

On person’s convenience can be another person’s confusion:

lengths = Float64[]
push!(lengths, Meters(1))
push!(lengths, CentiMeters(1))
@assert lengths == [100, 1]

But implicit conversion can be problematic.

Implicit conversion, when combined with the dual meaning of ::, can be confusing. see discussion here: Semantics of :: in return type vs. argument type annotations

must_be_meters_1(x)::Meters = x # this converts and asserts
must_be_meters_2(x) = x::Meters # this asserts
@assert must_be_meters_1(CentiMeters(1)) isa Meters
# @assert must_be_meters_2(CentiMeters(1)) isa Meters # fails

Implicit conversion can result in logic bugs, such as this one:

length = CentiMeters(100) |> Meters # I never defined a `Meters(::CentiMeters)` constructor.
@assert length == Meters(100) # 100 centemeters is not the same as 100 meters

In other cases, it can lead to data being copied unintentionally, such as when an array view is implicitly converted to (and hence copied) as an array. See here: Why does this conversion happen implicitly?

A layer of indirection could allow us to “switch off” implicit conversion. We could define:

Base.implicit_convert(args...) = convert(args...)

All places that use conversion implicitly can then call implicit_convert instead of convert.

And for our own types, we could:

Base.implicit_convert(Type{Meters}, x) = x
Base.implicit_convert(t, x::Meters) = x

This would allow each of the following lines of code to error:

x::Meters = CentiMeters(10)
f(x:Meters)::CentiMeters = x
length = CentiMeters(100) |> Meters

This would be:

  • non breaking
  • allow only certain, important types to be “switched off” while maintaining the convenience of implicit conversion by default
  • allow a much stricter application where implicit conversion is turned off for all types (strict mode feature?)

I think folks who come from statically typed languages would appreciate this option. I certainly would.

2 Likes
2

All of that is breaking.

Implicit conversions were inspired by statically typed languages in the first place. Dynamically typed languages tend to not associate types with variables, fields, or return values at all, so implicit conversions mostly show up as promotions in non-unary operations.

I’d discourage retroactive implicit units for Float64 and these associated converts, which have a more immediate role in the confusion than general implicit conversion. It’s not necessary for operations on mixed units, check how Unitful.jl does 1.0u"cm" + 1.0u"m".

3

implicit conversions mostly show up as promotions in non-unary operations.

and

  • default outer constructors
  • assignment to type-annotated variables
  • return values of functions annotated with a return type
1 Like
4

I mean, that’s just a matter of naming. Base.convert does mean a potentially implicit convert, and that’s not changing. What can be different is the fact that you can create your own explicit_convert function. Or, commonly, folks use the constructor to represent that sort of explicit change-of-type.

since convert can be called implicitly, its methods are restricted to cases that are considered “safe” or “unsurprising”

You can also take inspiration from the unit-like packages like Unitful:

3 Likes
5

In case the use case of physical units matters: The package Unitful.jl may help avoid the confusion.

6

The example with units of lengths was just for demonstration. I am aware of Unitful.jl, though I have never used it. Let’s focus the discussion on implicit conversion in general.

I think the indirection was necessary because convert is used both in impicit and explicit ways. Base has lots of direct calls to convert, and I thought it made sense to preserve the explicitness of it. It’s not a problem when done explicitly.

So the idea really is to distinguish between what is happening implicitly and what is happening implicitly. Hence my suggestion of a different implicit_convert or similar function. But if “Base.convert does mean a potentially implicit convert, and that’s not changing.”, then there’s no discussion to be had.

There are people who would like the language to distinguish between “implicit” and “explicit” conversion. If we allow convert to mean “explicit convert”, it leaves room for an “implicit convert”.

7

My point is that you can (and others do!) accomplish your goals with the opposite approach of either using constructors or an explicit_convert function without any changes. Right now.

Conversion — as that documentation page explains — should really be restricted to “safe” and “unsurprising” cases. And if it’s unsurprising, then it makes sense that it can be implicit. The way to switch it off is by not defining those surprising behaviors as methods of convert.

8

I see. You’re helping me clarify my own thinking here.

I think mostly, my problem is with the :: notation.

x::T in certain contexts means typeassert(x, T). That’s fine.

In other contexts, it meas typeassert(convert(T, x), T). That’s what I would want to switch to typeassert(implicit_convert(T, x), T).

This would allow other (nice?) implicit conversions to remain, such as the ones that allow Meter(1) + CentiMeter(1).

But that doesn’t solve other porblems, such the hidden copying of data mentioned in: Why does this conversion happen implicitly?. Or maybe it does, actually.

9

This easily implies Float64(x::Meters) and Float64(x::CentiMeters) for this example. I have a feeling that the only reason the convert(::Type{Float64},...) methods even exist is because those constructors were so instinctively uncomfortable, even though they’d accomplish the same thing in the Base.:+ method without the unwanted conversions. It’s actually uncomfortable for the same reason the converts are: we are deliberately sacrificing explicit units information for weirdly implicit units in Float64 arithmetic. I’d also further speculate the reason for this weird approach to promotion is an avoidance of implementing most of Length promotion for its arithmetic. Here’s another attempt at promotion, less complicated than the parametric Unitful.jl arithmetic of course:

# parts of the original code I can approve of
abstract type Length end

struct Meters <: Length
  value::Float64
end

struct CentiMeters <: Length
  value::Float64
end

Base.convert(::Type{Meters}, x::CentiMeters) = Meters(x.value / 100)
Base.convert(::Type{CentiMeters}, x::Meters) = CentiMeters(x.value * 100)

# new code, I'd prefer Meters but I'll stick to the intent
Base.promote_rule(::Type{CentiMeters}, ::Type{Meters}) = CentiMeters

# edits to original code
Base.:+(a::Length, b::Length) = +(promote(a,b)...)
Base.:+(a::T, b::T) where T<:Length = T(a.value + b.value)

It does everything you needed:

julia> begin
       x = CentiMeters(100) + Meters(2) # defaults to CentiMeters
       y::Meters = CentiMeters(100) + Meters(2) # but I can get Meters if I want (via implicit conversion and type assertion).
       @assert x isa CentiMeters && x.value == 300
       @assert y isa Meters && y.value == 3
       x, y
       end
(CentiMeters(300.0), Meters(3.0))

julia> lengths = Float64[]; push!(lengths, Meters(1)) # bad implicit units
ERROR: MethodError: Cannot `convert` an object of type Meters to an object of type Float64
...
julia> length = CentiMeters(100) |> Meters # bad constructor with redundant units
ERROR: MethodError: Cannot `convert` an object of type CentiMeters to an object of type Float64
...

And with only 1 more line. Bad shortcuts don’t need to be fixed with breaking changes to the core language.

1 Like
10

also just to clarify, conversion on push! or setindex! is not “implicit” in the sense that the language is inserting convert ; these are intentional implementations of these methods that explicitly call convert

11

Ok, let’s try a simplified but potentially incomplete example as a start. Without changing the code below, I want the last three lines to behave as described in the accompanied comment. Your’e allowed to add code below it to define/redefine convert, promote_rule, or whatever.

struct A
    value::Float64
end
struct B
    value::Float64
end
B(0)           #1. disallow; should error.
x::A = B(0.)   #2. disallow.
A(0.) + B(0.)  #3. allow; should return an A.

I don’t see how you can do this without changing the language, without splitting convert into implicit and explict variations. #1 and #2 require what I’m calling “implicit” convert, for now because they go through convert as a result the lowering of ::.

Implementation aside, is this such a big change in the language? x.field used to be lowered to getfield. Now it lowers to getproperty, which has a different meaning. The difference here is that we have to decide which call to convert is an “implicit” one. For the sake of this discussion, let’s draw the line at ::. Whenever :: calls to convert, it is an implicit call.

12

Yeah, the more I think about it, I myself am not clear on what is (or should be considered) implicit. But I think to start with, I think we could focus on whether there is or should be a distinction between “implicit” and “explicit”, and whether there should be a way to express that. For now I would say :: calling convert counts as “implicit”.

13

just for fun, I gave Claude this prompt:

julia currently inserts explicit convert in three cases: assignments to typeasserted variables, default outer constructors, and return values from functions with typeasserted return types. can you help me add a cli flag --no-implicit-convert that will turn off all three of these implicit convert points? ultrathink

lo and behold, about 15 minutes later, it spit out this branch GitHub - adienes/julia at ad/no-implicit-convert which indeed seems(?) to correctly implement ./julia --no-implicit-convert to disable these convert calls. I guess turns out it’s not such a complicated change

14

Easy:

julia> begin
       struct A
           value::Float64
       end
       struct B
           value::Float64
           B(x::Float64) = new(x) # suppress default constructors, works on A too
       end
       Base.:+(a::A, b::B) = A(a.value+b.value) # why promote? multiple dispatch!
       Base.:+(b::B, a::A) = a+b
       end

julia> B(0)
ERROR: MethodError: no method matching B(::Int64)
...
julia> x::A = B(0.)
ERROR: MethodError: Cannot `convert` an object of type B to an object of type A
...
julia> A(0.) + B(0.)
A(0.0)

A(0.) + B(0.) could involve promote, an extension of convert, which has implicit behaviors by design. However, you don’t want convert to work in basically every other case. Sorry, we can’t have promote without convert and we can’t make convert explicit-only, those are breaking changes. If simpler multiple dispatch and constructors can’t serve some future example, you’d have to implement a parallel explicit_promote and explicit_convert to dodge Julia’s implicit conversion. Those could probably be shorter names in a clear ExplicitConversions package.

Even in your original example, you were fine with Meters-CentiMeters conversion, you just reasonably didn’t like the conversions to Float64. You didn’t need that to implement promotion for Meters-CentiMeters addition to begin with.

15

Benny, you’re putting effort to engage, and I don’t want to create more confusion. But I didn’t learn much from your response that I didn’t already know. It only goes to show that my example was inadequate. You did need to defy my constraints and go inside the struct definition to redefine the internal constructor and bypass the implicit call to convert, though. And that is telling of the hacks people have to use to bypass the implicit conversion.

You’ve said a few times that what I’m suggesting is a breaking change and I don’t see how. If I define:

Base.implicit_convert(args...) = Base.convert(args...)

and

f(x)::T

in a function definition de-sugars to

Base.typeassert(Base.implicit_convert(T, f(x)), T)

instead of

Base.typeassert(Base.convert(T, f(x)), T)

Nothing really breaks, does it? How is that a breaking change?

16

Writing an inner constructor is not a “hack”, and sorry, I didn’t realize you were forbidding language fundamentals. To be fair, “or whatever” was an incredibly vague option.

  1. This directly contradicts the language specification, the definition of a breaking change. The convert call is really not an internal detail you can just replace.
  2. Your plan is to replace convert with implicit_convert, retain the previous implicit conversion behavior by making implicit_convert fall back to convert, and require us to write 2 boilerplate no-op methods to “turn off” implicit conversion for select types going forward. People can now write either implicit_convert or convert methods for implicit conversion by relying on the fallback, splitting the interface. When people opt out of the fallback, implicit_convert becomes a terribly deceptive name for a no-op, which becomes a very good reason to always manually write convert, leaving manual implicit_convert only for boilerplate. Boilerplate is a strong indicator that there’s something wrong with the API.
  3. A much earlier point still stands: you could’ve done the same thing with none of the drawbacks by implementing a much better named explicit_convert for those select types and letting explicit_convert fall back to convert otherwise. Implicit conversion is not split into explicit_convert methods, there are no deceptive boilerplate no-ops, and none of this is a breaking change.
  4. Implement an explicit_promote properly on top of explicit_convert, and you have exactly what you asked for, pending a fitting example. That parallel name is deceptive though because it’d be used to implement implicit promotion in operations as well. This is unusual, type-annotated languages either have some degree of implicit conversions in assignments going hand in hand with promotions in operations, or mandate explicit conversions and type-matched operations.