Why is BigFloat mutable?

In adding more tests to BorrowChecker.jl, I ran into the weird cases of BigInt and BigFloat. Apparently these are actually mutable:

julia> x = big(1.0)
1.0

julia> y = x
1.0

julia> y.prec *= 2
512

julia> x  # [Original float changed]
0.711763059342694544306441584158307023432639792317636119656901806469269447295820159150765983076895562999055705307595782779072780864355582835926727856355473418

Does anybody know why this is? Are there any operations that take advantage of this? I noticed that standard operations still create new objects:

julia> x = big(1.0)
1.0

julia> y = x
1.0

julia> y *= 2
2.0

julia> x
1.0

So I wasn’t sure where the mutability actually came into play, if at all. I mean, even String is immutable.

2 Likes

They’re mutable because they need to carry around a reference for GC and whatnot. They’re managed by external libraries.

Some packages do take advantage of this yes: GitHub - jump-dev/MutableArithmetics.jl: Interface for arithmetics on mutable types in Julia, GitHub - tkluck/InPlace.jl: InPlace.jl - in-place operations where possible

1 Like

Does the entire object need to be mutable for that though?

# gmp.jl
mutable struct BigInt <: Signed
    alloc::Cint
    size::Cint
    d::Ptr{Limb}
end

Fields are modified in-place in the source code, so yes.

So… just these?

ZERO.alloc, ZERO.size, ZERO.d = 0, 0, C_NULL
ONE.alloc, ONE.size, ONE.d = 1, 1, pointer(_ONE)

Why couldn’t this be done at instantiation time? Why does the entire object need to be mutable in user space?

That’s modifying in-place only two globals, that has to be done during __init__.

I see also

and haven’t looked much further

Well sync_rational! could be rewritten as

function sync_rational!(xq::_MPQ)
    xq.rat.num = BigInt(xq.num_alloc, xq.num_size, xq.num_d)
    xq.rat.den = BigInt(xq.den_alloc, xq.den_size, xq.den_d)
    return xq.rat
end

and the unsafe_store! should already be fine since z.d is a Ptr. It isn’t assigning z.d there.

I think it’d be better to just make those constants references then

const ZERO = Ref{BigInt}()
const ONE = Ref{BigInt}()

instead of making BigInt mutable everywhere for all users

BigFloat isn’t mutable any more on v1.12, since make faster BigFloats by vtjnash · Pull Request #55906 · JuliaLang/julia · GitHub

Not really. It’s meant to be immutable, but that’s not enforced at all levels. And:

julia> ismutable("")
true

julia> ismutabletype(String)
true
9 Likes

Ah, that’s great!! Thanks.

(I guess someone should make a similar PR for BigInt?)

1 Like

But keep in mind the underlying data is still mutable, it’s just that BigFloat isn’t a mutable struct any more. In particular, MutableArithmetics.jl, an important low-level package, still works fine for BigFloat, on nightly too:

julia> using MutableArithmetics: MutableArithmetics as MA

julia> x = big(.1)
0.1000000000000000055511151231257827021181583404541015625

julia> MA.operate!(+, x, x)
0.200000000000000011102230246251565404236316680908203125

julia> x
0.200000000000000011102230246251565404236316680908203125

That said, MutableArithmetics.jl needs to trespass on implementation details of Base to work on BigInt or BigFloat, a public interface to do the mutation has never existed: mutating APIs for BigInts and BigFloats? · Issue #31342 · JuliaLang/julia · GitHub

3 Likes

Thanks! I guess the good news is that BorrowChecker.is_static already flags these for moves (since the Ptr is mutable), so no changes are needed.

Though it’s a bit frightening that doing copy on an array of BigFloat will result in an array that is aliased to the original :scream:

julia> x = big.(randn(32));

julia> y = copy(x);  # shallow copy not enough

julia> MA.operate!(+, y[1], 3);

I guess this would modify x[1] too, which is kinda terrifying. At what point do the perf gains outweigh this much debugging difficulty… For this reason I feel like there probably shouldn’t ever be a public interface?

I’m not sure if this is actually documented, but the official story for Number (thus BigInt and BigFloat) is “treat it as immutable”. That justifies the behavior of copy(::BigFloat) being the identity.

On the other hand, the MutableArithmetics.jl interfaces are perfectly sane, too, in particular, mutable_copy and copy_if_mutable are provided:

1 Like

It is enforced throughout the public API, even going so far as to special case === and objectid to compare by value instead of memory address. String is clearly meant to behave as an immutable type in every respect. I suppose you can grab a pointer and go wild, but it’s probably not a relevant design goal for BorrowChecker.jl to accommodate such experiments.

The fact that ismutable seems to disagree is explicitly flagged in a note in the docstring: Essentials · The Julia Language

NOTE: For technical reasons, ismutable returns true for values of certain special types (for example String and Symbol) even though they cannot be mutated in a permissible way.

1 Like

The main reason String is treated as mutable is that any operation that creates a string requires mutating a String using some internal and slightly hacky functionality.

2 Likes

Off -topic, really, but it’s worth noting that this doesn’t suggest that y is mutable. The reference has changed, not the object itself.

1 Like

Thanks. Just curious: why would a low-level string precursor necessitate marking the resultant String object as mutable in user-space? Or does it actually create a String and then mutate it from within Julia?

Or does it actually create a String and then mutate it from within Julia?

yes.

string(a::Union{Char, String, SubString{String}, Symbol}...) = _string(a...)

function _string(a::Union{Char, String, SubString{String}, Symbol}...)
    n = 0
    for v in a
        # 4 types is too many for automatic Union-splitting, so we split manually
        # and allow one specializable call site per concrete type
        if v isa Char
            n += ncodeunits(v)
        elseif v isa String
            n += sizeof(v)
        elseif v isa SubString{String}
            n += sizeof(v)
        else
            n += sizeof(v::Symbol)
        end
    end
    out = _string_n(n)
    offs = 1
    for v in a
        if v isa Char
            offs += __unsafe_string!(out, v, offs)
        elseif v isa String || v isa SubString{String}
            offs += __unsafe_string!(out, v, offs)
        else
            offs += __unsafe_string!(out, v::Symbol, offs)
        end
    end
    return out
end

Thanks. I wonder if making a special internal _MutableString type for mutation at that stage would be better? That way String could actually be marked as immutable so that the compiler could do constant folding in user code (??)

Yes, “meant to”. That’s what I said in the first place: