Package for Rust-like borrow checker in Julia?

General Usage
83

Tweaked the API a bit. Now @bind x = 1 is for immutable variable-bound objects, while @bind :mut x = 1 is for mutable. I feel like @bind is a good word for the Julia programmer because it reminds them you are binding the object to a specific variable.

Also there are now loops. There are also a bunch of Base functions forwarded now that will automatically check the .moved property before doing a read:

julia> @bind x = [1.0]
Bound{Vector{Float64}}([1.0])

julia> @move y = x
Bound{Vector{Float64}}([1.0])

julia> sin(x[1])
ERROR: Cannot use x: value has been moved

Also, for anything that is isbits, BorrowChecker.jl will just automatically deepcopy instead of moving. This is actually the same behavior of Rust (via the Copy trait). If something is allocated on the stack, just create copies rather than unbinding, since there’s no need:

julia> using BorrowChecker

julia> @bind x = 1
Bound{Int64}(1)

julia> @take x
1

julia> @take x
1

But if something is not isbits, like a vector, it will unbind from the variable:

julia> @bind array = [1, 2, 3]
Bound{Vector{Int64}}([1, 2, 3])

julia> @move array2 = array
Bound{Vector{Int64}}([1, 2, 3])

julia> array
[moved]

My general design principle is just to try to get the API as close to Rust’s borrow checker as possible, because obviously they know way more than me about all the subtleties of each design choice.

You can freeze/thaw things with this interface too. Just move it to a mutable variable:

julia> @move :mut array3 = array2
BoundMut{Vector{Int64}}([1, 2, 3])

julia> array3[end] = 4
4

julia> array3
BoundMut{Vector{Int64}}([1, 2, 4])

julia> array2[end]  # bad!
ERROR: Cannot use array2: value has been moved

Then, for loops can now use @bind to get array elements. This moves the contents of the iterator:

julia> @bind :mut for var in 1:5
           @show var
       end
var = BoundMut{Int64}(1)
var = BoundMut{Int64}(2)
var = BoundMut{Int64}(3)
var = BoundMut{Int64}(4)
var = BoundMut{Int64}(5)

It also correctly binds the iteration symbol to the object, to help catch mistakenly leaving the BorrowChecker.jl system:

julia> @bind for var in 1:5
           @show var
           var2 = var  # Bad!
           @move var3 = var2
       end
var = Bound{Int64}(1)
ERROR: Variable `var2` holds an object that was reassigned from `var`.
84

Like the OP, I’m a huge fan of the Rust borrow checker, but like others I’m not sure whether something like the Rust borrow checker can be added to Julia such that it is both practically useful and yet doesn’t require reinventing the entire language. However, I also don’t think we need a borrow checker to achieve the stated goals of avoiding aliasing issues and data races. I’d say a much more Julian way to tackle the problem is to protect the critical objects with an access tracker and then throw an error whenever multiple concurrent access is detected. Something like this is what I’m thinking of:

mutable struct AccessTracked{T}
    value::T
    accessed:Bool
end

function access(f, args::AccessTracked...)
    @assert !any(getfield.(args, :accessed))
    setfield.(args, :access, true)
    try 
        f(getfield.(args, :value)...)
    finally
        setfield.(args, :access, false)
    end
end

Usage example:

function transfer(balances, i, j, v)
    access(balances) do balances
        balances[i] -= v
        balances[j] += v
    end
end

Like a borrow checker, this turns silent data corruption into an explicit error, but of course unlike with a borrow checker the check is now done at runtime rather than compile time. I’ll admit that that’s not quite as nice, but if this bothers you too much then probably you shouldn’t be coding in a dynamic language in the first place.

Also, if you want to avoid runtime overhead in production then you can just add a method

access(f, args...) = f(args..)

and feed a plain instead of a wrapped object into your code.

85

This is a broken implementation of a lock, that doesn’t protect the values properly in the presence of concurrency. Two tasks calling access at the same time may both read accessed as false and set the value to their preference, overwriting each other in some manner. You’d at least want an @atomic on that accessed and do atomic update of the property, only progressing if the old value was false.

86

This is similar to how BorrowChecker.jl works btw. Equivalent type:

mutable struct BoundMut{T}
    @atomic value::T
    @atomic moved::Bool
    @atomic immutable_borrows::Int
    @atomic mutable_borrows::Int
    const symbol::Symbol
end

.moved is like .accessed in your example.

With Cassette.jl we can make it so that calling any unknown function on a Bound* type will set .moved = true, making it so you can’t use that variable anymore in the scope.

Except if it is isbits, like a tuple of ints - in that case it just gets copied and the original variable is still available.

87

I agree that full borrow-checking seems like overkill for just data race prevention, but I’d rather a data race be statically inferrable so I can manually fix it (and probably learn a couple new things). Although a runtime error is much easier than making a test for invalid results, a single-thread process or sheer luck still misses it. This isn’t as tolerable as type instability where we’d still get the right answer but slower, given no MethodErrors, and there are static checks for even that.

88

BTW Julia 1.11 has Lockable (exported on 1.12) that can be useful for things like this:

julia> using Base: Lockable

julia> struct MyObj
       field::Int
       end

julia> x = Lockable(MyObj(1))
Lockable{MyObj, ReentrantLock}(MyObj(1), ReentrantLock(nothing, 0x00000000, 0x00, Base.GenericCondition{Base.Threads.SpinLock}(Base.IntrusiveLinkedList{Task}(nothing, nothing), Base.Threads.SpinLock(0)), (29, 0, -1)))

julia> x[]
ERROR: ConcurrencyViolationError("lock must be held")
Stacktrace:
 [1] concurrency_violation()
   @ Base ./condition.jl:8
 [2] assert_havelock(l::ReentrantLock, tid::Nothing)
   @ Base ./condition.jl:29
 [3] assert_havelock
   @ ./lock.jl:52 [inlined]
 [4] getindex(l::Lockable{MyObj, ReentrantLock})
   @ Base ./lock.jl:328
 [5] top-level scope
   @ REPL[8]:1

julia> @lock x x[]
MyObj(1)
89

Nice! Someone needs to put it in Compat.jl :pray:

90

I just made it so BorrowChecker.jl can be enabled/disabled with LocalPreferences.toml. Which could eventually mean you can use it in a similar way to DispatchDoctor.jl. Only enable it for a package’s testing, but disable it for production.

For example, with all the defaults:

julia> using BorrowChecker

julia> expects_raw_vec(x::Array) = x .+ 1;

julia> @bind w = zeros(2)
Bound{Vector{Float64}}([0.0, 0.0])

julia> w[1] = 3.0
ERROR: Cannot write to immutable

julia> @bind result = BorrowChecker.@managed begin
           # Cassette.jl sets up a "move"
           # for all UNKONWN functions
           expects_raw_vec(w)
       end;

julia> w  # Moved! We can't use it now.
[moved]

julia> result
Bound{Vector{Float64}}([1.0, 1.0])

But if I run

using BorrowChecker: BorrowChecker as BC

BC.disable_borrow_checker!(Main)  # Use Preferences.jl for proper modules

before running, then I can get this:

julia> using BorrowChecker

julia> expects_raw_vec(x::Array) = x .+ 1;

julia> @bind w = zeros(2); typeof(w)
Vector{Float64} (alias for Array{Float64, 1})

julia> w[1] = 3.0  # We can do illegal things again
3.0

julia> @bind result = BorrowChecker.@managed begin
           expects_raw_vec(w)
       end;

julia> w
2-element Vector{Float64}:
 3.0
 0.0

julia> result
2-element Vector{Float64}:
 4.0
 1.0

In other words, it just turns into normal ol’ Julia code! No overhead because there’s literally nothing from BorrowCheck.jl left.