3 posts were split to a new topic: Sharp edge between Threads.threadid() and task ids
Tweaked the API a bit. Now @bind x = 1 is for immutable variable-bound objects, while @bind :mut x = 1 is for mutable. I feel like @bind is a good word for the Julia programmer because it reminds them you are binding the object to a specific variable.
Also there are now loops. There are also a bunch of Base functions forwarded now that will automatically check the .moved property before doing a read:
julia> @bind x = [1.0]
Bound{Vector{Float64}}([1.0])
julia> @move y = x
Bound{Vector{Float64}}([1.0])
julia> sin(x[1])
ERROR: Cannot use x: value has been moved
Also, for anything that is isbits, BorrowChecker.jl will just automatically deepcopy instead of moving. This is actually the same behavior of Rust (via the Copy trait). If something is allocated on the stack, just create copies rather than unbinding, since there’s no need:
julia> using BorrowChecker
julia> @bind x = 1
Bound{Int64}(1)
julia> @take x
1
julia> @take x
1
But if something is not isbits, like a vector, it will unbind from the variable:
julia> @bind array = [1, 2, 3]
Bound{Vector{Int64}}([1, 2, 3])
julia> @move array2 = array
Bound{Vector{Int64}}([1, 2, 3])
julia> array
[moved]
My general design principle is just to try to get the API as close to Rust’s borrow checker as possible, because obviously they know way more than me about all the subtleties of each design choice.
You can freeze/thaw things with this interface too. Just move it to a mutable variable:
julia> @move :mut array3 = array2
BoundMut{Vector{Int64}}([1, 2, 3])
julia> array3[end] = 4
4
julia> array3
BoundMut{Vector{Int64}}([1, 2, 4])
julia> array2[end] # bad!
ERROR: Cannot use array2: value has been moved
Then, for loops can now use @bind to get array elements. This moves the contents of the iterator:
julia> @bind :mut for var in 1:5
@show var
end
var = BoundMut{Int64}(1)
var = BoundMut{Int64}(2)
var = BoundMut{Int64}(3)
var = BoundMut{Int64}(4)
var = BoundMut{Int64}(5)
It also correctly binds the iteration symbol to the object, to help catch mistakenly leaving the BorrowChecker.jl system:
julia> @bind for var in 1:5
@show var
var2 = var # Bad!
@move var3 = var2
end
var = Bound{Int64}(1)
ERROR: Variable `var2` holds an object that was reassigned from `var`.
Like the OP, I’m a huge fan of the Rust borrow checker, but like others I’m not sure whether something like the Rust borrow checker can be added to Julia such that it is both practically useful and yet doesn’t require reinventing the entire language. However, I also don’t think we need a borrow checker to achieve the stated goals of avoiding aliasing issues and data races. I’d say a much more Julian way to tackle the problem is to protect the critical objects with an access tracker and then throw an error whenever multiple concurrent access is detected. Something like this is what I’m thinking of:
mutable struct AccessTracked{T}
value::T
accessed:Bool
end
function access(f, args::AccessTracked...)
@assert !any(getfield.(args, :accessed))
setfield.(args, :access, true)
try
f(getfield.(args, :value)...)
finally
setfield.(args, :access, false)
end
end
Usage example:
function transfer(balances, i, j, v)
access(balances) do balances
balances[i] -= v
balances[j] += v
end
end
Like a borrow checker, this turns silent data corruption into an explicit error, but of course unlike with a borrow checker the check is now done at runtime rather than compile time. I’ll admit that that’s not quite as nice, but if this bothers you too much then probably you shouldn’t be coding in a dynamic language in the first place.
Also, if you want to avoid runtime overhead in production then you can just add a method
access(f, args...) = f(args..)
and feed a plain instead of a wrapped object into your code.
1 Like
This is a broken implementation of a lock, that doesn’t protect the values properly in the presence of concurrency. Two tasks calling access at the same time may both read accessed as false and set the value to their preference, overwriting each other in some manner. You’d at least want an @atomic on that accessed and do atomic update of the property, only progressing if the old value was false.
5 Likes
This is similar to how BorrowChecker.jl works btw. Equivalent type:
mutable struct BoundMut{T}
@atomic value::T
@atomic moved::Bool
@atomic immutable_borrows::Int
@atomic mutable_borrows::Int
const symbol::Symbol
end
.moved is like .accessed in your example.
With Cassette.jl we can make it so that calling any unknown function on a Bound* type will set .moved = true, making it so you can’t use that variable anymore in the scope.
Except if it is isbits, like a tuple of ints - in that case it just gets copied and the original variable is still available.
2 Likes
I agree that full borrow-checking seems like overkill for just data race prevention, but I’d rather a data race be statically inferrable so I can manually fix it (and probably learn a couple new things). Although a runtime error is much easier than making a test for invalid results, a single-thread process or sheer luck still misses it. This isn’t as tolerable as type instability where we’d still get the right answer but slower, given no MethodErrors, and there are static checks for even that.
1 Like
BTW Julia 1.11 has Lockable (exported on 1.12) that can be useful for things like this:
julia> using Base: Lockable
julia> struct MyObj
field::Int
end
julia> x = Lockable(MyObj(1))
Lockable{MyObj, ReentrantLock}(MyObj(1), ReentrantLock(nothing, 0x00000000, 0x00, Base.GenericCondition{Base.Threads.SpinLock}(Base.IntrusiveLinkedList{Task}(nothing, nothing), Base.Threads.SpinLock(0)), (29, 0, -1)))
julia> x[]
ERROR: ConcurrencyViolationError("lock must be held")
Stacktrace:
[1] concurrency_violation()
@ Base ./condition.jl:8
[2] assert_havelock(l::ReentrantLock, tid::Nothing)
@ Base ./condition.jl:29
[3] assert_havelock
@ ./lock.jl:52 [inlined]
[4] getindex(l::Lockable{MyObj, ReentrantLock})
@ Base ./lock.jl:328
[5] top-level scope
@ REPL[8]:1
julia> @lock x x[]
MyObj(1)
5 Likes