How do ScopedValues actually work?

Internals & Design
1

I understand that a ScopedValue is a key into Scope (a wrapper for a PersistentDict) that is accessible by Core.current_scope(). I understand that @with creates a new Scope with the specified value. What I don’t understand is how that new Scope becomes what is returned by Core.current_scope(), and how the original Scope is restored when the body of the @with expression concludes. According to base/scopedvalues.jl, @with returns

Expr(:tryfinally, esc(ex), nothing, :(Scope(Core.current_scope()::Union{Nothing, Scope}, $(exprs...))))

That expression doesn’t seem to correspond to any documented Julia control structure and I can’t see how it modifies and then restores what is returned by Core.current_scope. There seems to be some magic here.

2

Julia’s Task object has a scope field so it knows what scope it belongs to, and this field is inherited from parent tasks. You can se that in the original PR that introduced them: Scoped values by vchuravy · Pull Request #50958 · JuliaLang/julia · GitHub. You’ll notice the scope field is replacing a logstate field: logger’s already had dynamic scope so that with_logger would apply the logger only to the tasks executing within it, rather than globally to other pre-existing tasks. ScopedValues generalized this feature and reimplemented the dynamic log scope in terms of a ScopedValue. And with resets the tasks scope field to whatever it was before the with block: Scoped values by vchuravy · Pull Request #50958 · JuliaLang/julia · GitHub.

(My links are based on the original PR so some things may have changed a bit since).

2 Likes
3

We just stuck the compiler support for installing a scope into :tryfinally, because it’s conceptually similar in that it installs a task local value (exception stack vs scope stack), so it was easy to wire up. That’s considered an implementation detail though.

4

The code in the PR @ericphanson referred to,

    quote
        ct = $(Base.current_task)()
        current_scope = ct.scope::$(Union{Nothing, Scope})
        ct.scope = $(Scope)(current_scope, $(exprs...))
        $(Expr(:tryfinally, esc(ex), :(ct.scope = current_scope)))
    end

is no longer in scopedvalues.jl and has been replaced with the expression I quoted above. And I see by doing a dump on the current task that there is indeed scope field, but it cannot be directly queried or assigned to. Combining this with @Keno 's response, I conclude that the code above was essentially moved under the hood into Julia’s internal implementation of the :tryfinally expression. So it is indeed “magic” in the sense of not being implemented entirely in visible Julia code.

As far as I am concerned the two of you together answered my question but I can only mark one as the solution.