Sometimes I want to make a field in a type mutable, and I do like
struct S
a::Ref{Float64}
end
This looks similar to the reference type used in ccall like
@ccall f(a::Ref{Int})::Cdouble
so I always thought it’s good.
But I just learned that Ref{T} is an abstract type, and Base.RefValue{T} is concrete.
Does that mean the above code is bad for performance? Or Ref{Float64} is equivalent to Base.RefValue{Float64} in a struct definition?
it is bad, because as you say it is an abstract type, somehow the use of Base.RefValue is discoureaged as it is unexported, but I find quite useful in some cases anyway when you have a small number of mutable fields
Note that since v1.8 you can have mutable structs with const fields. It’s the inverse of what’s stated here, but can be similarly useful.
As you already mentioned, Ref is an abstract type.
Let’s check this in the REPL.
julia> isabstracttype(Ref)
true
julia> subtypes(Ref)
8-element Vector{Any}:
Base.CFunction
Base.RefArray
Base.RefValue
Core.Compiler.RefValue
Core.LLVMPtr
GenericMemoryRef
Ptr
UnsafePointers.UnsafePtr
So this tells us, that there is a bunch of subtypes of Ref, including the mentioned Base.RefValue.
julia> isabstracttype(Base.RefValue)
false
julia> isconcretetype(Base.RefValue)
false
julia> Base.RefValue isa UnionAll
true
julia> isconcretetype(Base.RefValue{Float64})
true
If we check further, we see that Base.RefValue is a UnionAll type and by inserting an actual type we produce a concrete type, e.g. Base.RefValue{Float64}.
Accordingly, for the use in method signatures Ref can be considered equivalent to Base.RefValue for most use cases.
(Technically they are not, but for the majority of users dispatching on the subtypes T<:Ref is not too common.)
For annotating fields in a struct or mutable struct, however, the are not.
To forego the use of internals (Base.RefValue) without compromising concrete fieldtypes one can for example use a pattern like below:
struct S
a::typeof(Ref(1.0))
end
… or make the field a type parameter:
struct S{T<:Ref{<:AbstractFloat}}
a::T
end
Thank you all.
Usually I want only one or two fields to be mutable. I really want the opposite: Instead of const fields in a mutable struct, I think mutable fields in a immutable struct are more useful.
Why cannot we have mutable for fields in the standard struct?
I’m quite annoyed, that this isn’t AbstractRef for what’s now Ref and Ref for what’s now Base.RefValue …
I’m using it quite often, and always stumble over it RefValue being longer and not exported by base 
Welp, guess this won’t change anytime soon.
As I understand, the issue is that mutable struct is fundamentally different than an immutable struct. An immutable “pure data” and, since it can’t be mutated, doesn’t need to observe or make observable any remote changes to its fields. But a mutable can have multiple references to it and they must all see the same thing everywhere even if it is only changed in one place.
The consequence is that mutable structs must always be stored on the heap and referenced by a pointer to that common data (except for short-lived cases where “escape analysis” proves this unnecessary). Immutable structs, not needing to worry about being changeable elsewhere or wanting to change things elsewhere, do not need to go on the heap and can be put wherever the compiler wants. This usually means on the stack or simply in registers. Note that a Vector/etc of immutables will still go on the heap because the container itself is mutable. But when an entry is loaded from the array for processing, that entry itself can be put anywhere and doesn’t have to reference back to its copy in the array.
A Ref in an immutable is a pointer to the heap where the data can be shared elsewhere, but the immutable parent struct doesn’t need to worry about the pointer itself being observed elsewhere and can do what it wants with it. A const field of a mutable struct must still go with all the other fields, wherever they are (i.e., the heap). It just tells Julia to yell at you if you try to change it.
So the reason there aren’t mutable fields within immutable structs as a reverse to const fields in mutable structs is because there are additional semantics tied up in the distinction between immutable struct and mutable struct. If you want to change a value in an immutable struct (locally, since it can’t have side effects elsewhere – you’ll need to return it if you need the new value elsewhere), the usual method in Julia is to re-build the struct with that field changed. Accessors.jl offers convenient tools for this. Although the semantics of the language prevent you from mutating the value, in simple cases the compiler will see what’s happening and mutate it anyway.
An example of this is how reshape on an Array will keep the underlying data (in the === sense) but build a new Array around it with the size changed.
In short, an immutable is much more restricted in what you can do to it. Unfortunately, this means that mutating it is not something you can currently (or probably ever) do. These restrictions are precisely what allows the compiler to optimize them aggressively, so immutables tend to perform better and it’s why they’re recommended (and default). I also usually find them easier to reason about, even if they’re less ergonomic.
Perhaps the most basic difference between mutables and immutables:
julia> struct S end
julia> mutable struct M end
julia> S() === S()
true
julia> M() === M()
false
Good to know the difference between mutable and immutable struct…
Yes, this is the real problem. Base.RefValue{T} is long and hard to remember. We use Ref(1) to instantiate the object, why not just make Ref{T} as its type?
I use Array{T,0} for this purpose. It is easy to remember and has been available in Julia since version 0.4, and probably earlier, so there is no worry that it may go away.
julia> struct T
s::Array{Float64,0}
t::Int
end
julia> t = T(zeros(),54)
T(fill(0.0), 54)
julia> t.s[]=9.9
9.9
julia> t
T(fill(9.9), 54)
Thank you. This is a good solution.
See Export Base.RefValue · Issue #40369 · JuliaLang/julia · GitHub - apparenly, you might as well make the whole struct mutable.
That discussion doesn’t provide any example or explanation of why using Ref’s in immutable structs causes performance issues.
It mentions “several” other github issues that complains of bad performance, but doesn’t provide any.
I wonder if the performance issues mentioned there weren’t precisely due to the fact that Refs are abstract and people should be using Base.RefValue in struct fields instead (or the nice trick shown above, Array{T,0})
Just put up a Ref-killer package: ZeroDimensionalArrays.jl
Also more efficient than Array{T, 0}.
Although the linked issue didn’t show a benchmark I wonder how you might come to this conclusion after actually reading all of it?
The “I wonder” means I don’t know that for sure…
I wonder that because I just can’t make sense of this performance problem. There is nothing special about Base.RefValue when compared to any other mutable struct.
If using Base.RefValue as a field in a immutable struct hurts performance then that should also be true for any mutable struct, I guess including Arrays. And I never saw people complaining of performance issues with any other mutable struct, usually just Refs (because it is not obvious it is an abstract type).
Not having tried any of this, I think the line of reasoning does in the issue does definitely make sense.
Simply because a single mutable struct at the outermost level filled with primitives and structs ( of primitives and structs( of… )) creates the least fragmented memory.
FYI the Julia-specific term is isbits:
Yes, that is true for any mutable struct. Since the issue was about RefValue and why they are not exported, I thought they were mentioning issues specifically with it, not the general drawbacks of using mutable structs as fields of immutable structs.
Although it indeed causes memory fragmentation, there are still tons of sound use-cases for that where this fragmentation doesn’t matter.
And if the sole reason for not exporting Base.RefValue is to prevent people from using a mutable struct as a field in a struct, I don’t think that objective is being achieved by not exporting it. Even worse, now people are unknowingly using an abstract mutable struct as fields on structs.
Just point everyone to the new package I linked to above. Once its registered.
Hypothetically even the Ref doc string could point users to use ZeroDimensionalArrays.jl when that makes sense.