ANN: LazilyInitializedFields.jl - Handling lazily initialized fields in a consistent way

Community
1

Hi,

I created a small package to handle lazily initialized fields. A lazily initialized field is a field in a struct that does not get initialized on the creation of the struct but at some later point. While it is a good idea, in general, to fully initialize structs, there are times where for e.g. performance reason you want to only initialize certain fields when they are actually needed. I recently had such a situation and I wasn’t 100% satisfied with the solutions Julia itself provided, so I made this package.

Installation, examples, and documentation can be found at LazilyInitializedFields.jl (thanks PkgPkage.jl). It goes through how you use the package and contrast it with some of the other common ways of dealing with lazy fields in Julia (using a Ref or using a Union field) and why these didn’t fulfill the requirements I had. The page also shows how the package is implemented which some people might find interesting.

Note that the package is still in the registration process so has to be added by URL (as shown in the docs).

Best,
Kristoffer

26 Likes
2

Nice! That seems extremely useful.

I only gave it a very quick look but from reading the docs it seems to me that all lazily initialized structs are mutable:

using LazilyInitializedFields

julia> @lazy struct Foo
           a::Int
           @lazy b::Int
       end

julia> f = Foo(1, uninit)
Foo(1, uninit)

julia> ismutable(f)
true

in which case it might make sense to make the @lazy macro available for mutable structs only

3

They are mutable from julias point of view but the standard way of mutating them is disabled.

julia> f.a = 2
ERROR: setproperty! for struct of type `Foo` has been disabled

You can of course get around this with setfield! but the point is that with standard usage they act non-mutable so you won’t mutate it by accident.

2 Likes
4

Nice package. Kudos for the excellent docs.

1 Like
5

Will this impact performance relative to using immutable structs?

Alright, that makes sense.
On another note though, mutating is then disabled for lazily initialized mutable structs as well (and that is true for all fields, not just the lazily initialized ones), which effectively makes them immutable with standard usage.
Not sure if that’s feasible but it might be desirable to allow standard mutation for lazily initialized mutable structs.

6

Yes, since they are mutable they will no longer be stored in-line in e.g. Vectors. I’ll add a snippet in the docs about this.

I think that should already work:

julia> @lazy mutable struct Foo
           a::Int
           @lazy b::Int
       end

julia> f = Foo(1, uninit)
Foo(1, uninit)

julia> f.a = 2
2

julia> f.b = 3
3
3 Likes
7

Sounds great. I’m guessing there is no way to actually make them actual immutables?

Yes, sorry. I realized that was true as soon as I posted it

8

No, since (pretty much by definition) you want to mutate the fields after the creation of the object.

4 Likes
9

Here is the new caveat section: LazilyInitializedFields.jl

2 Likes
10

I think it might be nice to provide a way to make the struct immutable and then users can use things like https://github.com/jw3126/Setfield.jl to update values without mutation if they desire it.

3 Likes
11

I personally don’t think that would be too useful because it is likely the object will be stored somewhere (maybe in multiple places) and it is likely you do in fact want to change the object itself, not just get a new, initialized object.

12

I personally think that my desire to partially initialize a struct often doesn’t have much to do if it’s mutable or not, but maybe that’s just me. I find Setfield.jl works quite nice for me when I want to update immutable structs.

Edit:

I should say, regardless this looks really cool and I think I’m likely to make use of it sometime!

2 Likes
13

I believe for immutable structs you could use b::Ref{Union{Uninitialized, T}} + custom getproperty like you already have, no? Now that non-isbits structs are often allocated inline, it could have good performance in spite of the Ref.

I like the idea behind the package. I can see myself using it for memoizing computations that may not happen.

struct MyMat
    A
    @lazy det=LinearAlgebra.det(A)
end

would be a nice syntax for it.

I hope eventually someone will collect all these tricks into one big macro.

@fancy mutable struct Foo{T}
   a::T   
   @lazy b
   @read_only c
   @cached z=det(a)
end

etc. Common Lisp’s CLOS macro was ridiculous, but it had some really interesting parts.

1 Like
14

Probably not just you :). It could be possible to have non-mutating versions of @init!, @uninit! that returns the modified struct and have a @staticlazy or something that doesn’t convert to a mutable struct. I might get around to it some point but if someone needs some PRs for hacktoberfest feel free to work on it :slight_smile:

2 Likes
15

Hm, yeah, that might be a better implementation than what I currently have. I’ll have to try it out.

That’s pretty cool with the initialization next to the object.

1 Like
16

One annoyance is:

julia> struct A
           a::Int
           b::Ref{Union{Nothing, Int}}
       end

julia> struct B{T}
           a::T
           b::Union{Nothing, Int}
       end

julia> struct C{T}
           a::T
           b::Ref{Union{Nothing, Int}}
       end

julia> A(1, 2)
A(1, Base.RefValue{Union{Nothing, Int64}}(2))

julia> B(1, 2)
B{Int64}(1, 2)

julia> C(1, 2)
ERROR: MethodError: no method matching C(::Int64, ::Int64)
Closest candidates are:
  C(::T, ::Ref{Union{Nothing, Int64}}) where T at REPL[3]:2
Stacktrace:
 [1] top-level scope at REPL[6]:1

julia> C{Int}(1,2)
C{Int64}(1, Base.RefValue{Union{Nothing, Int64}}(2))

Ref Autogenerate `(::Type{Foo})(x::T, y) where T` constructor for type Foo{T} · Issue #35053 · JuliaLang/julia · GitHub. I could try generate that constructor myself.

17

Nice package! Concerning the " Other methods of achieving lazily initialized fields" with “Use a ::Ref{T} field”: I might misunderstand, but this method doesn’t fail 5), i.e. the struct can be immutable?

2 Likes
18

Thanks, that’s indeed true.

1 Like
19

Another annoyance, Nothing is special:

julia> struct A
           b::Ref{Union{Nothing, Int}}
       end

julia> struct Uninitialized end

julia> struct B
           b::Ref{Union{Uninitialized, Int}}
       end

julia> A(2.0)
A(Base.RefValue{Union{Nothing, Int64}}(2))

julia> B(2.0)
ERROR: MethodError: Cannot `convert` an object of type
  Float64 to an object of type
  Union{Uninitialized, Int64}
Closest candidates are:
  convert(::Type{T}, ::T) where T at essentials.jl:205
1 Like
20

I implemented it for immutable structs without type parameters at least: add an optimization for immutable structs without type arguments by KristofferC · Pull Request #1 · KristofferC/LazilyInitializedFields.jl · GitHub.

julia> using LazilyInitializedFields

julia> @lazy struct Foo
           a::Int
           @lazy b::Int
       end

julia> f = Foo(1, uninit)
Foo(1, uninit)

julia> ismutable(f)
false

julia> @init! f.b = 2
2

julia> f.b
2

julia> @uninit! f.b
uninit

julia> @isinit f.b
false

I want to write some benchmarks first before merging just to see that things look ok with it though.

3 Likes