ANN: Setfield.jl Yet another package for working with immutables

package
announcement

#1

Setfield.jl is a package for modifying immutables. It allows to write code like

@set obj.a.b.c = foo

which yields a duplicate of obj with the appropriate field replaced. Compared to Reconstructables.jl it has the following advantages:

  • Fully typestable, therefore good performance
  • Does not require keyword constructors
  • Supports indexing, e.g. @set obj.my_tuple[5] = 123
  • Supports updates like @set obj.velocity[2] += 2

#2

Julia implementation of Lens using generated function! This is super cool :slight_smile:

Though I suppose the type stability is not coming from lens but rather because you are using the default constructor (or the order of arguments to the constructor has to match with that of field).

I wonder if it makes sene for Julia lang itself to have lens and build “immutable struct mutation” on top of it:


#3

what happens when the struct has one (or two) primitive fields and ten of these are allocated and stuffed next to each other in sequence forming an immediate, not indirect manner of indexing and the first field of the sixth struct has its value changed?


#4

I think there is no way to get full type stability here without generated functions at some point.
In order to manipulate fields, you need to process fieldnames(typeof(obj)) in one way or another. Without generated functions, this looks a priori like an arbitrary vector of symbols to the compiler. The compiler would need to

  • Figure out that its safe to “evaluate” this vector at compile time
  • Fully unroll the loop/recursion you use to process the fields
  • Constant propagate all the field symbols

I am not a compiler guy, but to me this seems incredibly hard.


#5

Not sure I understand the question, can you provide a code example?


#6

I see. Thanks. That makes sense.

I think this package would be perfect if it supports custom inner constructors with type parameters. Any plan to support constructor like B{T} (as in https://github.com/tkf/Reconstructables.jl#how-to-use-type-parameters)?


#7

Why not make the inner constructor an outer constructor?

julia> struct B{T, X, Y}
           x::X
           y::Y
       end

julia> B{T}(x::X, y::Y = 2) where {T, X, Y} = B{T, X, Y}(x, y)

julia> b = B{1}(2,3)
B{1,Int64,Int64}(2, 3)

julia> using Setfield

julia> @set b.x = 512
B{1,Int64,Int64}(512, 3)

#8

Can you “change” the type of b by b_new = @set b.x = 1.0?


#9

No you can’t do that. Note that you can’t do that with mutable structs either.


#10

To me changing the type like this seems strange. Can you give an example, where that would be useful?


#11

A good example that you might want to change is the whole family of problem struct in DifferentialEquations.jl:
http://docs.juliadiffeq.org/latest/types/ode_types.html

You may want to change the initial condition u0 type from Vector to Matrix, or regular array to static array or GPU array, etc. This is going to change the type parameters of the problem type.


#12

Okay I think we can do that. We need ways to hook into @set. I think

@set obj.x = 1.0

should call setproperty(b, Val{:x},1.0) and then the user could overload setproperty to get any kind of interesting behaviour. The default implementation of setproperty should be something like

  1. destructure obj into properties
  2. manipulate properties
  3. construct new_obj from typeof(obj) and properties

Each step should be overloadable. 3) is where you could hook in your

Reconstructables.constructor_of(::Type{<: B{T}}) where T = B{T}

What do you think?


#13

Hmm… In that case, do you need to call setproperty from set(l::FieldLens{field}, ...) which is a generated function? If so, my understanding is that it is not going to work since the codegen part of the generated function has to be pure. Allowing users to “register” their definition in setproperty is not pure.

I actually had a similar trouble with constructor_of and generated function. https://github.com/tkf/Reconstructables.jl/pull/1


#14

I think it can be done roughly like

function set(l::Fieldlens, obj, val)
    setproperty(obj, field(l), val)
end
function setproperty(obj, field, val) # field is say a Val
     fields = replace_field(obj, field, val) # fields is a tuple
     constructor_of(obj)(fields...)
end
@generated  replace_field...

#15

Cool, this looks like it could/should replace the reconstruct function in Parameters.jl.

Are generated functions needed? At least for the getter probably not. Consider (Julia 0.6):

julia> struct A; a::Int end                                                                                                                                              
                                                                                                                                                                         
julia> f(::Val{S}, obj) where S = getfield(obj, S)
f (generic function with 1 method)

julia> @code_warntype f(Val(:a), A(5))
Variables:
  #self# <optimized out>
  #unused# <optimized out>
  obj::A

Body:
  begin                                                                                                                                                                  
      return (Main.getfield)(obj::A, $(Expr(:static_parameter, 1)))::Int64                                                                                               
  end::Int64                                                                                                                                                             

(Note that in Julia 0.7 the Val trick is not needed anymore.)

I think the getter can be re-written in above style (https://github.com/jw3126/Setfield.jl/blob/4dcf12784d3ca1c800e3d9fe2b7c844b6029d2d0/src/lens.jl#L96).

For the setter, generated functions are probably needed, although in Julia 0.7 maybe they could be avoided by recursions. But I’m not sure whether that would be of any benefit.


#16

Cool thanks!


#17

It should be noted that keyword arguments are only an issue on v0.6, so there’s no reason to make such a big deal about that. Generated functions are not statically compilable, so that would be a big deal down the road.


#18

But it would be nice to use this pattern also without having keyword constructors. At least for as long as keyword constructors are not defined automatically.


#19

I guess in the long run the problem will be solved in Base anyway. Keyword constructors are nice for large structs. However they are only an option, if you own the struct. Adding one to foreign structs is annoying and leads to bugs. Also not every Point(x,y) should have its own keyword constructor.


#20

Oops. Forget what I said :slight_smile: You can call setproperty inside the code generated by a generated function. You wouldn’t call it inside the codegen code part so it’s not a problem.

Yeah, the code looks good to me.