# ANN: Setfield.jl Yet another package for working with immutables

**URL:** <https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267>\
**Category:** General Usage\
**Tags:** package, announcement\
**Created:** [February 22, 2018, 10:26pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267 "2018-02-22T22:26:46Z")\
**Posts on this page:** 20\
**Page:** 2

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [February 23, 2018, 8:52pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/21 "2018-02-23T20:52:12Z")

</div>

@ChrisRackauckas Btw I am curious, why are `@generated` functions should not statically be compilable?  
I would have guessed that `f(obj)` always works, if `f(::typeof(obj))` was seen at static compiletime and  
never works if `f(::typeof(obj))` was not seen at compiletime. Why does it matter if `f` is generated?

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 23, 2018, 8:54pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/22 "2018-02-23T20:54:35Z")

</div>

> [@jw3126](#):
>
> Adding one to foreign structs is annoying and leads to bugs.

That’s why you’d want to have something like `constructor_of` where you can register foreign “fake” constructor/factory function.

Keyword-only constructor is just a suboptimal solution to hook arbitrary pre-construction check/processing, which is (sometimes) needed to be done in the inner constructor. If it can be done with positional-only constructor then that’s also a nice solution.

> [@ChrisRackauckas](#):
>
> It should be noted that keyword arguments are only an issue on v0.6

Last time I checked, it was broken also in nightly 😢

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 23, 2018, 8:55pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/23 "2018-02-23T20:55:25Z")

</div>

> [@tkf](#):
>
> Last time I checked, it was broken also in nightly 😢

Really? Did you build it yourself? The nightlies haven’t built in a very long time so that would be the problem if you downloaded the binary.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 23, 2018, 8:58pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/24 "2018-02-23T20:58:42Z")

</div>

No, I didn’t build myself. It was the downloaded binary. But I thought it was showing something like “N days older than master” where N \< 10.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 23, 2018, 8:59pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/25 "2018-02-23T20:59:53Z")

</div>

Redhat using COPR?

[https://status.julialang.org/](https://status.julialang.org/)

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [February 23, 2018, 9:06pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/26 "2018-02-23T21:06:47Z")

</div>

FWIW I build a master from two days ago and it does not work:

```julia
julia> struct S{X,Y}
           x::X
           y::Y
           S(;x::X=1,y::Y=2) where {X,Y} = new{X,Y}(x,y)
           end

julia> Test.@inferred S(x=:a, y="")
ERROR: return type S{Symbol,String} does not match inferred return type Any
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] top-level scope

```

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 23, 2018, 9:07pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/27 "2018-02-23T21:07:22Z")

</div>

I was using [https://julialangnightlies-s3.julialang.org/bin/linux/x64/julia-latest-linux64.tar.gz](https://julialangnightlies-s3.julialang.org/bin/linux/x64/julia-latest-linux64.tar.gz) linked from [https://julialang.org/downloads/nightlies.html](https://julialang.org/downloads/nightlies.html)

I’m not sure which one in the [https://status.julialang.org/](https://status.julialang.org/) was that.

It seems that’s what Travis is using. For example, this shows “Commit a237986bf9 (2018-02-20 19:08 UTC)” and the `@inferred` with just keyword-only constructor is broken:

> **[Travis CI - Test and Deploy with Confidence](https://app.travis-ci.com/tkf/Reconstructables.jl/jobs/344120039)**
>
> Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you'll be testing your code in minutes.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 23, 2018, 9:10pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/28 "2018-02-23T21:10:08Z")

</div>

This didn’t handle it?

[https://github.com/JuliaLang/julia/pull/24795](https://github.com/JuliaLang/julia/pull/24795)

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [February 23, 2018, 9:24pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/29 "2018-02-23T21:24:16Z")

</div>

Apparently not. You think this is a bug?

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [February 23, 2018, 9:30pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/30 "2018-02-23T21:30:59Z")

</div>

Worth opening an issue to track it (if that hasn’t been done already).

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [February 23, 2018, 9:37pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/31 "2018-02-23T21:37:08Z")

</div>

[https://github.com/JuliaLang/julia/issues/25918](https://github.com/JuliaLang/julia/issues/25918)

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 24, 2018, 2:22am UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/32 "2018-02-24T02:22:28Z")

</div>

It was apparently quite easy to get “theoretical” type-stability from the keyword-only approach, with a similar `@generated` trick:

```julia
@generated function recon(thing; kwargs...)
    fields = [Expr(:kw, k, :(thing.$k)) for k in fieldnames(thing)]
    return quote
        constructor = $constructor_of(thing)
        constructor(; $(fields...), kwargs...)
    end
end

```

[https://github.com/tkf/Reconstructables.jl/commit/68075dafe54c749ecea1179c8d839e76b7d76753](https://github.com/tkf/Reconstructables.jl/commit/68075dafe54c749ecea1179c8d839e76b7d76753)

Of course, it’s only “theoretical” at this point since the constructor itself is not type-stable.

Possible benefits over lens/`setproperty` approach are that (1) you can do a “batch” update in which the constructor is called only once and (2) it generates only one (or two [\*]) function for each struct instead of `length(fieldnames(...))` functions.

[\*] I may need to turn `constructor_of` into a generated function for type-stability.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [February 24, 2018, 7:21am UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/33 "2018-02-24T07:21:16Z")

</div>

I would have thought that with the kw-constructor you shouldn’t need a generated function for “theoretical” type stability.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 24, 2018, 8:19am UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/34 "2018-02-24T08:19:50Z")

</div>

Well, I don’t know if it is actually needed since I’ve never tried it with Julia with type-stable keyword-only constructor. I just thought the `getfield` & `fieldnames` combo in the following code would be hard for Julia to make it type-stable. (or not?)

```julia
function recon(thing; kwargs...)
    constructor = constructor_of(thing)
    defaults = [(n => getfield(thing, n)) for n in fieldnames(typeof(thing))]
    return constructor(; defaults..., kwargs...)
end

```

The type of array `defaults` becomes `Array{Pair{Symbol}}` (the “value” part is undetermined) for sufficiently complex struct. If the type of `thing` has type parameters that depend on the fields, the return type cannot be determined, right? Can you avoid it?

Also, in Julia 0.6 and 0.7, it seems that the type of `defaults` itself cannot be inferred:

```julia
using Base.Test

struct S{X,Y,Z}
    x::X
    y::Y
    z::Z

    S(x::X, y::Y, z::Z) where {X, Y, Z} = new{X, Y, Z}(x, y, z)
end

get_fields(thing) =
    [(n => getfield(thing, n)) for n in fieldnames(typeof(thing))]

s = S(1, 2.0, "3")
@show @inferred get_fields(s) # fails

```

The most specific inference I can get out of variants of `get_fields` was when I used tuple and generator. But it still was not perfect.

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [February 24, 2018, 8:35am UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/35 "2018-02-24T08:35:45Z")

</div>

There is a way to avoid `@generated` functions, have full type stability and no need of a keyword constructor. It can be adapted to both the `Lens` design and your design. I sketch for simplicity the case of only updating a single field.

```julia
macro register(T)
    M = current_module()
    fields = eval(M, :(fieldnames($T)))
    quote
        function Reconstructables.recon(thing::$T, field, val)
            new_fields = map($fields) do fld
                fld == field ? val : getfield(thing, field)
            end
            constructor_of($T)(new_fields...)
        end
    end
end

```

The price is that you need to add `@register MyStruct` to each struct you want to use this feature with. This is even possible for structs living in other modules.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 24, 2018, 9:25am UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/36 "2018-02-24T09:25:14Z")

</div>

Isn’t it generating the similar amount code as the `@generated` function version? How is it better?

Also, I want to have custom inner constructors. In that case, you need to generate keyword-only or positional-only constructors anyway for it to have some systematic mutation mechanism. So, “you don’t need keyword-only constructors” is not a valid appealing point for use-cases I have in mind.

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [February 24, 2018, 2:40pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/37 "2018-02-24T14:40:33Z")

</div>

Good question. I don’t know if this is better then using `@generated`, as I don’t know what the problem with `@generated` is. If its only the amount of generated code, then I don’t think its any better. I even think there is no way to get the same performance with less generated code. At the end of the day each variant with perfect performance has to generate fully specialized `@code_llvm`. So I guess the total amount of generated code will always be the same for fully typestable variants.

---

<div class="post-metadata">

**Author:** ![mauro3](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mauro3/32/292_2.png) [@mauro3](https://discourse.julialang.org/u/mauro3)\
**Post date:** [February 24, 2018, 2:42pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/38 "2018-02-24T14:42:56Z")

</div>

> [@tkf](#):
>
> Also, I want to have custom inner constructors.

In Parameters.jl I require that there is a positional inner constructor for all fields defined (either automatically via `@with_kw` or by the user). That does all the enforcing of constraints and other things you may want to do. All other inner/outer constructors then go through that one. I think this scheme should be able to cover almost all cases.

---

<div class="post-metadata">

**Author:** ![jw3126](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/jw3126/32/3086_2.png) [@jw3126](https://discourse.julialang.org/u/jw3126)\
**Post date:** [February 24, 2018, 2:47pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/39 "2018-02-24T14:47:33Z")

</div>

Yeah I think this is good design. I usually do the same thing:

1. Have one and only one “low level” inner constructor. All other constuctors must go through this one in the end. Its only job is to enforce invariants, it does nothing else. It is positional and its signature is the fields of the struct.

2. Have whatever high level outer constructors you like.

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [February 24, 2018, 11:48pm UTC](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267/40 "2018-02-24T23:48:19Z")

</div>

I think that’s a source of bugs. If I understand correctly, that means users have to _manually_ call the inner constructor from the outer constructor with positional arguments. If there are many structs with more than 5 fields (say) it would be terrifying to do any kind of refactoring touching the fields.

(OK. We need to do that already when calling `new`. But please don’t increase the burden.)

Obviously, it does not imply that `@with_kw ` is a bad design since it also generates a keyword-only constructor as well.

If you let users to write a inner-constructor, it has to be usable by _humans_. Positional-only constructor is not for humans. Humans are good at referring to things by names, not positions. I think that’s the primary reason why we use structs instead of tuples.

Then what is required for packages like Reconstructables.jl and Setfield.jl are to generate constructor usable by programs. That means a keyword-only _or_ positional-only constructor. They both have pros and cons.

[Previous page](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267.md?page=1)

[Next page](https://discourse.julialang.org/t/ann-setfield-jl-yet-another-package-for-working-with-immutables/9267.md?page=3)