Would attribute inference be possible?

@attributeInferredType Student end

function main()
    s1 = Student()
    s2 = Student()
    s1.age = 19
    s2.age = 20
    s1.grade = 3.7
    #Here, you have two instances of students, one with only attribute age,
    #And another one with grade.

How would this be used?
There could be many scenarios. Here is an example.
Let’s say I have a game with 2 mods, one for unit experience and one for morale.

function initialize_experiences(units)
    for unit in units
        unit.experience = 0
function initialize_morale(units)
    for unit in units
        unit.morale = get_max_morale(unit.typeid) #This is actually a global dictionary lookup.

Now, depending on what mods got loaded, the attributes would change!
Yes, you could use a dictionary but that’s inefficient.

1 Like

You can pull off ad-hoc “attributes” (fields?) by overloading Base.setproperty! and storing things in a Dict{String, Any}. But no, it’s not possible for the compiler to infer the types of these ad-hoc fields. The compiler only sees the Student type and the typed fields it has in its definition, and that can’t change at runtime with the ad-hoc fields. If you only have a fixed set of fields, you could have the type definition specify them all but store a sentinel value or nothing when a field is “irrelevant”. This is of course impractical if the set is large.

I think the repo is quiet for now, but you might look at @uniment 's WIP for some inspiration.

1 Like

Tradeoffs still apply there:

1 Like

Yes, but here’s the thing.

Could a compiler powerful enough to infer the attributes of each object be implemented?

I believe the answer is yes.

In the example above, the compiler just needs to scan, create “Unit” as an abstract type, and then assign s1 and s2 to be type “Unit1” and “Unit2” respectively. Next, you trace over the object lifetime and figure out what have been assigned.

I don’t understand, nowhere in your example do you mention a Unit type, and s1 and s2 seem like instances of Student. s1 and s2’s types can diverge in separate function calls. However something like

forces s1 and s2 to be instances of the same return type of Student() at runtime, and that can’t change no matter what fields you assign afterward. Julia compiles per call signature, so it can’t adapt the return type of Student() to subsequent code, either. Maybe a tracing JIT can do it, though I’ve never heard of a compiler that “undoes” dynamism like this because this becomes unstable if field assignments become conditional e.g. if s1.age < 18 s1.grade = 3.8 end

Oops… you’re right… it’s Student type.

Well, maybe not now… but my idea stands, to have dynamic attributes and then optimize away the dynamism by tracing.

This sort of thing is pretty easy to do today if you make the types immutable. I’m using Accessors.jl here for convenience, but it’d be possible without too.

using Accessors
using Accessors.ConstructionBase

struct Student{K, V}
    data::NamedTuple{K, V}
Student(;kwargs...) = Student(NamedTuple(kwargs))
getdata(s::Student) = getfield(s, :data)

Base.propertynames(s::Student) = propertynames(getdata(s))
Base.getproperty(s::Student, prop::Symbol) = getproperty(getdata(s), prop)
ConstructionBase.setproperties(s::Student, props::NamedTuple) = Student(merge(getdata(s), props))
let s1 = Student(), s2 = Student()
    s1 = @set s1.age = 19
    s2 = @set s2.age = 20
    s1 = @set s1.grade = 3.7
    @info("", propertynames(s1), propertynames(s2), typeof(s1), typeof(s2))
 ┌ Info: 
 │   propertynames(s1) = (:age, :grade)
 │   propertynames(s2) = (:age,)
 │   typeof(s1) = Student{(:age, :grade), Tuple{Int64, Float64}}
 └   typeof(s2) = Student{(:age,), Tuple{Int64}}

Everything can be statically inferred here, no dynamism needed:

code_warntype(()) do
    s1 = Student()
    s2 = Student()
    s1 = @set s1.age = 19
    s2 = @set s2.age = 20
    s1 = @set s1.grade = 3.7
    s1, s2
MethodInstance for (::var"#7#8")()
  from (::var"#7#8")() @ Main In[10]:2
  s2::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}}
  s1::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}, Student{(:age, :grade), Tuple{Int64, Float64}}}
Body::Tuple{Student{(:age, :grade), Tuple{Int64, Float64}}, Student{(:age,), Tuple{Int64}}}
1 ─       (s1 = Main.Student())
│         (s2 = Main.Student())
│   %3  = s1::Core.Const(Student{(), Tuple{}}(NamedTuple()))
│   %4  = Core.apply_type(PropertyLens, :age)::Core.Const(PropertyLens{:age})
│   %5  = (%4)()::Core.Const((@optic _.age))
│   %6  = (Accessors.opticcompose)(%5)::Core.Const((@optic _.age))
│   %7  = (identity)(%6)::Core.Const((@optic _.age))
│         (s1 = (Accessors.set)(%3, %7, 19))
│   %9  = s2::Core.Const(Student{(), Tuple{}}(NamedTuple()))
│   %10 = Core.apply_type(PropertyLens, :age)::Core.Const(PropertyLens{:age})
│   %11 = (%10)()::Core.Const((@optic _.age))
│   %12 = (Accessors.opticcompose)(%11)::Core.Const((@optic _.age))
│   %13 = (identity)(%12)::Core.Const((@optic _.age))
│         (s2 = (Accessors.set)(%9, %13, 20))
│   %15 = s1::Core.Const(Student{(:age,), Tuple{Int64}}((age = 19,)))
│   %16 = Core.apply_type(PropertyLens, :grade)::Core.Const(PropertyLens{:grade})
│   %17 = (%16)()::Core.Const((@optic _.grade))
│   %18 = (Accessors.opticcompose)(%17)::Core.Const((@optic _.grade))
│   %19 = (identity)(%18)::Core.Const((@optic _.grade))
│         (s1 = (Accessors.set)(%15, %19, 3.7))
│   %21 = Core.tuple(s1::Core.Const(Student{(:age, :grade), Tuple{Int64, Float64}}((age = 19, grade = 3.7))), s2::Core.Const(Student{(:age,), Tuple{Int64}}((age = 20,))))::Core.Const((Student{(:age, :grade), Tuple{Int64, Float64}}((age = 19, grade = 3.7)), Student{(:age,), Tuple{Int64}}((age = 20,))))
└──       return %21

But as @set reassigns a variable with a new type, the union of inferred types increases in size. I think it’s ~4 before it falls back to Any. The call-wise compiler can’t retroactively apply and keep only the last one, as Tarny_GG_Channie is suggesting.

Genius! It works in this case! However, I’m still in doubt when complicated stuff happens like in the “unit” example. Also, this was a case where everything turned out to be constant, which might have some benefit to type inference. I’d still have to say it’s a good work.

That’s just the binding s1, it doesn’t affect the compiled code at all. If you look at the actually typed code, every line is concretely inferred. Notice that none of the SSA slots are labelled s1, they’re all %1, %2, …, etc.

It’s maybe more clear if I show the code_typed:

code_typed(()) do
    s1 = Student()
    s2 = Student()
    s1 = @set s1.age = 19
    s2 = @set s2.age = 20
    s1 = @set s1.grade = 3.7
    s1, s2
 1 ─     return (Student{(:age, :grade), Tuple{Int64, Float64}}((age = 19, grade = 3.7)), Student{(:age,), Tuple{Int64}}((age = 20,)))
 ) => Tuple{Student{(:age, :grade), Tuple{Int64, Float64}}, Student{(:age,), Tuple{Int64}}}

So we see the compiler fully understands the body of the code here.

1 Like

Now that I look at the code_warntype printout more closely, the types of s1 and s2 between each reassignment are indeed concretely inferred. The compiler is only uncertain if you make the reassignments conditional, like in if statements and for loops:

julia> maybe::Bool = true

julia> code_warntype(()) do
           s1 = Student()
           s2 = Student()
           if maybe s1 = @set s1.age = 19 end
           if maybe s2 = @set s2.age = 20 end
           if maybe s1 = @set s1.grade = 3.7 end
           s1, s2
MethodInstance for (::var"#13#14")()
  from (::var"#13#14")() in Main at REPL[22]:2
  s2::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}}
Body::Tuple{Student, Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}}}
1 Like

It depends. The unit example isn’t very fleshed out. You’d just need to understand with this approach that every time you add a new property to your struct, its type changes. You then need to have a good mental model of what sorts of things the compiler is able to reason about, and what sorts of things it’s not able to reason about, and adjust accordingly.

For instance, your function stub initialize_experiences would encounter problems because e.g. a Vector{Unit{(), Tuple{}}} has no fields experience, so we would need to allocate a whole new vector if we encountered it. However, we can do a “mutate or widen” strategy where we only reallocate the vector when necessary.


Some initial setup:
using Accessors
using Accessors.ConstructionBase

function propertytype end
function propertytypes end
macro nt_struct(_Name::Symbol)
    Name = esc(_Name)
    mod = @__MODULE__()
        struct $Name{K, V}
            data::NamedTuple{K, V}
        $Name(;kwargs...) = $Name(NamedTuple(kwargs))

        getdata(s::$Name) = getfield(s, :data)
        Base.propertynames(s::$Name) = propertynames(getdata(s))
        Base.getproperty(s::$Name, prop::Symbol) = getproperty(getdata(s), prop)
        ConstructionBase.setproperties(s::$Name, props::NamedTuple) = $Name(merge(getdata(s), props))
        $mod.propertytype(::$Name{K,V}, s::Symbol) where {K,V} = $propertytype($Name{K, V}, s)
        $mod.propertytype(::Type{$Name{K,V}}, s::Symbol) where {K,V} = fieldtype(NamedTuple{K, V}, s)
        $mod.propertytypes(::$Name{K,V}) where {K,V} = $propertytypes($Name{K, V}, s)
        $mod.propertytypes(::Type{$Name{K,V}}) where {K,V} = fieldtypes(NamedTuple{K, V})

and then you can do things like

@nt_struct Unit
function initialize_experience!!(units::Vector{Unit{names, types}}) where {names, types}
    if :experience ∈ names
        units_out = units
        UnitType = Unit{(names..., :experience), Tuple{propertytypes(eltype(units))..., Int}}
        units_out = Vector{UnitType}(undef, length(units))
    for (i, unit) ∈ enumerate(units)
        units_out[i] = @set unit.experience = 0

This sort of strategy forms the backbone of packages like Transducers.jl, via a package called BangBang.jl, though it doesn’t have great support for this use-case. Maybe we should add a map!! function to BangBang which does this.

The idea here is that throughout the loop, we check if it’s possible to do

setindex!(units, @set(unit.experience = 0), i)

if units can accommodate that type, then we are able to do a simple setindex!, if not, then we need to re-allocate units with a wider type that has the :experience field. calling it something like map!!.

This is all type stable and inferrable. No tracing JIT, or other speculative/dubious modifications to the language necessary. Sure, it’s more work than using a regular mutable type with a fixed known amount of fields, but hey, sometimes you want to be able to change things around like this.