Would attribute inference be possible?

[Tarny_GG_Channie]

@attributeInferredType Student end

function main()
    s1 = Student()
    s2 = Student()
    s1.age = 19
    s2.age = 20
    s1.grade = 3.7
    #Here, you have two instances of students, one with only attribute age,
    #And another one with grade.
end

How would this be used?
There could be many scenarios. Here is an example.
Let’s say I have a game with 2 mods, one for unit experience and one for morale.

function initialize_experiences(units)
    for unit in units
        unit.experience = 0
    end
end
function initialize_morale(units)
    for unit in units
        unit.morale = get_max_morale(unit.typeid) #This is actually a global dictionary lookup.
    end
end

Now, depending on what mods got loaded, the attributes would change!
Yes, you could use a dictionary but that’s inefficient.

[Benny]

@Tarny_GG_Channie

    s1.age = 19
    s2.age = 20
    s1.grade = 3.7
    #Here, you have two instances of students, one with only attribute age,
    #And another one with grade.

You can pull off ad-hoc “attributes” (fields?) by overloading Base.setproperty! and storing things in a Dict{String, Any}. But no, it’s not possible for the compiler to infer the types of these ad-hoc fields. The compiler only sees the Student type and the typed fields it has in its definition, and that can’t change at runtime with the ad-hoc fields. If you only have a fixed set of fields, you could have the type definition specify them all but store a sentinel value or nothing when a field is “irrelevant”. This is of course impractical if the set is large.

[mrufsvold]

I think the repo is quiet for now, but you might look at @uniment 's WIP for some inspiration.

GitHub - uniment/Objects.jl: Fast, Efficient, and Easy Objects for Julia

[Benny]

Tradeoffs still apply there:

Three subtypes of object are provided by storage technique: Dynamic, Static, and Mutable. Their external behaviors are identical, except for property-setting flexibility and performance.

• Dynamic: maximum flexibility—properties can be added or changed at any time, arbitrarily.
• Static: maximum performance—after construction, properties cannot be changed.
• Mutable: happy medium—properties can be changed at any time, but their types cannot change and new ones cannot be added.

…
Dynamic is very easy and casual to use, but unfortunately type instability causes lower performance. Good for hacking and playing, throwing stuff together before worrying about formalizing, but not so good for efficient runtime.

[Tarny_GG_Channie]

Yes, but here’s the thing.

Could a compiler powerful enough to infer the attributes of each object be implemented?

I believe the answer is yes.

In the example above, the compiler just needs to scan, create “Unit” as an abstract type, and then assign s1 and s2 to be type “Unit1” and “Unit2” respectively. Next, you trace over the object lifetime and figure out what have been assigned.

[Benny]

@Tarny_GG_Channie

In the example above, the compiler just needs to scan, create “Unit” as an abstract type, and then assign s1 and s2 to be type “Unit1” and “Unit2” respectively. Next, you trace over the object lifetime and figure out what have been assigned.

I don’t understand, nowhere in your example do you mention a Unit type, and s1 and s2 seem like instances of Student. s1 and s2’s types can diverge in separate function calls. However something like

@Tarny_GG_Channie

function main()
    s1 = Student()
    s2 = Student()

forces s1 and s2 to be instances of the same return type of Student() at runtime, and that can’t change no matter what fields you assign afterward. Julia compiles per call signature, so it can’t adapt the return type of Student() to subsequent code, either. Maybe a tracing JIT can do it, though I’ve never heard of a compiler that “undoes” dynamism like this because this becomes unstable if field assignments become conditional e.g. if s1.age < 18 s1.grade = 3.8 end

[Tarny_GG_Channie]

Oops… you’re right… it’s Student type.

Well, maybe not now… but my idea stands, to have dynamic attributes and then optimize away the dynamism by tracing.

[Mason]

This sort of thing is pretty easy to do today if you make the types immutable. I’m using Accessors.jl here for convenience, but it’d be possible without too.

using Accessors
using Accessors.ConstructionBase

struct Student{K, V}
    data::NamedTuple{K, V}
end
Student(;kwargs...) = Student(NamedTuple(kwargs))
getdata(s::Student) = getfield(s, :data)

Base.propertynames(s::Student) = propertynames(getdata(s))
Base.getproperty(s::Student, prop::Symbol) = getproperty(getdata(s), prop)
ConstructionBase.setproperties(s::Student, props::NamedTuple) = Student(merge(getdata(s), props))

let s1 = Student(), s2 = Student()
    s1 = @set s1.age = 19
    s2 = @set s2.age = 20
    s1 = @set s1.grade = 3.7
    propertynames(s1)
    propertynames(s2)
    @info("", propertynames(s1), propertynames(s2), typeof(s1), typeof(s2))
end

 ┌ Info: 
 │   propertynames(s1) = (:age, :grade)
 │   propertynames(s2) = (:age,)
 │   typeof(s1) = Student{(:age, :grade), Tuple{Int64, Float64}}
 └   typeof(s2) = Student{(:age,), Tuple{Int64}}

Everything can be statically inferred here, no dynamism needed:

code_warntype(()) do
    s1 = Student()
    s2 = Student()
    s1 = @set s1.age = 19
    s2 = @set s2.age = 20
    s1 = @set s1.grade = 3.7
    s1, s2
end

MethodInstance for (::var"#7#8")()
  from (::var"#7#8")() @ Main In[10]:2
Arguments
  #self#::Core.Const(var"#7#8"())
Locals
  s2::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}}
  s1::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}, Student{(:age, :grade), Tuple{Int64, Float64}}}
Body::Tuple{Student{(:age, :grade), Tuple{Int64, Float64}}, Student{(:age,), Tuple{Int64}}}
1 ─       (s1 = Main.Student())
│         (s2 = Main.Student())
│   %3  = s1::Core.Const(Student{(), Tuple{}}(NamedTuple()))
│   %4  = Core.apply_type(PropertyLens, :age)::Core.Const(PropertyLens{:age})
│   %5  = (%4)()::Core.Const((@optic _.age))
│   %6  = (Accessors.opticcompose)(%5)::Core.Const((@optic _.age))
│   %7  = (identity)(%6)::Core.Const((@optic _.age))
│         (s1 = (Accessors.set)(%3, %7, 19))
│   %9  = s2::Core.Const(Student{(), Tuple{}}(NamedTuple()))
│   %10 = Core.apply_type(PropertyLens, :age)::Core.Const(PropertyLens{:age})
│   %11 = (%10)()::Core.Const((@optic _.age))
│   %12 = (Accessors.opticcompose)(%11)::Core.Const((@optic _.age))
│   %13 = (identity)(%12)::Core.Const((@optic _.age))
│         (s2 = (Accessors.set)(%9, %13, 20))
│   %15 = s1::Core.Const(Student{(:age,), Tuple{Int64}}((age = 19,)))
│   %16 = Core.apply_type(PropertyLens, :grade)::Core.Const(PropertyLens{:grade})
│   %17 = (%16)()::Core.Const((@optic _.grade))
│   %18 = (Accessors.opticcompose)(%17)::Core.Const((@optic _.grade))
│   %19 = (identity)(%18)::Core.Const((@optic _.grade))
│         (s1 = (Accessors.set)(%15, %19, 3.7))
│   %21 = Core.tuple(s1::Core.Const(Student{(:age, :grade), Tuple{Int64, Float64}}((age = 19, grade = 3.7))), s2::Core.Const(Student{(:age,), Tuple{Int64}}((age = 20,))))::Core.Const((Student{(:age, :grade), Tuple{Int64, Float64}}((age = 19, grade = 3.7)), Student{(:age,), Tuple{Int64}}((age = 20,))))
└──       return %21

[Benny]

@Mason

s1::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}, Student{(:age, :grade), Tuple{Int64, Float64}}}

But as @set reassigns a variable with a new type, the union of inferred types increases in size. I think it’s ~4 before it falls back to Any. The call-wise compiler can’t retroactively apply and keep only the last one, as Tarny_GG_Channie is suggesting.

[Tarny_GG_Channie]

Genius! It works in this case! However, I’m still in doubt when complicated stuff happens like in the “unit” example. Also, this was a case where everything turned out to be constant, which might have some benefit to type inference. I’d still have to say it’s a good work.

[Mason]

@Benny

@Mason

s1::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}, Student{(:age, :grade), Tuple{Int64, Float64}}}

But as @set reassigns a variable with a new type, the union of inferred types increases in size.

That’s just the binding s1, it doesn’t affect the compiled code at all. If you look at the actually typed code, every line is concretely inferred. Notice that none of the SSA slots are labelled s1, they’re all %1, %2, …, etc.

It’s maybe more clear if I show the code_typed:

code_typed(()) do
    s1 = Student()
    s2 = Student()
    s1 = @set s1.age = 19
    s2 = @set s2.age = 20
    s1 = @set s1.grade = 3.7
    s1, s2
end

  CodeInfo(
 1 ─     return (Student{(:age, :grade), Tuple{Int64, Float64}}((age = 19, grade = 3.7)), Student{(:age,), Tuple{Int64}}((age = 20,)))
 ) => Tuple{Student{(:age, :grade), Tuple{Int64, Float64}}, Student{(:age,), Tuple{Int64}}}

So we see the compiler fully understands the body of the code here.

[Benny]

Now that I look at the code_warntype printout more closely, the types of s1 and s2 between each reassignment are indeed concretely inferred. The compiler is only uncertain if you make the reassignments conditional, like in if statements and for loops:

julia> maybe::Bool = true
true

julia> code_warntype(()) do
           s1 = Student()
           s2 = Student()
           if maybe s1 = @set s1.age = 19 end
           if maybe s2 = @set s2.age = 20 end
           if maybe s1 = @set s1.grade = 3.7 end
           s1, s2
       end
MethodInstance for (::var"#13#14")()
  from (::var"#13#14")() in Main at REPL[22]:2
Arguments
  #self#::Core.Const(var"#13#14"())
Locals
  s2::Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}}
  s1::Student
Body::Tuple{Student, Union{Student{(), Tuple{}}, Student{(:age,), Tuple{Int64}}}}
...

[Mason]

@Tarny_GG_Channie

It works in this case! However, I’m still in doubt when complicated stuff happens like in the “unit” example

It depends. The unit example isn’t very fleshed out. You’d just need to understand with this approach that every time you add a new property to your struct, its type changes. You then need to have a good mental model of what sorts of things the compiler is able to reason about, and what sorts of things it’s not able to reason about, and adjust accordingly.

For instance, your function stub initialize_experiences would encounter problems because e.g. a Vector{Unit{(), Tuple{}}} has no fields experience, so we would need to allocate a whole new vector if we encountered it. However, we can do a “mutate or widen” strategy where we only reallocate the vector when necessary.

E.g.:

Some initial setup:

using Accessors
using Accessors.ConstructionBase

function propertytype end
function propertytypes end
macro nt_struct(_Name::Symbol)
    Name = esc(_Name)
    mod = @__MODULE__()
    quote
        struct $Name{K, V}
            data::NamedTuple{K, V}
        end
        $Name(;kwargs...) = $Name(NamedTuple(kwargs))

        getdata(s::$Name) = getfield(s, :data)
        Base.propertynames(s::$Name) = propertynames(getdata(s))
        Base.getproperty(s::$Name, prop::Symbol) = getproperty(getdata(s), prop)
        ConstructionBase.setproperties(s::$Name, props::NamedTuple) = $Name(merge(getdata(s), props))
        $mod.propertytype(::$Name{K,V}, s::Symbol) where {K,V} = $propertytype($Name{K, V}, s)
        $mod.propertytype(::Type{$Name{K,V}}, s::Symbol) where {K,V} = fieldtype(NamedTuple{K, V}, s)
        $mod.propertytypes(::$Name{K,V}) where {K,V} = $propertytypes($Name{K, V}, s)
        $mod.propertytypes(::Type{$Name{K,V}}) where {K,V} = fieldtypes(NamedTuple{K, V})
    end
end

and then you can do things like

@nt_struct Unit
function initialize_experience!!(units::Vector{Unit{names, types}}) where {names, types}
    if :experience ∈ names
        units_out = units
    else
        UnitType = Unit{(names..., :experience), Tuple{propertytypes(eltype(units))..., Int}}
        units_out = Vector{UnitType}(undef, length(units))
    end
    for (i, unit) ∈ enumerate(units)
        units_out[i] = @set unit.experience = 0
    end
    units_out
end

This sort of strategy forms the backbone of packages like Transducers.jl, via a package called BangBang.jl, though it doesn’t have great support for this use-case. Maybe we should add a map!! function to BangBang which does this.

The idea here is that throughout the loop, we check if it’s possible to do

setindex!(units, @set(unit.experience = 0), i)

if units can accommodate that type, then we are able to do a simple setindex!, if not, then we need to re-allocate units with a wider type that has the :experience field. calling it something like map!!.

This is all type stable and inferrable. No tracing JIT, or other speculative/dubious modifications to the language necessary. Sure, it’s more work than using a regular mutable type with a fixed known amount of fields, but hey, sometimes you want to be able to change things around like this.