Optional parameters with default types in parametric structure constructors

Internals & Design
1

Hi folks,
I’m quite new to Julia (1.9) so please forgive me if this is a silly request or if I have overseen something. However, I’m currently facing serious troubles/incoveniences while implementing some parametric structures/objects. Especially, defining constructors which allow a flexible usage seems very cumbersome to me, where in my view, an easy solution would exist (or maybe I have overseen something which already exists).
Here, a small example of an actual structure which I want to implement with a ‘flexible’ constructor:

const DRK=Float64 # default real kind
const DIK=Int64 # default integer kind

EleType{RK}=Union{RK,AbstractArray{RK}} where {RK<:Real} # element type
MaxType{IK}=Union{IK,Missing} where {IK<:Integer} # integer or missing (=inf)

# (LAT)itude (VAL)ues
struct LatVal{ET<:EleType}
    u::ET # sin(th)
    t::ET # cos(th)
end
LatVal{ET}(th::ET) where {ET} = LatVal(sin.(th),cos.(th))
LatVal(th) = LatVal(sin.(th),cos.(th))

# FNAL (P)olynomials, (S)ectorial (ITER)ator
struct PSIter{ET<:EleType,IK<:Integer,MT<:MaxType{IK}}
    lVal::LatVal{ET}
    mMax::MT
    mMin::IK
    function PSIter{ET,IK,MT}(th::ET,mMax::MT=missing,mMin::IK=zero(IK)) where {ET<:EleType,IK,MT<:MaxType{IK}}
        new(LatVal(th),mMax,mMin)
    end
end
PSIter{ET,IK}(th::ET,mMax::MT,mMin::IK) where {ET<:EleType,IK,MT<:MaxType{IK}} =  PSIter{ET,IK,MT}(th,mMax,mMin)
PSIter{ET,IK}(th::ET,mMax::MT) where {ET<:EleType,IK,MT<:MaxType{IK}} =  PSIter{ET,IK,MT}(th,mMax)
PSIter{ET,IK}(th::ET) where {ET<:EleType,IK} =  PSIter{ET,IK,Missing}(th)
PSIter{ET}(th::ET,mMax,mMin::IK) where {ET<:EleType,IK} =  PSIter{ET,IK}(th,mMax,mMin)
PSIter{ET}(th::ET,mMax::IK) where {ET<:EleType,IK<:Integer} =  PSIter{ET,IK}(th,mMax)
PSIter{ET}(th::ET,mMax::Missing) where {ET<:EleType} =  PSIter{ET,DIK}(th,mMax)
PSIter{ET}(th::ET) where {ET<:EleType} =  PSIter{ET,DIK}(th)
PSIter(th::ET,mMax,mMin) where {ET<:EleType} =  PSIter{ET}(th,mMax,mMin)
PSIter(th::ET,mMax::IK) where {ET<:EleType,IK<:Integer} =  PSIter{ET}(th,mMax)
PSIter(th::ET,mMax::Missing) where {ET<:EleType} =  PSIter{ET}(th,mMax)
PSIter(th::ET) where {ET<:EleType} =  PSIter{ET,DIK}(th)

Note all the extra definitions which I need (12!) to allow omitting parameters and optional arguments. This makes the code very confusing and overloaded and I would have to do similar definitions for several other constructors. In my view, if one would allow optional parameters in the constructor, one could save ALL these extra definitions. I have something like this in mind:

# FNAL (P)olynomials, (S)ectorial (ITER)ator
struct PSIter{ET<:EleType,IK<:Integer,MT<:MaxType{IK}}
    lVal::LatVal{ET}
    mMax::MT
    mMin::IK
    function PSIter{ET=DRK,IK=DIK,MT=Missing}(th::ET,mMax::MT=missing,mMin::IK=zero(IK)) where {ET<:EleType,IK,MT<:MaxType{IK}}
        new(LatVal(th),mMax,mMin)
    end
end

With the (in my view correct) behavior:
[1. Use the specified parameter, if provided, try to convert actual arguments accordingly] > [2. Infer actual parameter from provided arguments] > [3. use the default parameter if not inferable]

Another things that bores me is that I’m not allowed to specify optional arguments in the extra definitions outside the inner constructor. Something like this is currently not working (in Julia 1.9):

PSIter{ET,IK}(th::ET,mMax::MT=missing,mMin::IK=zero(DIK)) where {ET<:EleType,IK,MT<:MaxType{IK}} =  PSIter{ET,IK,MT}(th,mMax,mMin)

So I have to write 3 definitions in my case (instead of one) to express optional arguments

What do you think about this? Have I missed something which already exists?

2

I think there is no need for this excessive amount of constructors. Instead, this should just work :tm:

const DRK=Float64 # default real kind
const DIK=Int64 # default integer kind

EleType{RK}=Union{RK,AbstractArray{RK}} where {RK<:Real} # element type
MaxType{IK}=Union{IK,Missing} where {IK<:Integer} # integer or missing (=inf)

struct LatVal{ET<:EleType}
  u::ET
  t::ET
end
LatVal(th) = LatVal(sin.(th),cos.(th))

struct PSIter{ET<:EleType,IK<:Integer}
    lVal::LatVal{ET}
    mMax::MaxType{IK}
    mMin::IK
    PSIter(th,mMax=missing,mMin=zero(DIK)) = new{typeof(th),typeof(mMin)}(LatVal(th),mMax,mMin)
end
1 Like
3

Thanks for your suggestions :slight_smile: . Yes, for the time beeing, maybe it is the best to omit the parameters altogether. However, then, I cannot explicitly state ET or IK anymore within the construct through parameters and I have to do possibly needed type conversions myself (at least if I understood Julias behavior correctly). Additionally, I think I need MT within the signature of PSIter to allow dynamic dispatch on it, e.g., for the Iterator property Base.IteratorSize() :

Base.IteratorSize(::Type{PSIter{ET,IK,Missing}}) where {ET,IK} = Base.IsInfinite()
Base.IteratorSize(::Type{PSIter{ET,IK,IK}}) where {ET,IK} = Base.HasLength()
4

Additionally, I think I need MT within the signature of PSIter to allow dynamic dispatch on it, e.g., for the Iterator property Base.IteratorSize() :

I guess you mean static dispatch (dynamic is the one where it needs to check at runtime and that is slow). Anyways, you are right, in that case you need the third parameter type. Instead try this

const DIK=Int64 # default integer kind

EleType{RK}=Union{RK,AbstractArray{RK}} where {RK<:Real} # element type
MaxType{IK}=Union{IK,Missing} where {IK<:Integer} # integer or missing (=inf)

struct LatVal{ET<:EleType}
  u::ET
  t::ET
end
LatVal(th) = LatVal(sin.(th),cos.(th))

struct PSIter{ET<:EleType,IK<:Integer,MK<:MaxType{IK}}
    lVal::LatVal{ET}
    mMax::MK
    mMin::IK
    function PSIter(th,mMax=missing,mMin=zero(DIK))
      if !ismissing(mMax)
        mMax, mMin = promote(mMax, mMin)
      end
      return new{typeof(th),typeof(mMin),typeof(mMax)}(LatVal(th),mMax,mMin)
    end
end

With this you can overload Base.IteratorSize as you wrote it above.

However, then, I cannot explicitly state ET or IK anymore within the construct through parameters and I have to do possibly needed type conversions myself (at least if I understood Julias behavior correctly).

You can still do

ulia> PSIter(1.0,missing,4)
PSIter{Float64, Int64, Missing}(LatVal{Float64}(0.8414709848078965, 0.5403023058681398), missing, 4)

julia> PSIter(1.0,UInt(4),4)
PSIter{Float64, UInt64, UInt64}(LatVal{Float64}(0.8414709848078965, 0.5403023058681398), 0x0000000000000004, 0x0000000000000004)

julia> PSIter(1.0,UInt(4))
PSIter{Float64, UInt64, UInt64}(LatVal{Float64}(0.8414709848078965, 0.5403023058681398), 0x0000000000000004, 0x0000000000000000)

julia> PSIter(1.0,missing)
PSIter{Float64, Int64, Missing}(LatVal{Float64}(0.8414709848078965, 0.5403023058681398), missing, 0)

EDIT: Apologize for all the edits, somehow I replaced all of that message instead of writing a new answer …

5

Yes, you’re right, it’s not ‘dynamic’ dispatch in the first place. It’s more like dispatch in general (I guess, it could still be dynamic if MT is not known beforehand).

I do not fully understand why I would need to check for ismissing() when I want to use Base.IteratorSize. Wouldn’t it be simpler to use the original constructor just without the parameters in the signature?:

struct PSIter{ET<:EleType,IK<:Integer,MT<:MaxType{IK}}
    lVal::LatVal{ET}
    mMax::MT
    mMin::IK
    function PSIter(th::ET,mMax::MT=missing,mMin::IK=zero(IK)) where {ET<:EleType,IK,MT<:MaxType{IK}}
        new{ET,IK,MT}(LatVal(th),mMax,mMin)
    end
end

This should do the promotion automatically if I understood it correctly. Btw., with manual type conversion I actually meant something like UInt64(4) as you proposed. I know, it doesn’t really hurt to do it this way. I just thought it would be more idiomatic to explicitly define the parameters of a parametric structure through the signature (since it is originally forseen by the language through the given syntax).

6

(I guess, it could still be dynamic if MK is not known beforehand)

Indeed, although that effect can be mitigated through function barriers.

Base.IteratorSize does not call the constructor and so does not care what you write in there. Instead it only looks at the object that has been returned from a constructor (the constructor is just an ordinary function). In fact, Base.IteratorSize just looks at the output of the typeof command applied to that instance of PSIter, e.g.

julia> it = PSIter(1.0)
PSIter{Float64, Int64, Missing}(LatVal{Float64}(0.8414709848078965, 0.5403023058681398), missing, 0)

julia> typeof(it) # that is the type of it, which is at this point known 'statically'
PSIter{Float64, Int64, Missing}

julia> typeof(it) isa Type
true

julia> Base.IteratorSize(typeof(it))
Base.IsInfinite()

This should do the promotion automatically if I understood it correctly.

No, Julia does not do automatic type promotions, see first section of this page Conversion and Promotion · The Julia Language

Btw., with manual type conversion I actually meant something like UInt64(4) as you proposed.

Well, I used UInt64(4) to have a case where MT, IK are different (because integer literals are by default of type Int64) in order to see that the promotion inside the constructor works.

EDIT: Had copied the wrong line for the Base.IteratorSize call.

7

Oh yeah, you are right, I just realized that. It just checks if both arguments are of the same type.
Phew! It gets more and more complicated: obviously, the default inner constructor new{...}(...) DOES type conversion according to the defined types (sorry, I always meant conversion while I said promotion) in presence of explicitly stated parameters {...}. Knowing this and knowing that you can define several customized inner constructors (where optional arguments work), I could reduce it to 7 constructors (taking also care of a few special cases which I oversaw at the beginning):

const DRK=Float64 # default real kind
const DIK=Int64 # default integer kind

EleType{RK}=Union{RK,AbstractArray{RK}} where {RK<:Real} # element type
MaxType{IK}=Union{IK,Missing} where {IK<:Integer}

# (LAT)itude (VAL)ues
struct LatVal{ET<:EleType}
    u::ET # sin(th)
    t::ET # cos(th)
end
LatVal{ET}(th) where {ET} = LatVal{ET}(sin.(th),cos.(th))
LatVal(th) = LatVal(sin.(th),cos.(th))
Base.convert(::Type{LatVal{EK}},th::EleType) where {EK} = LatVal{EK}(th)

# FNAL-(P)olynomials, (S)ectorial (ITER)ator
struct PSIter{ET<:EleType,IK<:Integer,MT<:MaxType{IK}}
    lVal::LatVal{ET}
    mMax::MT
    mMin::IK
    PSIter{ET,IK}(th,mMax,mMin=zero(IK)) where {ET,IK} = new{ET,IK,IK}(th,mMax,mMin)
    PSIter{ET,IK}(th,mMax::Missing=missing,mMin=zero(IK)) where {ET,IK} = new{ET,IK,Missing}(th,mMax,mMin)
    PSIter{ET}(th,mMax::Missing=missing,mMin::IK=zero(DIK)) where {ET,IK} =  PSIter{ET,IK}(th,mMax,mMin)
    PSIter{ET}(th,mMax::IK,mMin=zero(IK)) where {ET,IK<:Integer} =  new{ET,IK,IK}(th,mMax,mMin)
    PSIter(th::ET,x...) where {ET<:EleType} =  PSIter{ET}(th,x...)
    PSIter(th::LatVal{ET},x...) where {ET} =  PSIter{ET}(th,x...)
    PSIter(th::EleType{IK},x...) where {IK<:Integer} =  PSIter(LatVal(th),x...)
end

Base.IteratorEltype(::Type{PSIter}) = Base.HasEltype()
Base.eltype(::Type{PSIter{ET,IK,MT}}) where {ET,IK,MT} = ET
Base.IteratorSize(::Type{PSIter{ET,IK,Missing}}) where {ET,IK} = Base.IsInfinite()
Base.IteratorSize(::Type{PSIter{ET,IK,IK}}) where {ET,IK} = Base.HasLength()
Base.length(pSI::PSIter{ET,IK,IK}) where {ET,IK} = pSI.mMax-pSI.mMin+1

This code does now exactly what I expect it to do. It is much better as at the beginning and I think, I also learned a few things (thanks @fatteneder for the discussion). Though, I still believe having optional paramaters as initially proposed might help to simplify the shown code even further (to two or three constructors). The ability of automatic conversion through the default constructor is yet another reason why optional parameters could be useful…

EDIT: I found a small error in the constructors and I corrected it

8

Could you please explain why you think you need those 6 inner constructors?
I am pretty sure the version I posted above with only one constructor does the exact same thing and is less verbose … If you have an example that does not work, please share it.

Aside: You could also add a second inner constructor that dispatches on Missing, e.g.

struct PSIter{ET<:EleType,IK<:Integer,MK<:MaxType{IK}}
    lVal::LatVal{ET}
    mMax::MK
    mMin::IK
    function PSIter(th,mMax,mMin=zero(DIK))
      mMax, mMin = promote(mMax, mMin)
      return new{typeof(th),typeof(mMin),typeof(mMax)}(LatVal(th),mMax,mMin)
    end
    function PSIter(th,mMax::Missing,mMin=zero(DIK))
      return new{typeof(th),typeof(mMin),Missing}(LatVal(th),mMax,mMin)
    end
end

But the point is that (in this example) none of your (inner) constructors need to be parameterized in order to instaniate a PSIter{ET,IK,MT}, because all the necessary type info can be obtained from the constructor’s arguments (via typeof).

1 Like
9

As far as I can tell, @fatteneder has solved your problem with constructor proliferation, but I just wanted to chime in about a separate issue. I think you want to replace

EleType{RK}=Union{RK,AbstractArray{RK}} where {RK<:Real}

with

const EleType{RK} = Union{RK,<:AbstractArray{RK}} where {RK<:Real}

Note the <: within the union. Without it, the LatVal parameter remains an abstract type when u and t are arrays, preventing type inference.

That said, you may be better off orthogonalizing your design a bit, separating the concept of a latitude from a collection of latitudes. StructArrays.jl is often very handy in these cases. For example:

using StructArrays

struct Latitude{T<:Real}
    u::T
    t::T
end

Latitude(th) = Latitude(sin(th), cos(th))

latitudes(ths) = StructArray(Latitude.(ths))  # This can be optimized to avoid the intermediate allocation if you wish, see the StructArrays documentation

struct PSIter{T,L<:StructArray{Latitude{T}},...}
    l::L
    ...
end
3 Likes
10

Ok, here a few examples which perform implicit conversions in the virtue of the MyStruct{…}(…) syntax which are enabled by the additional constructors I defined above:

julia> PSIter{Float32}(2.0)
PSIter{Float32, Int64, Missing}(LatVal{Float32}(0.9092974f0, -0.41614684f0), missing, 0)

julia> PSIter{Float32}(2.0,0)
PSIter{Float32, Int64, Int64}(LatVal{Float32}(0.9092974f0, -0.41614684f0), 0, 0)

julia> PSIter{Float32,UInt16}(2.0,0)
PSIter{Float32, UInt16, UInt16}(LatVal{Float32}(0.9092974f0, -0.41614684f0), 0x0000, 0x0000)

julia> PSIter{Float32,UInt16}(2.0)
PSIter{Float32, UInt16, Missing}(LatVal{Float32}(0.9092974f0, -0.41614684f0), missing, 0x0000)

julia> PSIter{Float32,UInt16}(2.0,missing,2)
PSIter{Float32, UInt16, Missing}(LatVal{Float32}(0.9092974f0, -0.41614684f0), missing, 0x0002)

julia> PSIter{Float32}(2.0,missing,2)
PSIter{Float32, Int64, Missing}(LatVal{Float32}(0.9092974f0, -0.41614684f0), missing, 2)

julia> PSIter{Float32}(2.0,missing)
PSIter{Float32, Int64, Missing}(LatVal{Float32}(0.9092974f0, -0.41614684f0), missing, 0)

julia> PSIter{Float16}(2,missing)
PSIter{Float16, Int64, Missing}(LatVal{Float16}(Float16(0.909), Float16(-0.4163)), missing, 0)

I’m fully aware that the parametric constructors are not needed for instantiation and you could do the same with a non-parametric constructor. However, isn’t the syntax MyType{…}(…) meant to be used in this way? I think it’s a nice thing to be able to explicitly state types in this fashion.

11

Ok, now I see what you are trying to do.

Someone please correct me, but I think this not really the Julian way of writing code, at least rarely do you see the MyType{..}(...) syntax used in the wild to perform implicit conversions (you do see MyType{...}(...) used when there are parameteric types that can’t be inferred from the constructors arguments!).
Also: The fact that you have to do this with inner constructors is, I think, also a hint on that this is not Julian, because the MyType{...}(...) syntax was disallowed for functions (e.g. outer constructors) some time ago That is not correct, you can do that also with out constructors. The syntax MyStruct{...}(...) is just reserved to mean type application (cf. Reclaim parametric method syntax - #5 by Keno, How to correctly define and call templated/parametric methods using the new 'where' syntax).

However, you can still do what you want with only three (or four if you want to dispatch on ::Missing) inner constructors

const DRK=Float64 # default real kind
const DIK=Int64 # default integer kind

const EleType{RK}=Union{RK,AbstractArray{RK}} where {RK<:Real} # element type
const MaxType{IK}=Union{IK,Missing} where {IK<:Integer} # integer or missing (=inf)

struct LatVal{ET<:EleType}
  u::ET
  t::ET
end
LatVal(th) = LatVal(sin.(th),cos.(th))

struct PSIter{ET<:EleType,IK<:Integer,MK<:MaxType{IK}}
    lVal::LatVal{ET}
    mMax::MK
    mMin::IK

    function PSIter(th,mMax,mMin=zero(DIK))
      mMax, mMin = promote(mMax, mMin)
      return new{typeof(th),typeof(mMin),typeof(mMax)}(LatVal(th),mMax,mMin)
    end
    function PSIter(th,mMax::Missing,mMin=zero(DIK))
      return new{typeof(th),typeof(mMin),Missing}(LatVal(th),mMax,mMin)
    end

    PSIter{ET}(th,mMax=missing,mMin=zero(DIK)) where ET = PSIter(ET(th),mMax,mMin)
    PSIter{ET,IK}(th,mMax=missing,mMin=zero(DIK)) where {ET,IK} = PSIter(ET(th),mMax,IK(mMin))
end


Base.IteratorSize(::Type{PSIter{ET,IK,Missing}}) where {ET,IK} = Base.IsInfinite()
Base.IteratorSize(::Type{PSIter{ET,IK,IK}}) where {ET,IK} = Base.HasLength()
1 Like
12

Thanks for this hint. Well, I’m not in the position to debate about the Julian way since I’m very new to this language. If you tell me that using the MyType{...}(...) syntax is more or less deprecated then I think we can close this topic and I will opt for the ‘parameterless’ constructor(s) whenever possible.

One note on your solution: while it should do the same for most of the cases, the behavior is different for, e.g., PSIter{Float64,Int32}(0.5,2) since promote() would ignore IK then (since Int32 is simply promoted to Int64).

13

Indeed, that’s correct and I missed that.
Of course one could now fix that again with adding some more type computations and/or actually implement all the six constructors you had before. So that point is yours :slight_smile:

I got to say that it felt like this discussion turned into a code golfing challenge, probably because I misunderstood the initial question slightly. So perhaps you might have a point that there could be something added to the language that would simplify that, although I feel like this question must have popped up somewhere else already. It might also be that there is a simple trick I don’t know of.
I hope someone else more knowledgeable could chime in and clarify this.

To be clear, the syntax is not deprecated, it is only reserved for (inner) struct construction(I had this wrong before, will fix it above too) type application, so you should be good to use it for a while.
However, keep in mind that you won’t be able to do this for ordinary functions, unless you add extra boiler plate by defining a struct to get the implicit conversions going with the syntax you had in mind.