Julia Type system manual confusions

[sgaure]

@raman_kumar

These first three gives error.

primitive type Float16 <: AbstractFloat 16 end
primitive type Float32 <: AbstractFloat 32 end
primitive type Float64 <: AbstractFloat 64 end

The whole list in there has been cut from the file boot.jl which is the file which sets up the standard types. You can start an editor on that file with:

julia> @edit Nothing()
...
abstract type Number end                                                                                      
abstract type Real     <: Number end                                                                          
abstract type AbstractFloat <: Real end                                                                       
abstract type Integer  <: Real end                                                                            
abstract type Signed   <: Integer end                                                                         
abstract type Unsigned <: Integer end                                                                         
                                                                                                              
primitive type Float16 <: AbstractFloat 16 end                                                                
primitive type Float32 <: AbstractFloat 32 end                                                                
primitive type Float64 <: AbstractFloat 64 end                                                                
                                                                                                              
primitive type BFloat16 <: AbstractFloat 16 end                                                               
                                                                                                              
#primitive type Bool <: Integer 8 end                                                                         
abstract type AbstractChar end                                                                                
primitive type Char <: AbstractChar 32 end                                                                    
                                                                                                              
primitive type Int8    <: Signed   8 end                                                                      
#primitive type UInt8   <: Unsigned 8 end                                                                     
primitive type Int16   <: Signed   16 end                                                                     
#primitive type UInt16  <: Unsigned 16 end                                                                    
#primitive type Int32   <: Signed   32 end                                                                    
#primitive type UInt32  <: Unsigned 32 end                                                                    
#primitive type Int64   <: Signed   64 end                                                                    
#primitive type UInt64  <: Unsigned 64 end                                                                    
primitive type Int128  <: Signed   128 end                                                                    
primitive type UInt128 <: Unsigned 128 end                                                                    

Those that are commented out have been predefined in the C-code of julia because they are needed before the boot.jl starts (for some parsing, errors during parsing, etc.). While you can replace most definitions in julia, some low level things can’t be replaced.

[raman_kumar]

doc/src/manual/types.md?plain=1

while

julia> Int64 <: Real
true

julia> abstract type Pointy{T<:Real} end
julia> Pointy{Int64}
Pointy{Int64}
julia> Pointy{1}
ERROR: TypeError: in Pointy, in T, expected T<:Real, got a value of type Int64
julia> Pointy(1)
ERROR: MethodError: no method matching Pointy(::Int64)

Why it gives error?

[adienes]

because 1 is a value, not a type

[sgaure]

@raman_kumar

Why it gives error?

The parameter has been specified to be a type which is a subtype of Real. Int64 is a subtype of real, but 1 is not. 1 isn’t even a type, it’s an object of type Int64.

julia> Int64 <: Real
true

julia> 1 <: Real
ERROR: TypeError: in <:, expected Type, got a value of type Int64

julia> 1 isa Int64
true

julia> 1 isa Real
true

julia> Int64 isa Real
false

[raman_kumar]

doc/src/manual/types.md?plain=1

julia> abstract type Pointy{T} end

julia> Pointy{1} <: Pointy
true

julia> Pointy{1}
Pointy{1}

gives true
while

julia> abstract type Pointy{T<:Real} end
julia> Pointy{1}
ERROR: TypeError: in Pointy, in T, expected T<:Real, got a value of type Int64

[sgaure]

Yes, if you don’t constrain the parameter to be a type, it can also be Ints, Floats and other objects of bits type.

julia> a = (1, 2.3, pi)
(1, 2.3, π)

julia> Pointy{a}
Pointy{(1, 2.3, π)}

julia> isbits(a)
true

julia> Pointy{1.0 + 3im}
Pointy{1.0 + 3.0im}

This mechanism is used to e.g. make arrays of various dimension:

julia> Array{Float64,1}
Vector{Float64} (alias for Array{Float64, 1})

julia> Array{Float64,2}
Matrix{Float64} (alias for Array{Float64, 2})

[raman_kumar]

doc/src/manual/types.md?plain=1

Oh Thanks, so parameter T can be even a value. But what is their use i.e. T=3.4 etc values ?

[sgaure]

@raman_kumar

Oh Thanks, so parameter T can be even a value. But is their use T=3.4 etc values ?

Yes, the parameter can be values of some “simple” types called bits types. I.e. Float, Int, Char and tuples of such like (1, 2). Not e.g. a string "foo" or vector [1, 2]. In general you have Pointy{a} <: Pointy, but you don’t have Pointy{a} <: Pointy{b} if a is different from b.

[raman_kumar]

So here Type don’t work as constructor.

doc/src/manual/types.md?plain=1

julia> abstract type Pointy{T} end

julia> Pointy(1)
ERROR: MethodError: no method matching Pointy(::Int64)

[eldee]

@raman_kumar

But what is their use i.e. T=3.4 etc values ?

By directly informing the compiler of the value, you might get faster runtimes (at the expense of longer compilation). This only makes sense when you have only a small number of possible values, as you don’t want to be constantly recompiling.

To give a very artificial example, suppose you have a system determined by one of two constant 2 \times 2 matrices, and an exponent of \pm \pi. Compare

using StaticArrays, BenchmarkTools

struct S1{M, E} end  # i.e. Val{(M, E)}
struct S2
    matrix::SMatrix{2, 2, Float64, 4}
    exponent::Float64
end

function perform_computations(::S1{M, E}) where {M, E}
    return prod(t -> t^E, M)  # (Doesn't really use the matrix structure of M, but whatever)
end

function perform_computations(s2::S2)
    return prod(t -> t^s2.exponent, s2.matrix)
end

matrix1 = @SMatrix [1. 2.
                    3. 4.]
exponent2 = -3.1415

s1 = S1{matrix1, exponent2}()
s2 = S2(matrix1, exponent2)

@btime perform_computations($s1)  #   1.200 ns (0 allocations: 0 bytes), 4.6138809473272176e-5
@btime perform_computations($s2)  #  57.548 ns (0 allocations: 0 bytes), 4.6138809473272176e-5

The ::S1{M, E} version is faster after the first run, because the actual computation has been compiled away:

julia> @code_llvm perform_computations(s1)
; Function Signature: perform_computations(Main.S1{StaticArraysCore.SArray{Tuple{2, 2}, Float64, 2, 4}(data=(1, 3, 2, 4)), -3.1415})
;  @ REPL[4]:1 within `perform_computations`
; Function Attrs: uwtable
define double @julia_perform_computations_9311() #0 {
top:
  ret double 0x3F0830A56C0F92CD
}

So in very particular cases such an approach might be useful for performance-critical code. Examples of using Int values as type parameters for better performance are much easier to find in practice, like the SMatrix above.

[eldee]

Well, abstract types by definition cannot be instantiated, so it should come as no surprise that there is no constructor.

[Vasily_Pisarev]

That’s because you can’t instantiate an abstractly-typed objects.

You need something like

struct PointyInstance{T}<:Pointy{T}
    x::T
end

Then you can call constructor PointyInstance(1) and optionally define external constructors

function Pointy{T}(x) where {T}
    return PointyInstance{T}(x)
end

function Pointy(x)
    return PointyInstance(x)
end

[sgaure]

@raman_kumar

So here Type don’t work as constructor.

If you want a constructor for an abstract type, you must make it yourself. And you can’t instantiate an abstract type, you must let the constructor make some other type, e.g. like this:

abstract type Pointy{T} end

(::Type{Pointy})() = 7
(::Type{Pointy{n}})() where {n} = (n / 2, 2n)

julia> Pointy()
7

julia> Pointy{23}()
(11.5, 46)

There are already some such constructors around, e.g. for AbstractFloat, which defaults to construct a Float64:

julia> AbstractFloat(2)
2.0

The AbstractFloat constructors are defined like:

AbstractFloat(x::Bool)    = Float64(x)
AbstractFloat(x::Int8)    = Float64(x)
AbstractFloat(x::Int16)   = Float64(x)
AbstractFloat(x::Int32)   = Float64(x)
AbstractFloat(x::Int64)   = Float64(x)
AbstractFloat(x::Int128)  = Float64(x)
AbstractFloat(x::UInt8)   = Float64(x)
AbstractFloat(x::UInt16)  = Float64(x)
AbstractFloat(x::UInt32)  = Float64(x)
AbstractFloat(x::UInt64)  = Float64(x)
AbstractFloat(x::UInt128) = Float64(x)

Type @edit AbstractFloat(0) to view them in an editor

[raman_kumar]

Here in 1197 line. julia/doc/src/manual/types.md at e986983706bc6cad31400724be7e6804e285b913 · JuliaLang/julia · GitHub

Names and types for functions defined at different locations are distinct, but not guaranteed to be printed the same way across sessions.

• Please explain this line especially word distinct. If the names and types are unique then why they are not printed same way?

Line 1291 given below is also tough to grasp.

doc/src/manual/types.md?plain=1

[Vasily_Pisarev]

@raman_kumar

Please explain this line especially word distinct. If the names and types are unique then why they are not printed same way?

Basically, each defined anonymous function has its unique type but they may not be the same if you run the same program multiple times, or in different environments, or using a different Julia version.

As an example, the following is not guaranteed to print the same on each run:

$ julia -e "foo = x -> x^2; show(foo)"

@raman_kumar

Line 1291 given below is also tough to grasp.

Earlier in that section there is basically a disclaimer that it would be non-obvious.

Until we discuss Parametric Methods and conversions, it is difficult to explain the utility of this construct

If you get to type promotion and conversion in your own code, you’ll get that part, otherwise you may just leave it off for now.

[raman_kumar]

doc/src/manual/methods.md?plain=1

Since types of 1 and 2 in mysum(1, 2) will be known at run time then how can compiler can do method look up ahead of time?

Here types of x and y are subtypes of ::Real .
mysum(x::Real, y::Real) = x + y

• Does compiler compiles method code for all possible combinations of x and y types before run-time?
• and at run-time simply inlines/inserts compiled code matching combination of x and y types?

[nsajko]

@raman_kumar

Does compiler compiles method code for all possible combinations of x and y types before run-time?

No. That would not be possible, anyway.

There are some developer docs on Julia’s type inference:

• Inference · The Julia Language

Also some blog post links in there.

However, I would suggest opening separate topics for further questions.

[Benny]

@raman_kumar

Since types of 1 and 2 in mysum(1, 2) will be known at run time then how can compiler can do method look up ahead of time?

Runtime types can be known at compile-time as well. This is not particular to Julia; method JIT-compiled implementations of languages generally trigger small compilation phases after a runtime call provides runtime values. It could also be triggered before a call with values if their types were manually specified e.g. Julia’s precompile.

Even AOT-compilation phases can infer runtime types from literal values or constant expressions and dispatch methods over them, maybe even execute those methods at compile-time. Recall what I said earlier using a Java example:

@Benny

The method a.speak() is defined to dispatch over the runtime type, so Cat() meows and Dog() barks. When these method calls are dispatched depends on implementation. In this simple case, a compiler can actually infer that a is first a Cat() then a Dog(), thus statically dispatch their respective speak() calls, a kind of devirtualization.

There’s a fair, if wrong, argument to be made here that those runtime types are literally compile-time types as well. Julia’s type inference can figure out all the types and method dispatches of many practical programs at compile-time; that’s the point of type stability. However, type inference is just an implementation feature, and it does sometimes identify diverging, more abstract types at compile-time. Ultimately on the level of the language specification, only runtime/dynamic types exist. This is a common stumbling block for people because the accessible information about the difference between statically typed and dynamically typed languages don’t account for the specification/implementation distinction and oversimplify the difference to the point of making stereotypes. I’m almost motivated to writing a separate post about Julia not having static types.

[sgaure]

@raman_kumar

• Does compiler compiles method code for all possible combinations of x and y types before run-time?
• and at run-time simply inlines/inserts compiled code matching combination of x and y types?

It can’t be the first, since all possible types are not known at compile time. Consider the code snippet

function f(x, y)
    x + sin(y)
end
a1 = f(0, 1)
a2 = f(a1, 1.0)

The function f contains two function calls, a call to sin and a call to +. When the compiler encounters the call f(0, 1), it looks up the method table for f to see if it can be called with two Ints. Indeed, there’s a method where both x and y can be anything, i.e. Any. The compiler fetches this definition, finds the the function calls inside, and checks whether sin can be called with an Int:

julia> methods(sin)
...
 [10] sin(x::Real)

Sure there is. It looks inside. The first thing that happens there is xf = float(x), then the call sin(xf). It checks if float can be called with an Int. Sure. It will return a Float64. It checks if sin can be called with a Float64. Sure, it will return a Float64. It now knows that the call sin(y) inside f will return a Float64.

Then there’s the + call inside f. It will be called with x which is an Int, and a Float64. It checks if + can be called with an Int and a Float64, sure there is:

julia> methods(+, Tuple{Int,Float64})
# 1 method for generic function "+" from Base:
 [1] +(x::Number, y::Number)

It looks inside, and figures out that here is another float(x) inside there somewhere. The + call with an Int and a Float64 will convert the x to a Float64 and call Core.Intrinsics.add_float, with two Float64, which returns a Float64.

So, f(0, 1) will return a Float64. It now compiles f, including + and sin, for two Int inputs, knowing the types of everything that goes on inside. And it knows that a1 will be a Float64.

Now, the compiler finds the f(a1, 1.0). It now goes through the same steps for f, but now f is called with two Float64s. It compiles f for these inputs also. The results of the compilation is saved in the function definition of f, and can be reused when further calls are encountered:

julia> mt = methods(f)
# 1 method for generic function "f" from Main:
 [1] f(x, y)

julia> m = mt[1]
f(x, y)

julia> m.specializations
svec(MethodInstance for f(::Int64, ::Int64), MethodInstance for f(::Float64, ::Float64), nothing, nothing, nothing, nothing, nothing)

So, the next time the compiler encounters a call to f, it checks if there is already a MethodInstance for the argument types inside the specializations list.

Now, this is all fine in this example. The type of everything can be inferred from the type of the input arguments. Sometimes this is not the case, e.g. here, where put things inside the function h

g(x) = x < 0 ? 0 : x
function h(x,y)
    a1 = f(x, y)
    a2 = f(a1, 1.0)
    a3 = g(a2)
    a3 + a2
end
h(0, 1)

Now, a2 is a Float64, but the compiler doesn’t get to see the value of a2, only the type. It can’t figure out what type g will return. It’s either 0, which is an Int, or x, which is a Float64. When it subsequently encounters a3 + a2, it does not know what types this + will be called with, it’s either an Int and a Float64, or two Float64s. It must insert code for handling both cases. I.e. it must do “dynamic dispatch”, which slows down things considerably. (Though, if there are only two cases like here it has a quick way to handle this, called “union splitting”). And in this case it figures out that h(0,1) anyway returns a Float64:

julia> @code_warntype h(0,1)
MethodInstance for h(::Int64, ::Int64)
  from h(x, y) @ Main REPL[74]:1
Arguments
  #self#::Core.Const(Main.h)
  x::Int64
  y::Int64
Locals
  a3::Union{Float64, Int64}        # The type of a3 is not fully inferred
  a2::Float64
  a1::Float64
Body::Float64
...

We can make an even worse example:

g(x) = x < 0 ? x < -1 ? x < -2 ? Int8(0) : Int16(0) : 0 : x
function h(x,y)
           a1 = f(x, y)
           a2 = f(a1, 1.0)
           a3 = g(a2)
           a3 + a3
end
julia> @code_warntype h(0,1)
MethodInstance for h(::Int64, ::Int64)
  from h(x, y) @ Main REPL[122]:1
Arguments
  #self#::Core.Const(Main.h)
  x::Int64
  y::Int64
Locals
  a3::Union{Float64, Int16, Int64, Int8}
  a2::Float64
  a1::Float64
Body::Any

Now, the return type of g is one of four different types. Thus a3 can have any of them, it is too many for doing union splitting for the a3 + a3, and nothing is inferred about the return type of h(0,1), i.e. it’s inferred as Any.

The short answer to your question is that the function will be compiled when it’s actually called, when the types of the arguments are known. But often the compiler can do this in advance because it can infer the types of the arguments, like with the sin and + inside f above.