Help with where syntax in Julia v0.6

question

#1

I am trying to use the where syntax for the first time, what is wrong with the following usage?

abstract type AbstractPoint{N,T} end
abstract type AbstractDomain{P} end
abstract type AbstractGrid{P} <: AbstractDomain{P} end

immutable RegularGrid{P<:AbstractPoint{N,T} where {N,T}} <: AbstractGrid{P}
  dims::Dims{N}
  units::NTuple{N,T}
end

The error message says that N is undefined:

ERROR: UndefVarError: N not defined

List of most desired features for Julia v1.x
#2

My understanding is that you have to make all parameters part of the type signature, ie

immutable RegularGrid{N, T, P<:AbstractPoint{N,T}} <: AbstractGrid{P}
  dims::Dims{N}
  units::NTuple{N,T}
end

#3

If that is the case, I missed the point of having new syntax in the language. I was thinking Julia v0.6 introduced the where syntax to fix among other issues, that issue of nested parametric types. I don’t want a RegularGrid of N,T,P, I want a RegularGrid of points P, which already contain the dimension of the problem and the coordinate type.


#4

I might be wrong, but I briefly discussed this with someone in the gitter channel. The conclusion we reached is that the where syntax doesn’t work directly on type definitions, just to parametrize functions or defining new aliases for types.

Doesn’t your tyoe definition work if you just don’t put the where part there?


#5

It doesn’t work without the where either, I tried it since Julia v0.5 and was super happy that I would have it in Julia v0.6 with the introduction of where. Apparently not? :pensive:


#6

I believe it’s kind of a scoping issue, I ran into a similar issue myself on Saturday.

IIUC, the type variables N and T “declared” by the where clause that is part of “AbstractPoint{N,T} where {N,T}” are not visible outside of that “scope”, they are “local” in some sense, whereas you are trying to use them as type variables in the RegularGrid type.

In this case, I’d hoped something like:

abstract type AP{N,T} end
abstract type AD{P} end
abstract type AG{P} <: AD{P} end

(ntyp(::Type{A}) where A<:AP{N,T} where {N,T}) = T
(nval(::Type{A}) where A<:AP{N,T} where {N,T}) = N

struct RegularGrid{P<:AP} <: AG{P}
         dims::Dims{ntyp(P)}
         units::NTuple{nval(P),ntyp(P)}
       end

would work, but this runs into the issue (bug, in my estimation :slight_smile: ) that any expressions with type parameters are evaluated immediately, instead of when the type parameters are actually bound.


#7

Thank you @ScottPJones, I tried similar approach in the past, but I find it surprising that something like this doesn’t work out of the box. Not being able to manipulate the parameters of a type seems very limiting.

I hope someone can confirm that this is not possible to be achieved with where syntax or can suggest a fix that doesn’t involve defining accessor functions for every type.


#8

Right, currently field types can only depend on parameters of the type, and not parameters of the parameters. Adding the feature that @ScottPJones describes will provide a good workaround.

The problem is that getting parameters is more of a structural operation than a type operation. For example, what is the parameter of Complex in Union{Complex{Int}, Complex{Float32}}, or Union{Complex{Int}, Int}, or Complex{T} where T?


#9

I agree, but there are workarounds for the time being (although it will be a lot cleaner when this is fixed).
The improvements to the type system that are now released :clap::clap::clap: in v0.6.0 are absolutely great (but we are all greedy programmers, right? :grinning:)


#10

Hi @jeff.bezanson, will this feature be available in future releases?


#11

Yes I think we should add this feature at some point.


#12

@jeff.bezanson is there an issue on GitHub already tracking this intent? I can open one otherwise.


#13

#14

I feel that it’s worth observing that having field types depend only on the immediate parameters (typically?) ensures that they must have well-defined values. This is not the case if they are parameters of parameters, as demonstrated by Jeff’s examples. This isn’t necessarily a deal-breaker, but I think it does imply they will be hard to test that the package code behaves correctly in off-nominal cases.

Adding the feature that @ScottPJones describes will provide a good workaround.

I’m not sure we need to implement this in Base. I’ve already implemented the #18466 proposal as a package for anyone who wants the functionality: https://github.com/vtjnash/ComputedFieldTypes.jl
However, unlike other packages of its ilk (FastAnonymous, FunctionWrappers, etc), this functionality doesn’t seem particularly compelling to me to add to base.


#15

@jameson you mean the ability to use the types N and T directly in my example isn’t a good feature? I personally believe it is and in my humble opinion it should be considered part of the language. That pattern is quite common with any type of container type or abstract type storing metadata like the dimension of a problem. Having to define accessor functions everywhere seems too verbose and distracting.

If for some reason, this machinery slows things down or introduces issues, I’d like to learn more.


#16

Yes, there are ways you can work around it, however the current behavior can be considered a bug
(trying to evaluate unbound type parameters), and fixing this makes it more consistent and easier to understand for people learning the language.

I’d rather not see questions about “why doesn’t this work” continuously pop up on Gitter and Discourse.


#17

I agree. We could easily make this an error with a clear error message. The current behavior is mostly just an accident of not being more careful when lowering syntax to make sure the expression makes sense. If this is coming up often on discourse / gitter, that implies detecting the error would be helpful. Otherwise, it doesn’t seem particular urgent.

Right, I don’t think so. It’s possible that they don’t have a value at all (instead having none, multiple, or infinite values), so accessing them would need to throw some sort of error (UndefVar? MethodError?) or return some sort of sentinel object (TypeVar? Void?). This is pretty closely related to https://github.com/JuliaLang/julia/issues/21026. This probably isn’t much of an issue for the compiler (indeed, inference already has to be careful and try to handle cases like this correctly), but I think it would be more confusing for the user (confusing, unclear runtime errors instead of a fairly straightforward syntax correction to list the extra type parameters as required).


#18

You seem to not like at all the idea of doing what Jeff has said a few times makes sense and is doable, making a thunk and evaluating it only when the type parameters have all been bound.
Besides being what people expect, it also is more efficient than the workaround, which involves figuring out these parameters every time an instance of the type is constructed, instead of figuring things out when the concrete type is actually instantiated (i.e. when all of the parameters are bound).

Part of what makes Julia performant is doing as much as possible earlier, at compile-time, instead of leaving things to be done at run-time.


#19

This is shear and utter nonsense. You can only run functions during runtime. Inference constant folds expressions very frequently, which is what yields the better performance, not the type parameterization. I agree with Jeff that this is doable. I just think this assumption that the compiler somehow cares about whether https://github.com/vtjnash/ComputedFieldTypes.jl is a macro or builtin syntax is misinformed. And I think that changing the syntax lowering to give the ComputedFieldTypes.jl behavior instead of an error would just increase that confusion (the lowering changes are essentially the same in either case, although emitting an error is much easier to test).


#20

I have a parameterized type MyFloat{NBits}.
I create from that a concrete type MyFloat{256}.
Then I need to create 100 million of instances of these types.

At the time I create the type MyFloat{256} (i.e. the parameter is bound), I want to have it calculate that I need
4 64-bit unsigned limbs on a 64-bit platform, or 8 32-bit limbs on a 32-bit platform.
With the change that we’ve been talking about, those calculations would occur once, when the type was instantiated, instead of the current workaround of having to have the type actually have extra parameters, one for the number of limbs and another for the type of the limb, and to have the constructor perform the calculations (and that would be happening 100 million times, instead of just once).

Your ComputedFieldTypes has the same performance issue, AFAICT.