New parametric functions syntax has a readability problem

[essenciary]

I’ve been upgrading some of my code base to support the new parametric functions syntax – I’m using such functions extensively, with return types declarations. My feedback is that if this has been a step forward for the compiler, it’s also a step backward for humans.

1. the function declaration no longer flows naturally. Before you’d have the function name and arguments first and the return type at the end. Which made sense - you could glance at a function and quickly see the list of arguments and the expected return type. Now you have to mentally peel off the where part at the end of every function which adds a lot of noise.

2. you can no longer separate the arguments tuple from the return type with spaces. First, this is unexpected, because it works when you don’t have parametric functions but errors with parametric functions. Second, it makes the function declaration even harder to read, with everything gobbled up in one long line. Third, you end up with things like (i::Int)::Int which makes me dizzy.

3. EDIT: [I understand the situation will get even worse, by soon having to wrap function declarations in `(…)`]. Short form function with return types have to be wrapped in (...) – this is way too lispy for 2018. Estetically this will be Zune territory when everybody these days expects iPods.

4. Although at some level I find using the actual word “where” in a function definition mildly exotic, it has a SQL feeling to it that’s… unnatural. Why wasn’t something less verbose chosen, like “<=”, “<-” or whatever combination.

This is not a critique, just user feedback – but at first glance, this choice of syntax seems far than ideal. And although maybe it shouldn’t, language aesthetics do matter, especially for early adoption. Defining functions is something that users will do all the time and a lot. As a consequence, reading functions will be done even more often. These changes make the syntax less readable.

[Tamas_Papp]

Your editor’s highlighting should help with readability. If it doesn’t work to your satisfaction, you could tweak it or perhaps open an issue at the relevant support repo.

Regarding the specific points, (1) and (4) are very subjective (I find the syntax quite natural), I can’t replicate (2), eg

function foo(x::T) ::Float64 where T
    1.0
end

works fine for me.

Can you clarify (3) with an example?

[Evizero]

I don’t want to distract from your points, nor make some straw man argument, but I do think the following is worth bringing up to paint a more complete picture.

Looking through the code of Genie.jl, I think a big part of your experience is rooted in your very verbose type annotation convention.

For example you use signatures such as this without ever using U nor T in the method. And you do this in multiple places. I can appreciate your points under that light, but I would also argue that this is not the encouraged style for writing julia code.

In a more minimalistic style of coding I would even argue that convenience additions such as Array{<:Integer} improve the readability.

[essenciary]

That code is not representative - it’s just a refactored implementation where I omitted to update the declaration. Obviously, it doesn’t make sense to complicate the declarations with unused parametric types. I’ll update them, thanks for pointing that out.

What I had in mind are functions like the ones in https://github.com/essenciary/SearchLight.jl/blob/master/src/SearchLight.jl for example (though beware, the GitHub syntax highlighter messes things up here).

[essenciary]

Yes, I also believe that better editor support would make things better. It may be that my current theme is not very helpful. Also, a specialized Julia functions panel might help… I’m using VSCode with the Nord theme. Can anybody recommend any good Julia focused theme?

Re (2) I forgot to mention that it’s about short form functions - more details here: Proper way to declare return types parametrically in short-form functions in Julia 0.6

Re (1) and (3) they may be subjective, but the view reflects the decisions made by other languages in their implementation of generics. The previous syntax, where the types are added after the function name and before the argument tuple, was similar to choices made by other modern languages, like Swift and C#. Besides the fact that this represents the result of thorough language design choices, it’s also what developers have become accustomed to.

[tim.holy]

Echoing @Tamas_Papp, can you elaborate about point 3? I’m not sure what you’re referring to.

My feedback is that if this has been a step forward for the compiler, it’s also a step backward for humans.

I can understand what you mean, but I think there have been advances for humans too. Consider a = MyContainer{Float64}(5): this means I want to create an object of type MyContainer{Float64}, and I’m going to store the value 5 in it (and presumably I want to convert that 5 to 5.0). Prior to the syntax change, I’d have to define the constructor something like

(::Type{MyContainer{T}}){T}(x) = MyContainer{T}(convert(T, x))

With the new syntax, it’s just

MyContainer{T}(x) where T = MyContainer{T}(convert(T, x))

and MyContainer{T} here means the same thing that it does as when you call it in code.
It’s an even bigger win for types that have multiple parameters:

# The following is used for calls like Foo(a, b) and takes A and B from the types of a and b
Foo(a::A, b::B) where {A,B} = Foo{A,B}(a, b)
# The following is used for calls like Foo{Float64}(a, b)
Foo{A}(a, b::B) where {A,B} = Foo{A,B}(convert(A, a), b)
# The following is used for calls like Foo{Float64,Int}(a, b)
Foo{A,B}(a, b) where {A,B} = Foo{A,B}(convert(A, a), convert(B, b))

Though I wasn’t heavily involved in designing the new syntax, I think it’s fair to say that examples like these are the main reason for defining the parameters (via the where) at the end of the signature rather than at the beginning.

[essenciary]

@tim.holy Thanks

Re (3), here’s a quick made up example, requiring the function declaration to be wrapped in parenthesis:

julia> to_string(x::T) where T = string(x)
to_string (generic function with 1 method)

julia> to_string(x::T)::String where T = string(x)
ERROR: UndefVarError: T not defined

julia> (to_string(x::T)::String) where T = string(x)
to_string (generic function with 1 method)

(I’ve updated my (3) comment to reflect that wrapping in (...) is actually a requirement now. I was having in mind older discussions like Best practice for parametric return type declarations in v0.7 (and v1.0) and Proper way to declare return types parametrically in short-form functions in Julia 0.6).

Yes, I agree that the first convert example was very complicated and the new syntax made it much better. But even in the above examples, I can’t see why the parametric expression could not be provided as part of the argument list.

Say:

Foo(a::A%, b::B%)

or

Foo(a::A% <: AbstractType, b::B% <: OtherAbstractType)

where % stands for the where part.

[tim.holy]

Thanks for clarifying, I hadn’t followed that. I don’t use return type annotations often, not out of any principle though.

I suspect the main motivation for the parentheses would be something like

(foo(x::T)::T) where T<:AbstractFloat

which indicates that the return type is the same as the input arg, as opposed to

foo(x::AbstractFloat)::AbstractFloat

where you are not making such a guarantee. I don’t dispute that it seems like an extra burden in cases where you aren’t wanting to impose such a constraint.

As far as the % notation goes, I think the issue is that where does indeed provide a lisp-like level of specificity for the “level” at which a UnionAll operation occurs. For example:

julia> Tuple{Any, Any} <: Tuple{T where T, T where T}
true

julia> Tuple{Any, Any} <: Tuple{T, T} where T
false

In the first example, the where happens to each argument individually, so it doesn’t require the two parameters to be equal to another. In the second case, the tuple-type requires that both parameters are the same.

That said, I suppose your % notation could be made to work in conjunction with parentheses. Here, I don’t have any insights or opinions worth sharing, but perhaps others will chime in.

[essenciary]

The topic pops up here and there on discourse, so people have been bitten - another one: Return type assertions with type parameters
They didn’t generate a lot of feedback – I just stumbled into them as I got the error and looked for a solution.

Don’t know what trips the compiler. Some other thread was speculating operator precedence.

Yes, maybe it has to do mostly with my coding style - parametric short-form functions with return type annotations might not be all that common

[jeff.bezanson]

Yes, I agree this is an issue with short-form function definitions. Argument types, a return type, and a where clause is just too much stuff to easily fit on one line with clear syntax. I would recommend using the longer form in such cases.

@essenciary

Why wasn’t something less verbose chosen, like “<=”, “<-” or whatever combination.

As you said above, (i::Int)::Int already looks pretty confusing, and adding more punctuation would make things worse IMO. (i::A)::B <- {A <: C, B} is starting to get line-noise-ish.

[essenciary]

Hah, yes, I have to admit that in this case, where is actually more readable than <-.

[ScottPJones]

My only complaint now with where now simply with the verbosity, especially when displaying types, which I believe could be rectified. Here’s a very recent example:

(convert(::Type{T}, ::Type{STR})
 where {T<:AbstractString, STR<:Str{CSEnc,S,C,H}}
 where {CSEnc<:CSE{CS,E}, S, C, H}
 where {CS<:CharSet{ST}, E<:Encoding{ENC}}
 where {ST, ENC}) =
     T(string("Str{CSE{CharSet{",ST,",},Encoding{",ENC,"}},",S,",",C,",",H,"}"))

I’d appreciate a shorter way of saying this, would the following work? (i.e. does ; mean anything currently within a where clause?)

convert(::Type{T}, ::Type{STR}) where {
    T<:AbstractString, STR<:Str{CSEnc,S,C,H}; CSEnc<:CSE{CS,E}, S, C, H;
    CS<:CharSet{ST}, E<:Encoding{ENC}; ST, ENC} =
    T(string("Str{CSE{CharSet{",ST,",},Encoding{",ENC,"}},",S,",",C,",",H,"}"))

Or maybe there is some other, better way of handling a complicated type like this?
(it usually isn’t so complex in practice, I’m trying out a scheme of having optional fields, by having the last three type parameters be either something like Nothing, or UnitRange, or Vector{UInt8} or UInt64, to be able to have immutable struct types that contain the substring offset/length (or don’t take any space at all), a cached UTF-8 or UTF-16 representation (like Python 3 does), as well as optionally a precalculated hash value).

Displaying instances of these types looks kind of messy - although at least I’m able to define convert (to String) and show methods for the type itself, for the predefined ones).