Syntax Surprises

[Chris_Foster]

@uniment

no no, as proposed above we might be required to have “excess” parentheses if we wish to have floating-point juxtaposition, in order to eliminate ambiguities.

Well I don’t think (2.5)y is an improvement over 2.5*y as it’s both longer and less explicit. So if the parens were disallowed in that case I’d be cool with it.

On the whole juxtaposition syntax is a little icky because it does introduce some syntax surprises. But I’d also forgotten about complex numeric “literals” and unitful quantities. These are cases where juxtaposition really is great.

@uniment

myfun((((((((((((((((1))))))))))))))))

I’m not arguing against this form (as ugly as it is). People should always be able use parentheses to group subexpressions, and this is just a degenerate example of that.

But in contrast, a keyword argument is about how the function is called, not just what is passed to the function: In some sense, the = keyword syntax is “part of” the function call brackets and should not be separated from them in some arbitrary deeply nested parens.

@DNF

I find the ambiguity acceptable, since 2e9 is so established. The case insensitivity is unfortunate, though.

I agree with this assessment. Case insensitivity is unfortunate and could possibly be addressed if we had a way to migrate syntax variants.

@uniment

Maybe the right thing to do is to focus this concern toward a Julia standard linter

Yes we should have good linting and these things should be easy to detect because they’re just syntax. JuliaSyntax.jl is very much being developed with tooling in mind.

[mikmoore]

I wouldn’t have a linter flag variables such as f or f1 outright, but it would be good to flag instances where the symbol exists and a numeric juxtaposition would otherwise be shadowed by a float literal (4f+1 or 4f1, for example).

[uniment]

@mikmoore

flag instances where the symbol exists and a numeric juxtaposition would otherwise be shadowed by a float literal

Ah, now this is the best of all possible worlds!

@Chris_Foster

we should have good linting

It’d be awesome to have a linter built-in to the package manager, so e.g. I could generate a lint report prior to an upcoming package release by writing:

]
lint MyPackage
test MyPackage

and maybe get a linter in the REPL. Then whenever someone is new to Julia and starts writing things like myVar = 5, they can immediately receive feedback that it’s unidiomatic to use camelCase and be chastised into snake_case, or when they write (2x + 4)/3*a they can receive feedback that * after / should have whitespace.

I like the idea of a linter giving hints and generating reports, but not [on its own] forcing adherence. This causes convergence on standard practice by nagging (the same way my car pretty quickly gets me to buckle my seatbelt by beeping), but you’re still free to break guidelines if it proves useful—you just have to be willing to put up with the nagging and the extra sign-offs, and other people (such as peers and managers) get better visibility into which guidelines you’re violating and can compel you into compliance if there’s no good reason to violate.

It could also be interesting to allow for linter error severity to be reduced, if the line before an error is used for a comment on why breaking the linter rule is justified there.

[uniment]

@Chris_Foster

the = keyword syntax is “part of” the function call brackets and should not be separated from them in some arbitrary deeply nested parens

I see what you’re saying now…

Function Abduction

julia> f(args...; kwargs...) = (; args, kwargs)
f (generic function with 1 method)

julia> f(((((a=1)))), ((((b=2)))), ((((c=3))));) # 🤔🦆
(args = (), kwargs = Base.Pairs(:a => 1, :b => 2, :c => 3))

julia> f(((((a=1)))), ((((b=2))),), ((((c=3))));) # 🦆🦆
(args = ((b = 2,),), kwargs = Base.Pairs(:a => 1, :c => 3))

It does feel like this ought to be a syntax error.

(also, Base.Pairs are kind of a WAT of their own, although it’s not a syntax surprise:)

julia> f(a=1)
(args = (), kwargs = Base.Pairs(:a => 1))

julia> Base.Pairs(:a => 1)
ERROR: MethodError: no method matching Base.Pairs(::Pair{Symbol, Int64})

[adienes]

if you get rid of all the superfluous parens it makes a lot more sense

julia> f(a=1, ((b=2),), c=3)
(args = ((b = 2,),), kwargs = Base.Pairs(:a => 1, :c => 3))

Where it becomes clear that the source of confusion is the fact that kwargs are allowed in non-final position (unlike python ). I do agree with you that any linter should be able to catch ‘obviously’ redundant parens, aka those which look like (( .. ))

Also I do not think the example of Base.Pairs(:a => 1) erroring is particularly problematic. Just consider

julia> typeof(:a => 1)
Pair{Symbol, Int64}

julia> Pair(:a => 1)
ERROR: MethodError: no method matching Pair(::Pair{Symbol, Int64})

It is certainly not always the case that a struct X will have a constructor X(::typeof(X))

[uniment]

@adienes

get rid of all the superfluous parens

One more to go:

f(a=1, (b=2,), c=3)

but there are actually four syntax surprises contained in that example:

1. It’s possible to have superfluous parens around keyword args.
2. It’s possible to have superfluous parens around NamedTuple fieldname-value pairs.
3. Keyword args are allowed in non-final position,
4. even if they are succeeded by a semicolon!

Would you expect this?

julia> f((a=1, b=2), (c=3); d=4)
(args = ((a = 1, b = 2),), kwargs = Base.Pairs(:c => 3, :d => 4))

@adienes

not always the case that a struct X will have a constructor X(::typeof(X))

That’s not at issue here. At issue is a break from Julia’s convention that, where reasonable, an object’s show method should print working code which illustrates how to construct the object.

Obviously Julia doesn’t print working code for vectors, matrices, and functions, but it’s often unreasonable to try, and at least for those it’s pretty obvious that what has been printed isn’t executable code.

The show method for Base.Pairs here has two WATs:

1. it presents text that very much resembles a call to its constructor, but it’s not, and
2. in this context it promotes the idea that Pairs of key => value is a preferred way to package and send keyword arguments around, when infact that causes type-instability (see for example, here).

[mikmoore]

@adienes

I do not think the example of Base.Pairs(:a => 1) erroring is particularly problematic.

When feasible, Julia tends to display objects in an input-compatible way, like in

julia> (rand(1,1),)
([0.5278893349360236;;],)

Note the trailing ;; to indicate that this is a matrix and to allow it to be pasted as valid code.

So when I see something print as Base.Pairs(:a => 1), I suspect I’ll be able to type that into the REPL and get a valid object. It’s certainly not required, but as I said Julia tends towards this where possible. I’d have stronger feelings about this particular instance if Base.Pairs wasn’t a construct used exclusively by the internals of kwarg functions.

[StefanKarpinski]

Yeah, as much as syntax conflicts like this are conceptually annoying and make for great syntax “wat” examples, this just doesn’t seem like it’s actually a big issue in practice and so isn’t worth the churn or breakage to eliminate wats. Sometimes I wear more of a “what if we did this?” hat; but other times I have to put a more classic, boring hat, which defaults to leavings alone that are good enough.

[stevengj]

That being said, I agree with comments above that obfuscated syntax is absolutely something that would be good to check in a linter, e.g. give a warning if the user has a variable named f0 in the same scope that uses single-precision literals like 2f0.

(One of the reasons that this doesn’t show up very often, I suspect, is that production Julia code doesn’t use single-precision literals all that often — if you want your code to handle single precision, then typically you want it to handle all precisions, and you write precision-independent literals based on the types of the function arguments.) Not really relevant since the hypothetical confusion is in the opposite direction (but requires variables named e or f and for you to write something like 2e-1 without noticing it looks like a literal).

[uniment]

@StefanKarpinski

Sometimes I wear more of a “what if we did this?” hat

I would like to express that I appreciate this. We can’t be confident we’ve found an optimal solution if we don’t occasionally entertain the possibility that we haven’t.

I very much like numeric literal juxtaposition; the improved readability reduces error rate; it’s simple, intuitive, and hard to misinterpret. Most of the WATs are only encountered if you go looking for trouble.

My concern is, I wouldn’t want to pick the language that solved the interpreted vs. compiled two-language problem, only to run into another two-language problem a few years later (small-scale scripts vs. production code). Numeric juxtaposition is hard to get wrong, except for ambiguity over e, E, and f which raises the possibility of runtime black swans; sooner or later some knucklehead on the team is bound to mess it up (probably me) and ruin the party. I’d want assurance that, on a risk-adjusted basis, I’m not paying a penalty for the feature.

A linter solves this. I don’t need it right now, but having it on the roadmap puts my unease to rest: that when the time comes for a project to scale and a bunch of under-supervised 18-year-olds start getting thrown at it, or when Elon decides to use Julia for FSD, automated checks can help steer them away from trouble.

[uniment]

@stevengj I’d also like to express gratitude for steadfastly holding your ground and forcing us to figure out how to make it work the way it is.

[klacru]

Surprise - can we rely on that?

julia> foo((a,b...)...) = [a, b]
foo (generic function with 1 method)

julia> foo((1,2,3), (4,5,6), (7,8))
2-element Vector{Tuple{Any, Any, Vararg{Int64}}}:
 (1, 2, 3)
 ((4, 5, 6), (7, 8))

[giordano]

@klacru

can we rely on that?

Yes.

[klacru]

I can’t see the example matches the manual. Over there we don’t have the double ellipsis.

[giordano]

Did you miss the sections about varargs functions and destructuring assignment and multiple return values?

[klacru]

No. I don’t get it. My example behaves like foo(a, b...) = [a, b]. There is no example or rule explaining the syntax I used.

[uniment]

Hah, that’s a fun one. It’s conceptually similar to having a bunch of extraneous parentheses, with this weird exception:

Function Abduction (Part Deux)

julia> f(((((x,)...,)...,)...,)...) = x+27
f (generic function with 1 method)

julia> f(15)
42

julia> f(42, "where", :do, `these`, r"go?!?")
69

It’s also handy for masking the method signature, to prevent people from knowing the argument names:

julia> methods(f)
# 1 method for generic function "f" from Main:
 [1] f(...)
     @ REPL[1]:1

Here’s another fun one. Apparently underscore _ is allowed as a fieldname for now

Under Scores Under Stress

julia> Main._ = "hi!"
"hi!"

julia> hellos = (a = "hello!", b = "hola!", _ = "hallo!")
(a = "hello!", b = "hola!", _ = "hallo!")

julia> (; b, _, a) = hellos  # destructure syntax
(a = "hello!", b = "hola!", _ = "hallo!")

julia> @show Main._
       @show a
       @show b
       @show _
Main._ = "hi!"
a = "hello!"
b = "hola!"
ERROR: syntax: all-underscore identifier used as rvalue around show.jl:1128

And another. The surprise in this one is the unprivileged treatment of a local function’s name despite using named function syntax, and it should not be possible for performance reasons, but it’s a fun brain teaser for now:

Method Impossible

julia> const f = let start=4, c=start+1, f(x)=x
           f() = if f(c-=1)==start; "Your mission, should you choose to accept it. 📦"
                 elseif c>0; "This message will self-destruct in $c times. "*"🕐🕑🕒"[4c-3]
                 else f=nothing; "Self-destructing! 🔥" end
       end;

julia> println(f())
       println(f())
       println(f())
       println(f())
       println(f())
       println(f())
Your mission, should you choose to accept it. 📦
This message will self-destruct in 3 times. 🕒
This message will self-destruct in 2 times. 🕑
This message will self-destruct in 1 times. 🕐
Self-destructing! 🔥
ERROR: MethodError: objects of type Nothing are not callable

[sijo]

@klacru

My example behaves like foo(a, b...) = [a, b]. There is no example or rule explaining the syntax I used.

With the declaration foo(x...), you get all the arguments in a tuple called x (the vararg feature described in the doc). When you write foo((a,b...)...) you effectively replace x with (a,b...). This is a destructuring assignment, of the kind described at the end of this section. You get the first value of x in a and the rest in b. (And as @uniment showed, you can also write (a,) to get the first value and discard the rest.)

[klacru]

@sijo

With the declaration foo(x...), you get all the arguments in a tuple called x (the vararg feature described in the doc).

Ok, that is what I read from the doc.

@sijo

When you write foo((a,b...)...) you effectively replace x with (a,b...).

There is not mention of effective replacement in the docs, only ellipsis after a variable appears there.
Also foo((a,b)...) is not effectivly replaced by foo(a, b).

[klacru]

@sijo

And as @uniment showed, you can also write (a,)

Yes. You can do quite a lot of things, which are not documented; and then guess what is the semantics behind that. That is not like it should be!