New features targeting the strict subset of Julia

Internals & Design
1

I just came across Add `strict` mechanism for opting into stricter subsets of the language · Issue #54903 · JuliaLang/julia · GitHub

For me, this would be an extremely welcome development. If I can just tell my students “put this line into your package” and we solve a lot of the footguns and make the code more readable for the students on follow up projects that’s amazing.

Edit: The immediate intent to remove some foot guns and sources of bugs is well taken, and clearly articulated in the issue.

I am curious about the intended scope long term perspective for this mechanism and how it interacts with other means of backwards compatible language evolution. Maybe this is looking too far ahead, but in principle this proposal allows adding new syntax to Julia that is ignored without strict but introduces new semantics with strict. Is that an idea that people have considered?

E.g. It would be in principle possible to have something like

function f(x, y  :| dont-fabulate)
  fabulate(x)
  fabulate(y) # disallowed in strict mode
end

Where julia version 1.24 and up ignores everything after :| making, it valid non-strict Julia, but strict Julia has new semantic guarantees.

This would probably necessitate thinking more about the interaction of strict code and non-strict code. E.g. in this completely stupid special case, if my non y fabulating function f2

function f2(x, y  :| dont-fabulate, g  :| non-fabulating)
  fabulate(x)
  g(y)
end

is defined in a strict package, but is called with a g is defined in a non-strict environment, what guarantees could we still get?

2

No, you’re describing a superset. The issue lists potential options that describe what subsets could be like.

5 Likes
3

The superset is in introducing ignored syntax in a 1.x update, not through the strict mechanism. Julia 1.10 has valid programs that were not valid programs in 1.0. Backwards compatibility means that a valid program does not become invalid when updating Julia, but invalid programs can become valid.

Now strict mode gives you a way to define subsets.

My observation is that you can expand the world of valid Julia programs in, say, 1.13 to those that have terms like :|(...) in them, with the semantics that these terms are ignored. This is in some sense an entirely trivial superset. It’s another way to write comments if you want.

But you can now use the strict mechanism to create interesting subsets of this trivial superset. I think this is an interesting, backward compatible design space.

4

Actually, thinking about what I just wrote, an interesting strict rule along these lines could be that all methods that are exported or public have to have docstrings.

5

No, the idea is to explicitly disallow ambiguous syntaxes and semantics, not add new ones.

It is worth emphasizing again that this feature is only intended to disallow undesirable programs that are otherwise semantically valid.

You’re describing a whole-cloth new feature. It’s inclusion (or coding thereof) isn’t made any easier with or without a strict mode.

1 Like
6

I understand the intent. My point is simply that this combination of extending the set of valid programs with the intent to then restrict it is permissible under your second quote.

The fact that it’s permissible under the rules as written doesn’t mean it’s a good idea, of course. And it seems it wasn’t considered in the issue. I just thought it was an interesting idea to toy around with how the strict mechanism and backwards compatible language evolution can interact. I didn’t want to spam the issue with this tangential considerations, hence this topic…

7

There are no rules here, there’s not even a draft implementation yet. The sky’s the limit on what’s possible. But you were explicitly asking how folks thought it would be intended to be used. These are my thoughts.

8

How could syntax evolution be indicated to users: as warnings in the compiler or also through a linter?

9

True, I phrased that part very badly. Sorry about that. I was thinking about long term perspectives, not the immediate intent, which is well communicated in the issue. I edited it a bit.

10

Rust’s editions could be an interesting reference for backward-compatible language evolution: What are editions? - The Rust Edition Guide (Caveat: I’ve never written a single line of Rust.)

2 Likes
11

Already mentioned on the page of the issue in question at Github.

12

Extending syntax in a language subset is a paradox. My guess is you were confused by one of the options describing :=. It’s actually proposing that := be added to the language itself, then there can be a strict mode option that disallows = in favor of it.

You tried to resolve the issue of extended syntax causing errors by having it be ignored, thus changing the semantics. That is not at all what subsets of languages are like; subsets have the same semantics regardless of whether you’re enforcing the subset. You’re describing a deviant parser of a superset, which nobody wants.

On a cursory read, Rust editions seem to support breaking supersets, like adding keywords that are no longer valid variables; the distinction from a typical major revision is that crates compiled with different editions must be able to interoperate, which I assume comes with limitations. Migration of code (largely automated) to another edition must still fix the breaking changes, which is probably why editions seem to be rare and centralized. Apparently, imported Rust macros can be used because they are parsed by their edition’s parser, and the edition differences are absent in the IR and compiled together smoothly. I don’t think that can be done in Julia’s runtime metaprogramming, seems really easy to eval some code in the wrong context, and supersets seem to go beyond the scope of the issue’s intent.

1 Like
13

I am not confused, you are missing the point. Julia 1.x is a superset of julia 1.y for x>y. Strict introduces the possibility to define subsets. My observation is that there are interesting non-trivial subsets of trivial supersets. I never talked about “extending the syntax in a language subset”.

All of this is backwards compatible, so it’s quite different to rust editions.

1 Like
14

With all due respect, that is exactly what you’re doing to :| here:

Making the non-strict parser ignore :| doesn’t really change that, it only adds extra code. If I need to fabulate(y), writing dont-fabulate is useless and misleading fluff, and if I can’t fabulate(y), I just wouldn’t write the call. If I really felt paranoid that someone might attempt to fabulate(y) in the future but there was some weird reason it shouldn’t happen, that’s much clearer in a comment. If I really want to freak them out, I could wrap Unfabulatable(y) and define fabulate(x::Unfabulatable) = x.

15

Sorry I don’t really know what your point is. It seems you agree now that what I propose is a valid combination of backward incompatible language extension and the strict mechanism.

I explicitly went for some nonsense fabulating to not get into discussing concrete issues. But if you want an example that is less conceptual: Imagine adding something like ownership and borrowing sigils that are ignored in normal Julia, but where you could opt in with strict-check-ownership. Such a proposal is probably not possible for a dozen other reasons but it illustrates that in this design space there are some things people care about.

As an aside, before wheeling out the old “just put it in a comment” cliche, please consider whether it really is adding anything to the conversation.

16

That is a very optimistic and incorrect interpretation of “misleading fluff.”

Unfortunately you did introduce a serious concrete issue that people reasonably addressed. If you had an idea what those other examples look like, you probably should share those instead. If not, I offer that you are suggesting things along the lines of Python’s type hints, which is functionally useless syntax intended for third-party static type checkers but could be used for anything. An important difference between your example and type hints is that type hints are not misleading, just unenforced.

We are in a comment section, and I do in fact believe I am adding to a conversation. If you would rather that I stop participating, you could ask me nicely.

2 Likes
17

It seems misunderstanding persists.

In my view I did not introduce “a serious concrete issue” anywhere.

Also strict Julia is structurally exactly like Python Type Hints. You can trivially define strict-python as first running the type checker and then python. And if you write python type hints and then don’t check them, the type hints can be arbitrarily wrong and misleading too.

Finally, documentation and docstrings are never automatically checked against the code. Which is exactly why its easy for them to go out of sync with an evolving code base, and why they end up being wrong and misleading so often.

But I think this discussion has been sufficiently derailed that it’s pretty pointless to continue here.

18

I see that, but people are free to disagree and discuss said issue.

No it isn’t, and the intents are very different. Python with type hints is written the same way regardless of whether you choose to do static analysis; strict Julia is intended to force you to write a subset.

Wrong at runtime because it’s unenforced, perhaps. It’s hardly misleading because the intent is unambiguous, whereas this was explicitly self-contradictory:

function f(x, y  :| dont-fabulate) # you say this
  fabulate(x)
  fabulate(y) # but you then do this
end

People would never intentionally contradict a type hint like that, it defeats the purpose. If people don’t want the type hint, they just won’t write the type hint.

Not sure what makes you think this, I for one have been consistently discussing your original post, and I don’t think it’s possible for you to derail your own conversation.

19

Obviously people wouldn’t intentionally contradict a type hint. People also wouldn’t intentionally contradict a fabulation annotations.

Again, what is your point? Do you object to designing languages and annotations to minimize and catch unintentional mistakes? That annotations can be abused by malicious actors writing intentionally misleading code?

For the record, the first example in the mypy documentation is such an obvious type contradiction. The point is obviously not that anyone would write this code, the point is to illustrate the mechanism:

def greeting(name: str) -> str:
    return 'Hello ' + name

greeting(3)         # Argument 1 to "greeting" has incompatible type "int"; expected "str"
greeting(b'Alice')  # Argument 1 to "greeting" has incompatible type "bytes"; expected "str"
20

So what did you mean by calling this contradiction “valid non-strict Julia” then? Did you mean to say it’s actually a silent mistake? The vast majority of people wouldn’t consider those synonymous.

It also errors at greeting(3), the point is actually that mypy’s static type checker can find type errors throughout the code for you to fix instead of hitting and fixing runtime errors one at a time. It’s a far cry from perfectly running code like your example.