[Article] Dear Google Cloud: Your Deprecation Policy is Killing You

If I think back to the v1.0 release and we went from v0.6 to v0.7. The v0.7 was basically the v1.0 release, but with deprecations to give people a chance to update their packages. The v1.0 was the same as the v0.7 release except those deprecations became errors. We would never introduce a breaking change without deprecating it first. So why can’t we just leave it deprecated (rather than removing deprecated methods) for backward compatibility? For v0.6 to v1.0 it made sense, but maybe we could do something different for v1.0 to v2.0.

2 Likes

I’d like to emphasise that there is also a crucial difference between an online service and an offline software. I cannot roll back the Google Cloud API to an old version as an end-user, but I can recreate an environment with Julia 1.0 and all the dependencies easily.

That of course does not solve the problem that I still have to possibly rewrite code in order to run under Julia 2.0 but at least I can run it. I think that nowadays with Docker and other containerisation solutions, we are in a much better position than in past days where you had either nothing or slow VM solutions.

11 Likes

You just put an upper bound on the ostensible benefits — they are not worth your time.

Why would they be worth someone else’s, if you are one of the people who cares about this issue?

3 Likes

Of course, I will help if I can. I didn’t find that comment particularly helpful (and flagged it actually) and should have just ignored it.

It is with good intentions I bring this up. I think v2.0 is still far enough away that it makes sense to have this conversation. Like I said, knowing the Julia devs, I would be surprised if it isn’t discussed somewhere already. I never really thought about it and went along with the semver mantra of “deprecate, then break”, but the article had me wondering if we can do better for v2.0. That is all.

2 Likes

Let things deprecated without erroring out will be at some time point in future very annoying and frustrating. Package maintainers could postponing needed updates to their package, because it still works. Maybe deprecation warnings can be switched off? Even worse: no need at all to update a package.
Now imagine a new user who just has to use some number of packages: o my god, what is this? The terminal is full of warnings? Not very trusting.

A second point: I am sure, having all things backwards compatible will be quite costly in terms of performance and memory usage. Just not optimal.

On the other side: making everything optimal as soon as it appears would be very breaking. Not a good idea, it would limit or stop creativity of users/developers.

So, the conclusion is clear: find a reasonable compromise of how long to keep things stable and when to break old stuff.

Now: are the Julia developers doing this compromise well and right for me? It doesn’t matter, because I still have some production code running within Julia 0.3 and it is just working perfect and self contained.

4 Likes

Very related: Thoughts on eventual Julia 2.0 transition

4 Likes

Maybe I am misunderstanding something here, but what you propose here for 2.0 sounds to me like it could just be called a 1.x release. Deprecations aren’t possible for every feature, especially when it’s changes in the parser or bigger language features. The point of a 2.0 release would be to make some breaking changes that perhaps were missed in 1.0 and that can’t be introduced in non-breaking ways. That of course doesn’t mean that 2.0 will just break stuff at random, I think the bar will be pretty high for breaking changes. It also looks like 2.0 is still a few years off, since 1.x currently serves us quite well, but there are some smallish breaking features I’d really like to eventually see in 2.0.

7 Likes

Thanks :+1:

I dunno. To me, this all sounds like a ringing endorsement for keeping deprecations around forever.

If I pick up an old code from 20 years ago and it still runs, but with deprecations suggesting I update to improved syntax / methods, that feels like a giant win to me.

I totally agree. This is not a hot burning issue by any means. I’m sure v2.0 will go smooth even if there are a few small breaking changes here and there, but can there be no breaking changes at all and keep deprecations around? I don’t see the downside, but am open to being educated.

Totally agree, but I think we are probably past this kind of change even for v2.0. I would think any breaking change in v2.0 could surely come with deprecations.

The comment is about trying to see this problem from a different perspective. I’m very much in favour of backward compatibility and in the SW project i run, we’re providing backward compatiblity to the users of the tool or interfaces into the tool. It’s definitly hard work and puts an awful lot of constraints on new features. Still we (like my tool development) cannot avoid it, as we still compare today’s output with output of some years ago (with years > 10). Steps by step if we can spend the effort, we ‘modernize’ the SW so doing SW maintenance, but this comes at a cost.

The julia situation is something like this: you can have a breaking 2.0 at 100% development speed or a non-breaking 2.0 at a development speed of 100-x %. Out of my experience this x is rather a two-digit number and you are really lucky if the first digit is 2.

I can really recommend to read the milestones on julia issues, currently there is not much backward compatiblity visible. I understand this and see this as consistent Julia culture (i really don’t complain here). Changing the culture would mean that more and more people engage in contributing.

4 Likes

Thanks for clarifying :+1:

You could be right that x > 30%, but it seems a little hard for me to believe that would be the case with Julia. I think x could actually even be negative with Julia.

Looking back at v1.0, we had:

  1. Release v0.L (where “L” is “last” and, in this case, L=7) with deprecations.
  2. Release v1.0 with deprecations removed so prior deprecations become errors.

Looking forward to v2.0, we could do the same thing:

  1. Release v1.L (where “L” is “last”) with deprecations.
  2. Release v2.0 with deprecations removed so prior deprecations become errors.

Alternatively, we could:

  1. Release v1.L-1 without deprecations.
  2. Release v2.0 with deprecations (which is effectively v1.L from above) and keep deprecations forever so that all v1 code still runs.

The latter seems like less work.

1 Like

This is not thought through to the end:

It may be less work for now (which I doubt, but ok for my argument), but it will accumulate for every future deprecations (3.0, 4.0,…). You have to take all deprecations into account for all future if you

For me it seems, that you don’t have much experience with maintaining and developing further a long term software project.

3 Likes

To be clear, it’s not that the actual non-removal of deprecations is costly. The cost comes when you want to maintain, refactor, or add new features and run into a bog of internal complexity and a combinatorial explosion of weird corner cases.

7 Likes

Google was always notorious for “deprecating” their services:

I know at least 10 different Google’s messaging / social network / video call apps, all dying one by one.

2 Likes

Based on how 0.7/1.0 was handled, I am assuming that this is what will happen.

This is nice because there remains a practical upgrade path remains out there forever. People could still port code from Julia 0.6 to 1.0 just by running 0.7, fixing everything (--depwarn=error is your friend), and moving on.

Of course with multiple major versions, this may become tedious. But after decades, bit rot sets in anyway; also, code that has not been touched for a long time is possibly not that useful to anyone.

10 Likes

In 20 years, do you still want all that baggage in Julia 3?

It’s going to accumulate and slow down compilation.

Einstein said to make everything as simple as possible, but not simpler than that. For the sake of keeping things simple, authors should eventually move on and upgrade. Otherwise, just re-create an older environment.

8 Likes

This is true, but I think it is also true for most of the young Julia team who have been working on nothing but Julia since college so it is useful to have your perspective :slight_smile:

However, from the article, it appears that Java does precisely this too, so people who do have experience are also doing it, so worth thinking about.

I’m sure you’re right, but I also think that Julia, with multiple dispatch, is somewhat uniquely placed to handle this smoothly. I think most maintainers would not want their users to see a bunch of deprecations so would work hard to keep everything current. Thus, avoiding the complexity at the package level.

The question is, how much complexity is the language willing to absorb to reduce complexity for the users and package maintainers? In either case, I don’t expect all that much complexity because Julia has, so far, done a fantastic job with compatibility.

I want to say, I’m not worried about Julia 2.0 (which doesn’t even have a due date yet, not sure well ever get there, for any one or more of the ideas below, just discussion points), but I have full confidence in the Julia developers doing the right thing, and the old Julia LTS pre-2.0 would still work.

There’s some really good discussion here (not just on this specific name change, but on them in general, pros and cons, e.g. for beginners), and I quote Karpinski as I see he has the same idea I do:


Karpinski:

It would be trivial to have using Julia1 package that exports compatibility aliases (and methods) for >Julia 1.x names, which could be used as long as people want to.

Tim Holy (Jun 16, 2020):

[…] But yes, I don’t personally see any urgency for 2.0, so I expect that no one needs to be blocking out transition-time on their calendar anytime soon.

I took a look (again) at the Julia 2.0 milestone (I have one issue there that was marked 2.0) and some examples are interesting, and I comment on below.

A. For functions we could allow new functions (as we do all the time) AND keep old, that we have renamed, but it might be better to drop them (still usable with using method above and/or have choose-able with a command-line switch?).

The best answer is to @deprecate one onemult and simply not have a function called one .

As with Julia 0.7 we would deprecate, but if we wouldn’t get rid of one in subsequent 2.0, I’m not sure what the point of the change is, since we could have using Julia1 (or Compat?).

A number of functions have been suggested to be dropped or at least no longer exported, all such could be handled this way…

B. … but operators are more difficult. For all sane programs you should get the same result for left-to-right, vs right-to-left evaluation, i.e. the same calculated number/matrix (am I wrong about some rounding issues? I at least mean almost the same, modulo some catastrophic cancellation floating-point issues). But for + and * the associative needs to be one or the other (by default), and this can make some code order-of-magnitude faster. So I kind of want them to “break” this if it helps speed, and hurts almost no code (I’m not sure it would slow down, but let’s say you add print-statements, for debugging, they would come out in opposite order, why “breaking”).

See workaround: https://github.com/JuliaLang/julia/issues/34999#issuecomment-595511495

C. Important to realize, Julia 1.0 only gave us syntax-guarantee (and some stability for function names), e.g. excluded was the experimental thread API (JavaCall.jl had issues because of it, now solved I understand) and while the name of the random number generation functions didn’t change, there was no (explicit) guarantee about the exact sequence of random-looking generated numbers (since “random”, seems like an edge-case to want that), later explicit warning given, so people do not assume implicit-guarantee:

[This only affects test, not users of your library, as while tests breaking, code would still work after such transition.]

In theory, this could have been solved by using Random_new in case there had been an explicit guarantee, but only for single-threaded programs. In general multi-threaded programs have non-determinism (a language like Rust is to work better with it), with or especially with random number generation.

1 Like

Just want to point out that nested quaternions and octonions are non-associative. Even though quaternions are associative, nested quaternions are not.

Also, deprecating one seems unnecessary?

1 Like

Deterministic multi-threaded or even distributed simulations are possible, if proper care is taken. Afaiu libraries like DiffEq for SDEs do take proper care.
We might even get that on the language level (for the default generator) in 1.6, if https://github.com/JuliaLang/julia/pull/34852 progresses to sufficient quality. (this would make deterministic multi-threaded random seamless and easy for the masses, not just possible for experts who code their own framework for splitting random streams).

2 Likes

There is no such thing as a non-breaking 2.0. It’s an oxymoron. If it were non-breaking, then we’d call it 1.7, 1.8, 1.9, etc., etc. Now there are things that can help the transition — things like the Compat package — but requiring the addition of helper tools is also breaking.

The class of changes that is easily deprecatable/assistable by external packages are also exactly those that are easily mechanically transformed via a tool like femtocleaner. They’re also the easiest ones to tackle all around, and can be handled seamlessly in a non-breaking manner. Indeed, there are already cases where we’re introducing better idioms into Julia 1.x but leaving the old things around.

Version 2.0 isn’t for such changes.

13 Likes