Thoughts on eventual Julia 2.0 transition

Internals & Design
41

I want to chime in here because I strive for this but there are cases where you simply cannot get 100% coverage. An example I recently ran into: consider code where you open a file (assume successfully), and then you do something with the IO object in a try/catch block. Here’s the concrete example in my case:

function savegraph(fn::AbstractString, g::AbstractGraph, gname::AbstractString,
     format::LGCompressedFormat)
     io = open(fn, "w")
     try
         io = GzipCompressorStream(io)
         return savegraph(io, g, gname, LightGraphs.LGFormat())
     catch
         rethrow()
     finally
         close(io)
     end
...

How do you reliably test that try/catch block? You could, I suppose, induce a race condition whereby the disk gets filled up between the time the file is opened for writing and the time savegraph is called, but that seems to be a crazy solution to get a couple of lines of code coverage that tests a system level failure.

3 Likes
42

Yes this is another good example for an exception from the 100% rule, like every try…catch, thats why this is called exception handling :grinning:

The idea behind good coverage of code in test routines is, that I can easy and even automatically test, if a package runs in general on a given version of julia. For this to achieve one needs a good coverage, but not 100%, because, in your example, the exception is an exception, so if the package fails in this case, because rethrow() is not part of the julia version, it doesn’t matter, because it is an exception and not the normal flow.

What I want to do: I am somehow happy with julia 0.7 and I am not able to move to 1.0, because some absolutely important packages I need are still only available for 0.7 or below. Another great package developer starts developing his package GreatStuff.jl with 1.0 and adds julia 1.0- to his Require file. Now I want to install GreatStuff.jl in julia 0.7 and clearly Pkg.test(“GreatStuff”) will work out fine and I can use it with proper caution but it will not break my system if I use it. Happy I am and I can solve my daily tasks.

In this case we knew this beforehand, things with julia are still easy and not so complex, systems are small, more or less experiments, not yet production.

Now imagine large production systems with deep nested dependencies of packages, own development and external ones, highly meshed, millions lines of julia code. I want to see, that these systems still can be moved forward to new major julia versions with reasonable effort. This must be the goal for a general purpose language.

1 Like
43

Hmmm, putting in 1.0 as minimum requirement is a bit too ambitious in my opinion. I’d contact the developer and ask him to change it to 0.7. At least I see no reason why I’d force someone to +1.0 instead of +0.7…

2 Likes
44

Becasue he just started his julia career with 1.0 and didn’t know about 0.7 as a transition version from 0.6.

The whole point is not about a single package. Its about a huge package ecosystem with dependencies and users which rely on that ecosystem over years and with large production systems.

45

An interesting blog post about eventually transition Go to version 2.0:

In particular,

Go 2 must bring along all those developers. We must ask them to unlearn old habits and learn new ones only when the reward is great. For example, before Go 1, the method implemented by error types was named String . In Go 1, we renamed it Error , to distinguish error types from other types that can format themselves. […] That kind of clarifying renaming was an important change to make in Go 1 but would be too disruptive for Go 2 without a very good reason.

1 Like
46

It is just position (maybe I had not describe it clearly with my poor English): “Don’t count your chickens before they hatch.”

Especially if there is still possibility that some debacle could happen.

1 Like
47

I found another place where this is not true! There is

broadcastable(x::Union{Symbol,AbstractString,Function,UndefInitializer,Nothing,RoundingMode,Missing,Val}) = Ref(x)
...
broadcastable(x) = collect(x)

inside broadcast.jl in standard library.

So we get (unexpectedly if there is understanding String as collection):

julia> mask = occursin.("ab", ["bc", ])
1-element BitArray{1}:
 false

If we want String as collection we need to do:

julia> mask = occursin.(collect("ab"), ["bc"])
2-element BitArray{1}:
  false
 true

But we don’t usually look at Regex as collection so we could probably think about:

julia> Base.broadcastable(x::Regex) = Ref(x)

julia> mask = occursin.(r"ab", ["ba", "ab"])  
2-element BitArray{1}:
 false
  true

julia> mask = occursin.([r"ab"], ["ba", "ab"]) # this trick is not necessary after redefining Base.broadcastable for Regex.

We probably want to add Regex to “self-reference-broadcastable” types.

Is this part of non breakable compatibility for 1.x or we could change this to Julia 1.3 (or 1.4?) after deprecation period? Or we could probably add it immediately because collect(x::Regex) is error in current implementation?

As this is big example where Strings are not treated as collections, does it help to convince you that it could be good to redefine this?

in(::AbstractString, ::AbstractString) = error("use occursin(x, y) for string containment")
1 Like
48

Yes, strings behave as scalars in the context of broadcasting because it’s generally much more useful. I agree that’s inconsistent, but I guess convenience trumps systematicity in some situations. Anyway, that’s not an argument to change the behavior of in, as it wouldn’t be consistent with either of the existing behaviors.