Is changing an error message in a package a breaking change?

General Usage
1

Let’s say that I have a module

julia> module a
       f(x::Int) = x >= 0 ? √x : error("negative")
       end
Main.a

that I subsequently release as a package. Now an user writes a code that specifically relies on the error message string being what it is, eg:

julia> testfn(n) = try
           Main.a.f(n)
       catch e
           buf = IOBuffer()
           showerror(buf, e)
           message = String(take!(buf))
       end
testfn (generic function with 1 method)

julia> @assert testfn(-3) == "negative"

In this case changing the error message in the package becomes a breaking change, even though the code throws an ErrorException in either case. Is this a case that package maintainers need to consider, and should this change be included in a major release (or a minor release in pre-1.0)?

2

I don’t know what exactly constitutes a ‘breaking change’ in the Julia ecosystem, but relying on error message content is terrible programming practice. This would mean, for example, that fixing typos would be a breaking change.

Virtually any change is conceivably breaking, even pure performance improvements.

(Appropriate xkcd link: xkcd: Workflow)

6 Likes
3

My view would be “no”. Since there is only 1 error that could be generated people probably didn’t check it’s value, just that an error was thrown. That doesn’t mean someone didn’t in which case yes this would break their code, but I think that would be rare enough to not worry about it.

4

In general I recommend looking to TensorFlows documentation on what they consider to be a breaking change

In particular they exclude:

  • Changing the type of exception thrown, unless the docs specified that that function threw that type of exception.** (which is even stronger than changing the message. They are allowed to change the type).
  • Changing something from throwing a exception to returning the correct result, even if it was documented that it would throw an exception. (so functionality is allowed to be added)
  • Return value of floating point operations. You can rely only on accuracy saying the same, not exact value.
2 Likes
5

They should not do this. Error messages are meant to be read by people, you may change the formatting any time.

If you want to expose an API for errors, use one of the existing error types (here, DomainError) or introduce your own if necessary. See

https://docs.julialang.org/en/v1/manual/control-flow/#Exception-Handling-1

for an example.

Changing the types of these errors could be considered a breaking change.

6

Julia packages should use semver. Semver specifies that software using semver must declare a public API (point 1). So the not very useful answer to your question is that changing the error message is a breaking change if the contents of the error message are part of your declared public API.

Separately, I would argue along with most in the thread that the contents of an error message should generally not be part of your public API.

1 Like