What interface bugs have you ran into in the ecosystem?

General Usage
21

The interfaces on the “Interfaces” manual page are documented, I’ll grant you that :slight_smile: But there are other interfaces in Base, for example the IO and Stream interfaces mentioned above are not documented as far as I know.

For the documented ones, I agree they are reasonably well-specified but to me that’s not far from wishy-washy… I mean we’re at the mercy of how people interpret English in the manual page. For example that page mentions “required methods” for Iteration vs “methods to implement” for Indexing. Someone might thing the latter methods are not absolutely required…

If we had formal interfaces, there would be no question if “indexable” requires firstindex/lastindex and there would be no question whether LinearAlgebra.I is “indexable”. Clearly we’re not there since Benny and I disagree.

22

Formal interfaces allow one to do simple checks, eg check for methods and maybe their types. But in most languages they usually do not describe or enforce correctness (except in languages that support design by contract, eg Eiffel).

It is unclear whether a formal DSL to describe interfaces in Julia specifically would have more benefits than costs (in complexity). Personally, I don’t think it would add much, but this is something about which reasonable people can disagree. Note however that while some people raise the issue from time to time, core devs apparently don’t consider it a priority.

Regarding the specific question above: <:AbstractArray indexing does not require iterate, because it has fallbacks, but you can do it if eg it provides a faster implementation.

1 Like
23

Yes, I think this whole thread was largely about the concrete benefits of these “simple checks” (no one was suggesting that interfaces would enforce correctness?) Personally I think interfaces would be a huge win for writing robust software in Julia.

24

I think it doesn’t. I.e. lastindex not used always:

and basically never, indexing (reading) an array requires getindex, yes, and writing an array setindex! [and technically neither may be required… can you have write-only array, or view, or read-only?]

firstindex only used when begin used, e.g. A[begin + 1]? Which is indexing, yes, plus extra (applying to all arrays?); and lastindex applying for end e.g. A[end - 1] only applying to most arrays, including all in Base/distributed with Julia, and Strings, but presumably not all subtyping AbstractArray.

I see also works for using SparseArrays; sparse([1, 2])[end]

25

I must say, it is quite convenient in IDEs for e.g. java or scala to click a button “auto-generate implementations” that plugs in the boilerplate of all methods you must implement for a non-abstract class in order to adhere to the interfaces / baseclass you extend.

I’m doubtful like you that the costs would be justified by the benefits, but the benefits can be quite substantial!

26

The solution to boilerplate is not to generate it with your IDE, but to get rid of the need to write it at all.

The best code is code not written because it is taken care of by fallbacks. A lot of this works nicely with current interfaces: the core is super-small, and then you can tag on optional methods as needed.

4 Likes
27

I think that illustrates my point that you cannot be sure of anything when interfaces are informal :slight_smile:

28

It doesn’t say all 4 methods must be implemented (ranges don’t have setindex!, we’d still consider them indexable), and if you define a new type with all 4 of those methods, iteration would still not be implemented automatically. You would have to implement iterate, and you could make it depend on getindex like the AbstractArray interface’s fallback does.

And I’m just saying it doesn’t always, it depends on the method.

Iterators.reverse says nothing about starting at a last element; it just needs to iterate in reverse order. Obviously for a linear sequence, you would need to start at a last element to avoid skipping any elements, but for a cyclic sequence, you could start anywhere and reach all elements. Starting at the last element of the wrapped linear instance is reasonable. I’d agree that the documented semantics of last(itr, n) don’t currently fit Cycle, but since removing that method is a breaking change, I’d figure they’d just expand the semantics from last n elements to n elements of reverse iteration, perhaps clarifying that reverse iteration needs to start at an element where it can reach all elements.

Yeah that makes a sort of sense, it’s just not (usually) multimethod dispatch.

Multiple dispatch and types implementing multiple interfaces just don’t mix, at least not without a duct-taped footgun, but it is possible to at least specify stricter interfaces in code with a method that takes types and checks its supertypes (to a limited depth for sanity’s sake). However, interfaces are useful when they’re not strict (why waste time implementing methods you won’t use), and we don’t have a reason to make them strict because we can’t dispatch over them like supertypes nor can we always tie them to 1 type like the ones documented in the Manual (for example, we could make a SendMail interface that is implemented 2 separate times for the 2 directions between 2 types, and argument position is the only way to distinguish sender and recipient, though trait-based dispatch like SendStyle(sender) can follow that).

29

The method signatures necessary to implement an interface to get to the fallbacks are the boilerplate in Julia.

I’m happy to fill in the interior of those methods. What I (and I suspect foobar_lv2) want is the ability to drop in a default implementation for my type where the interior of each is just error("IMPLEMENT ME"). Then, I am confident that my type will satisfy whatever interface I’m trying to implement.

This is a solved problem in other languages. Rust’s LSP has a code action which will do exactly this for a trait and type combination. Julia is not Rust for a number of reasons, not the least of which is the static/dynamic split which makes solving these problems in Julia much harder, and to expect Rust’s static analysis for Julia is never going to happen.

However, the absence of a tool to do this in one form or another makes maintaining a large codebase across many developers with varying levels of expertise difficult. I would take the LSP autopopulating solution, or some kind of static analysis to tell me when a method is missing for a type. The best I’ve got right now is to manually check things in tests using something like RequiredInterfaces.jl, which is functional but suboptimal because it requires running the test suite to discover what’s missing.

It’s hard to guide people to what they need to implement because it requires manually translating from one source to another (Julia arrays page to code).

It’s hard to know when compatibility has broken until it breaks during usage (because a new method was added to the interface).

It’s hard to know when an interface has changed and suddenly I need to index with a custom indexer type instead of an integer, because we added a new abstraction for indexing.

These are not problems developers face in (some) other languages. These are problems that I find myself facing daily in Julia, working on a team in industry. It is very possible to be extremely productive in Julia without interfaces, but as the size of your team grows, and the spectrum of experience widens, it becomes more difficult to maintain quality and conformity across a growing codebase. As someone who wants Julia to see wider adoption, that means improving the developer experience for individuals and teams.

I respect that this is not the priority for core developers, and I don’t have time to contribute to language to solve this problem myself, so I will continue being productive without it. My point is that this is a real problem that software developers working with Julia do face with practical consequences, and while I am able to be productive without it, both my and many other people’s lives would be easier if we had first class support in the language or tooling for this.

8 Likes
30

Fwiw it’s apparently garnered enough interest that this experiment exists

(sorry for spamming this link all the time like a maniac)

31

Formal specifications of methods & contracts are not going to solve the everyday problems people encounter. They are much too cumbersome to actually sufficiently define & automatically check, requiring powerful SMT solvers just to check whether a type conforms to your interface.

In my book, any actually workable solution MUST make do without e.g. z3, mostly because of performance.

32

I’ve been working on this problem for a while now; I’d argue it’s less a function of interest so much as it being extremely hard. The obvious cases are usually pretty easy; the issue is that Julia’s type system is extensive enough that building a system that covers enough of it to feel “complete” is theoretically challenging. It took my group about four years to figure out subtyping well enough to be happy with it; a statically-checked trait/interface system that gives you those stubs is probably going to be comparable.

6 Likes
33

I just wanted to jump in and say thank you for all the examples and the discussion; I’d argue that this very lack of agreement and confusion about what interfaces are and should be, as well about what should be in a specification, is a very good motivation for wanting a system that nails down more about what the community thinks of as an interface.

To @Tamas_Papp’s comment about utility. The value, in my mind, of having a specification that methods simply exist and possibly their return type is enabling reliable interoperation between packages. While interface mis/mal/under-implementation in terms of methods not existing is usually manageable for a single project (since you know what all the interfaces are, have your own internal definition of “interface”, etc), where it goes horrifically wrong is when you have multiple libraries written by different people at different times talking to one another.

Julia makes it easy to simply assume the “sanity” of a value and continue on until the bitter end when you find out 12 methods deep that the thingie doesn’t have the method you need and you now spit back an incomprehensible error message about “method not found” to the user. Moreover, this message may be highly uninformative about what the actual error is; if the value the method returned was wrong then you could give a sensible message pretty easily, but the nonexistence of the method is much trickier.

Avoiding this - while still enabling composition - demands that you, the developer of the library - reflectively check the existence of all needed methods at some “sane” boundary just to make sure you won’t have a method not found error somewhere deep inside the library. To my knowledge, nobody does this, including me, and it leads to weird and brittle compositions.

4 Likes
34

My view is that these have decreasing importance:

  • specifying most of the interface in code, so you can check if a particular value fails
  • checking the interface with a bunch of generated examples
  • guaranteeing the interface with 100% formal verification

But IMHO you can get most of the benefit just by being able to express your requirements/guarantees in code in a standardized API. That way users can experiment and ask “am I allowed to pass xyz” during dev/testing without having to suffer slow argument checks during real-world execution. Just having a common language for communication between function author and user will go a long way imho.

Modern solvers are fast enough now that Rust is experimenting with a Prolog-style trait solver (SLG Solver), so that kind of stuff will be interesting to explore in the future, but personally I consider it to be of secondary importance.

35

:100:

I would say that the next best thing is establishing a convention in the ecosystem that the interface provider (e.g. Base) should give several (one for each reasonable choice of what to implement as core and what to leave out for fallbacks) small but complete examples that you can copy-paste-modify, together with a generic test suite that you can run with your implementation.

A positive example for such an example implementation would be julia/test/testhelpers/ImmutableArrays.jl at master · JuliaLang/julia · GitHub – stuff like that should be front and center of the interface documentation for prospective implementors of the interface!

A negative example for testing would be e.g. DataStructures.jl/test/test_swiss_dict.jl at master · JuliaCollections/DataStructures.jl · GitHub vs julia/test/dict.jl at master · JuliaLang/julia · GitHub

Ideally Base (as interface provider of AbstractDict) would have an extensive test suite for AbstractDict implementations that SwissDict could just use instead of having to redo that, such that you only need to write tests for the unique things your implementation of the interface can do.

As a bonus, e.g. SwissDict would automatically profit from expanding testsets for all weird new corner cases that are discovered, and vice versa.

A good test suite is a graveyard of all the things that can plausibly go wrong (or have in fact implausibly gone wrong in the past) – and something that can plausibly go wrong in one AbstractDict implementation can plausibly go wrong in another one. (vulnerability extrapolation is a whole cottage industry!)

5 Likes
36

Sorry for necroposting, I just wanted to mention two moderate-scale interface specification prototypes that I have contributed to:

Of course those wouldn’t catch every bug, but they would catch a significant amount.

1 Like