# What interface bugs have you ran into in the ecosystem?

**URL:** <https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001>\
**Category:** General Usage\
**Tags:** question\
**Created:** [January 19, 2024, 8:11am UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001 "2024-01-19T08:11:23Z")\
**Posts on this page:** 16\
**Page:** 2

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [January 23, 2024, 1:52pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/21 "2024-01-23T13:52:54Z")

</div>

The interfaces on the “Interfaces” manual page are documented, I’ll grant you that 🙂 But there are other interfaces in Base, for example the IO and Stream interfaces mentioned [above](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/7) are not documented as far as I know.

For the documented ones, I agree they are reasonably well-specified but to me that’s not far from wishy-washy… I mean we’re at the mercy of how people interpret English in the manual page. For example that page mentions “required methods” for Iteration vs “methods to implement” for Indexing. Someone might thing the latter methods are not absolutely required…

If we had formal interfaces, there would be no question if “indexable” requires `firstindex`/`lastindex` and there would be no question whether `LinearAlgebra.I` is “indexable”. Clearly we’re not there since Benny and I [disagree](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/18).

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 23, 2024, 2:08pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/22 "2024-01-23T14:08:59Z")

</div>

> [@sijo](#):
>
> If we had formal interfaces

Formal interfaces allow one to do _simple_ checks, eg check for methods and maybe their types. But in most languages they usually do not describe or enforce correctness (except in languages that support design by contract, eg Eiffel).

It is unclear whether a formal DSL to describe interfaces in Julia specifically would have more benefits than costs (in complexity). Personally, I don’t think it would add much, but this is something about which reasonable people can disagree. Note however that while some people raise the issue from time to time, core devs apparently don’t consider it a priority.

Regarding the specific question above: [`<:AbstractArray` indexing](https://docs.julialang.org/en/v1/manual/interfaces/#man-interface-array) does not _require_ `iterate`, because it has fallbacks, but you can do it if eg it provides a faster implementation.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [January 23, 2024, 2:26pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/23 "2024-01-23T14:26:22Z")

</div>

> [@Tamas\_Papp](#):
>
> Formal interfaces allow one to do _simple_ checks, eg check for methods and maybe their types. But in most languages they usually do not describe or enforce correctness (except in languages that support design by contract, eg Eiffel).

Yes, I think this whole thread was largely about the concrete benefits of these “simple checks” (no one was suggesting that interfaces would enforce correctness?) Personally I think interfaces would be a huge win for writing robust software in Julia.

---

<div class="post-metadata">

**Author:** ![Palli](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/palli/32/3380_2.png) [@Palli](https://discourse.julialang.org/u/Palli)\
**Post date:** [January 23, 2024, 2:27pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/24 "2024-01-23T14:27:09Z")

</div>

> [@sijo](#):
>
> since indexing requires `getindex`, `firstindex` and `lastindex`

I think it doesn’t. I.e. `lastindex` not used always:

> **[GitHub - JuliaArrays/InfiniteArrays.jl: A Julia package for representing...](https://github.com/JuliaArrays/InfiniteArrays.jl)**
>
> A Julia package for representing infinite-dimensional arrays - GitHub - JuliaArrays/InfiniteArrays.jl: A Julia package for representing infinite-dimensional arrays

and basically never, indexing (reading) an array requires `getindex`, yes, and writing an array setindex! [and technically neither may be required… can you have write-only array, or view, or read-only?]

`firstindex` only used when `begin` used, e.g. `A[begin + 1]`? Which is indexing, yes, plus extra (applying to all arrays?); and `lastindex` applying for `end` e.g. `A[end - 1]` only applying to most arrays, including all in Base/distributed with Julia, and `String`s, but presumably not all subtyping AbstractArray.

I see also works for using `SparseArrays; sparse([1, 2])[end]`

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [January 23, 2024, 2:34pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/25 "2024-01-23T14:34:11Z")

</div>

> [@Tamas\_Papp](#):
>
> It is unclear whether a formal DSL to describe interfaces in Julia specifically would have more benefits than costs (in complexity). Personally, I don’t think it would add much, but this is something about which reasonable people can disagree.

I must say, it is quite convenient in IDEs for e.g. java or scala to click a button “auto-generate implementations” that plugs in the boilerplate of all methods you must implement for a non-abstract class in order to adhere to the interfaces / baseclass you extend.

I’m doubtful like you that the costs would be justified by the benefits, but the benefits can be quite substantial!

---

<div class="post-metadata">

**Author:** ![Tamas\_Papp](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tamas_papp/32/25949_2.png) [@Tamas\_Papp](https://discourse.julialang.org/u/Tamas_Papp)\
**Post date:** [January 23, 2024, 2:39pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/26 "2024-01-23T14:39:28Z")

</div>

> [@foobar\_lv2](#):
>
> “auto-generate implementations” that plugs in the boilerplate

The solution to boilerplate is not to generate it with your IDE, but to **get rid of the need to write it at all**.

The best code is code not written because it is taken care of by fallbacks. A lot of this works nicely with current interfaces: the core is super-small, and then you can tag on optional methods as needed.

---

<div class="post-metadata">

**Author:** ![sijo](https://avatars.discourse-cdn.com/v4/letter/s/da6949/32.png) [@sijo](https://discourse.julialang.org/u/sijo)\
**Post date:** [January 23, 2024, 2:45pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/27 "2024-01-23T14:45:55Z")

</div>

I think that illustrates my point that you cannot be sure of anything when interfaces are informal 🙂

---

<div class="post-metadata">

**Author:** ![Benny](https://avatars.discourse-cdn.com/v4/letter/b/49beb7/32.png) [@Benny](https://discourse.julialang.org/u/Benny)\
**Post date:** [January 23, 2024, 6:39pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/28 "2024-01-23T18:39:22Z")

</div>

> [@sijo](#):
>
> The manual [says](https://docs.julialang.org/en/v1/manual/interfaces/#Indexing) that `getindex`, `setindex!`, `firstindex` and `lastindex` must be implemented. That’s all you need for iteration?

It doesn’t say all 4 methods must be implemented (ranges don’t have `setindex!`, we’d still consider them indexable), and if you define a new type with all 4 of those methods, iteration would still not be implemented automatically. You would have to implement `iterate`, and you _could_ make it depend on `getindex` like the `AbstractArray` interface’s fallback does.

> [@sijo](#):
>
> I’m just saying the _semantics_ of `last` are related to iteration.

And I’m just saying it doesn’t always, it depends on the method.

> [@sijo](#):
>
> I think the `Cycle` case is a bug since it doesn’t respect the semantics of `last` (reported [here](https://github.com/JuliaLang/julia/issues/53017), we’ll see what others think).

`Iterators.reverse` says nothing about starting at a `last` element; it just needs to iterate in reverse order. Obviously for a linear sequence, you would need to start at a last element to avoid skipping any elements, but for a cyclic sequence, you could start anywhere and reach all elements. Starting at the last element of the wrapped linear instance is reasonable. I’d agree that the documented semantics of `last(itr, n)` don’t currently fit `Cycle`, but since removing that method is a breaking change, I’d figure they’d just expand the semantics from last n elements to n elements of reverse iteration, perhaps clarifying that reverse iteration needs to start at an element where it can reach all elements.

> [@sijo](#):
>
> I meant imperative code (with `if`, `else`, etc.) to handle dispatch

Yeah that makes a sort of sense, it’s just not (usually) multimethod dispatch.

> [@sijo](#):
>
> you cannot be sure of anything when interfaces are informal

Multiple dispatch and types implementing multiple interfaces just don’t mix, at least not without a duct-taped footgun, but it is possible to at least specify stricter interfaces in code with a method that takes types and checks its supertypes (to a limited depth for sanity’s sake). However, interfaces are useful when they’re not strict (why waste time implementing methods you won’t use), and we don’t have a reason to make them strict because we can’t dispatch over them like supertypes nor can we always tie them to 1 type like the ones documented in the Manual (for example, we could make a `SendMail` interface that is implemented 2 separate times for the 2 directions between 2 types, and argument position is the only way to distinguish sender and recipient, though trait-based dispatch like `SendStyle(sender)` can follow that).

---

<div class="post-metadata">

**Author:** ![sbuercklin](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/sbuercklin/32/15728_2.png) [@sbuercklin](https://discourse.julialang.org/u/sbuercklin)\
**Post date:** [January 23, 2024, 7:29pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/29 "2024-01-23T19:29:57Z")

</div>

The method signatures necessary to implement an interface to get to the fallbacks **are the boilerplate** in Julia.

I’m happy to fill in the interior of those methods. What I (and I suspect foobar\_lv2) want is the ability to drop in a default implementation for my type where the interior of each is just `error("IMPLEMENT ME")`. Then, I am confident that my type will satisfy whatever interface I’m trying to implement.

This is a solved problem in other languages. Rust’s LSP has a code action which will do exactly this for a trait and type combination. Julia is not Rust for a number of reasons, not the least of which is the static/dynamic split which makes solving these problems in Julia **much harder** , and to expect Rust’s static analysis for Julia is never going to happen.

However, the absence of a tool to do this in one form or another makes maintaining a large codebase across many developers with varying levels of expertise difficult. I would take the LSP autopopulating solution, or some kind of static analysis to tell me when a method is missing for a type. The best I’ve got right now is to manually check things in tests using something like `RequiredInterfaces.jl`, which is functional but suboptimal because it requires running the test suite to discover what’s missing.

It’s hard to guide people to what they need to implement because it requires manually translating from one source to another (Julia arrays page to code).

It’s hard to know when compatibility has broken until it breaks during usage (because a new method was added to the interface).

It’s hard to know when an interface has changed and suddenly I need to index with a custom indexer type instead of an integer, because we added a new abstraction for indexing.

These are not problems developers face in (some) other languages. These are problems that I find myself facing daily in Julia, working on a team in industry. It is very possible to be extremely productive in Julia without interfaces, but as the size of your team grows, and the spectrum of experience widens, it becomes more difficult to maintain quality and conformity across a growing codebase. As someone who wants Julia to see wider adoption, that means improving the developer experience for individuals and teams.

I respect that this is not the priority for core developers, and I don’t have time to contribute to language to solve this problem myself, so I will continue being productive without it. My point is that this is a real problem that software developers working with Julia do face with practical consequences, and while I am able to be productive without it, both my and many other people’s lives would be easier if we had first class support in the language or tooling for this.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [January 23, 2024, 7:35pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/30 "2024-01-23T19:35:44Z")

</div>

> [@Tamas\_Papp](#):
>
> Note however that while some people raise the issue from time to time, core devs apparently don’t consider it a priority.

Fwiw it’s apparently garnered enough interest that this experiment exists

> **[GitHub - Keno/InterfaceSpecs.jl: Playground for formal specifications of...](https://github.com/Keno/InterfaceSpecs.jl)**
>
> Playground for formal specifications of interfaces in Julia - GitHub - Keno/InterfaceSpecs.jl: Playground for formal specifications of interfaces in Julia

(sorry for spamming this link all the time like a maniac)

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [January 23, 2024, 8:52pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/31 "2024-01-23T20:52:44Z")

</div>

Formal specifications of methods & contracts are not going to solve the everyday problems people encounter. They are much too cumbersome to actually sufficiently define & automatically check, requiring powerful SMT solvers just to check whether a type conforms to your interface.

In my book, any actually workable solution MUST make do without e.g. z3, mostly because of performance.

---

<div class="post-metadata">

**Author:** ![ckfinite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ckfinite/32/8341_2.png) [@ckfinite](https://discourse.julialang.org/u/ckfinite)\
**Post date:** [January 23, 2024, 9:12pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/32 "2024-01-23T21:12:56Z")

</div>

> [@sbuercklin](#):
>
> I respect that this is not the priority for core developers, and I don’t have time to contribute to language to solve this problem myself, so I will continue being productive without it.

I’ve been working on this problem for a while now; I’d argue it’s less a function of interest so much as it being extremely hard. The obvious cases are usually pretty easy; the issue is that Julia’s type system is extensive enough that building a system that covers enough of it to feel “complete” is theoretically challenging. It took my group about four years to figure out subtyping well enough to be happy with it; a statically-checked trait/interface system that gives you those stubs is probably going to be comparable.

---

<div class="post-metadata">

**Author:** ![ckfinite](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ckfinite/32/8341_2.png) [@ckfinite](https://discourse.julialang.org/u/ckfinite)\
**Post date:** [January 23, 2024, 9:21pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/33 "2024-01-23T21:21:11Z")

</div>

I just wanted to jump in and say thank you for all the examples and the discussion; I’d argue that this very lack of agreement and confusion about what interfaces are and should be, as well about what should be in a specification, is a very good motivation for wanting a system that nails down more about what the community thinks of as an interface.

To @Tamas_Papp’s comment about utility. The value, in my mind, of having a specification that methods simply exist and _possibly_ their return type is enabling reliable interoperation between packages. While interface mis/mal/under-implementation in terms of methods not existing is _usually_ manageable for a single project (since you know what all the interfaces are, have your own internal definition of “interface”, etc), where it goes horrifically wrong is when you have multiple libraries written by different people at different times talking to one another.

Julia makes it easy to simply assume the “sanity” of a value and continue on until the bitter end when you find out 12 methods deep that the thingie doesn’t have the method you need and you now spit back an incomprehensible error message about “method not found” to the user. Moreover, this message may be highly uninformative about what the actual error is; if the value the method returned was wrong then you could give a sensible message pretty easily, but the nonexistence of the method is much trickier.

Avoiding this - while still enabling composition - demands that you, the developer of the library - reflectively check the existence of all needed methods at some “sane” boundary just to make sure you won’t have a method not found error somewhere deep inside the library. To my knowledge, nobody does this, including me, and it leads to weird and brittle compositions.

---

<div class="post-metadata">

**Author:** ![jar1](https://avatars.discourse-cdn.com/v4/letter/j/c0e974/32.png) [@jar1](https://discourse.julialang.org/u/jar1)\
**Post date:** [January 23, 2024, 9:34pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/34 "2024-01-23T21:34:21Z")

</div>

My view is that these have decreasing importance:

- specifying most of the interface in code, so you can check if a particular value fails
- checking the interface with a bunch of generated examples
- guaranteeing the interface with 100% formal verification

But IMHO you can get most of the benefit just by being able to express your requirements/guarantees in code in a standardized API. That way users can experiment and ask “am I allowed to pass xyz” during dev/testing without having to suffer slow argument checks during real-world execution. Just having a common language for communication between function author and user will go a long way imho.

Modern solvers are fast enough now that Rust is experimenting with a Prolog-style trait solver ([SLG Solver](https://rust-lang.github.io/chalk/book/engine/slg.html)), so that kind of stuff will be interesting to explore in the future, but personally I consider it to be of secondary importance.

---

<div class="post-metadata">

**Author:** ![foobar\_lv2](https://avatars.discourse-cdn.com/v4/letter/f/ee59a6/32.png) [@foobar\_lv2](https://discourse.julialang.org/u/foobar_lv2)\
**Post date:** [January 23, 2024, 9:40pm UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/35 "2024-01-23T21:40:18Z")

</div>

> [@sbuercklin](#):
>
> I’m happy to fill in the interior of those methods. What I (and I suspect foobar\_lv2) want is the ability to drop in a default implementation for my type where the interior of each is just `error("IMPLEMENT ME")`. Then, I am confident that my type will satisfy whatever interface I’m trying to implement.  
> […]  
> I respect that this is not the priority for core developers, and I don’t have time to contribute to language to solve this problem myself, so I will continue being productive without it. My point is that this is a real problem that software developers working with Julia do face with practical consequences, and while I am able to be productive without it, both my and many other people’s lives would be easier if we had first class support in the language or tooling for this.

💯

> [@ckfinite](#):
>
> I’d argue it’s less a function of interest so much as it being extremely hard.

I would say that the next best thing is establishing a convention in the ecosystem that the interface provider (e.g. Base) should give several (one for each reasonable choice of what to implement as core and what to leave out for fallbacks) small but complete examples that you can copy-paste-modify, together with a generic test suite that you can run with your implementation.

A positive example for such an example implementation would be [julia/test/testhelpers/ImmutableArrays.jl at master · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/blob/master/test/testhelpers/ImmutableArrays.jl) – stuff like that should be front and center of the interface documentation for prospective implementors of the interface!

A negative example for testing would be e.g. [DataStructures.jl/test/test\_swiss\_dict.jl at master · JuliaCollections/DataStructures.jl · GitHub](https://github.com/JuliaCollections/DataStructures.jl/blob/master/test/test_swiss_dict.jl) vs [julia/test/dict.jl at master · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/blob/master/test/dict.jl)

Ideally Base (as interface provider of AbstractDict) would have an extensive test suite for AbstractDict implementations that SwissDict could just use instead of having to redo that, such that you only need to write tests for the unique things your implementation of the interface can do.

As a bonus, e.g. SwissDict would automatically profit from expanding testsets for all weird new corner cases that are discovered, and vice versa.

A good test suite is a graveyard of all the things that can plausibly go wrong (or have in fact implausibly gone wrong in the past) – and something that can plausibly go wrong in one AbstractDict implementation can plausibly go wrong in another one. (vulnerability extrapolation is a whole cottage industry!)

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/27854_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [January 31, 2024, 8:33am UTC](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001/36 "2024-01-31T08:33:32Z")

</div>

Sorry for necroposting, I just wanted to mention two moderate-scale interface specification prototypes that I have contributed to:

- [GitHub - rafaqz/Interfaces.jl: Macros to define and implement interfaces, to ensure they are checked and correct.](https://github.com/rafaqz/Interfaces.jl) and especially the `BaseInterfaces` subpackage
- [GitHub - gdalle/GraphsInterfaceChecker.jl: Formalization of the AbstractGraph interface from Graphs.jl, specified using Interfaces.jl.](https://github.com/gdalle/GraphsInterfaceChecker.jl) for the JuliaGraphs ecosystem

Of course those wouldn’t catch every bug, but they would catch a significant amount.

[Previous page](https://discourse.julialang.org/t/what-interface-bugs-have-you-ran-into-in-the-ecosystem/109001.md?page=1)