Towards the creation of an interface testing package

General Usage
21

Ok there is a real example in the readme and the code is a lot closer to actually working, not that I have run any of it yet.

@TheCedarPrince if you want to read over the examples and give me some feedback here or on github about the general idea and syntax, that would be a huge help (anyone else too).

2 Likes
22

Absolutely!
This is quite exciting and eager to see what I can comment on!

UPDATE: I am going to sleep on this. I like where the thought is heading but do have some questions. Will write them up shortly.

23

See also Invariants.jl (documentation), which could serve as a base package to build a package for interface definitions on, and has taken some inspiration from previous discussions with @Raf

2 Likes
24

I haven’t used Invariants yet in the examples as I’m trying to write it in the simplest possible way, but I think returning Invariants will be the recommended method, instead of building any of the messaging code into Interfaces.jl.

Have you thought about making invariants a functor (invariant::Invariant)(x) = satisfies(invariant, x)? then we could just return an invariant for each condition of the interface instead of an anonymous function, making nicer syntax than x -> satisfies(invariant(......), x), and without a dependency on Invariants.jl.

1 Like
25

Interfaces.jl and BaseInterfaces.jl are passing tests now.

Basically a working proof on concept, maybe enough to try using in a package?

11 Likes
26

Just some thoughts… IMO A big part of the composability complaints come from the fact that currently there is a lot of composability that works perfectly, but in general is undiscoverable. The process of discovering what works and what doesn’t results in a lot of these bugs and a feeling like it’s guesswork to determine what will work between packages and what won’t.

Is there any interest for a solution that attacks a similar problem but without trying to get an interface standardized? Perhaps with help from an interface/testing library.

I’m thinking of a website or something that essentially curates common packages and simply rates their composability with other packages, user submitted comments/issues allowed.

Possibly between two packages, maintainers of both should be able to submit composability issue and the consent of both is required to resolve them. (and if they can’t agree to resolve….we’ll isn’t that the definition of a composability issue? ;)) Trolling is not much of a concern as again this is a manually curated list.

So that even if it’s not enforced in code, Julia users can at least have some idea of which packages will work together and tie in to the core ecosystem and which won’t.

This currently exists as a “curated” package list GitHub - svaksha/Julia.jl: Curated decibans of Julia programming language. . But it’s not actually very curated at all with many packages being no longer maintained, and again, no indication of what you can use with what. Simple requirements should be something like:

  • Project is maintained and of enough interest to the community that maintenance is unlikely to end
  • Sufficient documentation of package via Documenter.jl
  • Reasonable but not programmatically enforced tests showing correctness when composed within other curated packages that are reasonable for users to assume work together
  • etc…
4 Likes
27

I’m not sure curating is worth the effort. Better to write a test suite that pits packages against each other and we get to really find out what works with what in practice.

So for example, test packages that accept arrays with every single type of array. So instead of some rating there is just a test that fails.

Having a test package that provides “every single kind of array” would be super useful. Then packages can run whatever function they want over them all.

9 Likes