# Did Julia community do something to improve its correctness?

**URL:** <https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515>\
**Category:** General Usage\
**Created:** [August 5, 2023, 7:16am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515 "2023-08-05T07:16:42Z")\
**Posts on this page:** 20\
**Page:** 7

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [October 25, 2023, 3:27am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/166 "2023-10-25T03:27:16Z")

</div>

> [@ParadaCarleton](#):
>
> If what you _really_ want is an example from SciML, QMC.jl was riddled with correctness issues before I essentially rewrote it from scratch last year.

For reference to this thread, the issue was that the proposed methods for QMC sampling used in global sensitivity analysis proposed by Saltelli ([https://www.sciencedirect.com/science/article/abs/pii/S0010465509003087](https://www.sciencedirect.com/science/article/abs/pii/S0010465509003087)) turn out to not necessarily be convergent ways of sampling a QMC process. We had originally implemented this based on examples of other sensitivity analysis packages, such as R’s sensitivity ([https://cran.r-project.org/web/packages/sensitivity/sensitivity.pdf](https://cran.r-project.org/web/packages/sensitivity/sensitivity.pdf)) and Python’s SALib ([SALib - Sensitivity Analysis Library in Python — SALib's documentation](https://salib.readthedocs.io/en/latest/)) which reference this same paper as effectively the modern way to do Sobol sensitivity analysis. However, community members correctly pointed out this convergence issue, with a nice analysis seen in [update Documentation, readme and docstrings by dmetivie · Pull Request #79 · SciML/QuasiMonteCarlo.jl · GitHub](https://github.com/SciML/QuasiMonteCarlo.jl/pull/79#discussion_r1222946191), so we have since corrected the QuasiMonteCarlo.jl package to not use Saltelli’s method and updated GlobalSensitivity.jl around this as well (thank you for your contributions). This is our example of the Julia community not caring about improving correctness.

---

<div class="post-metadata">

**Author:** ![ParadaCarleton](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/paradacarleton/32/20005_2.png) [@ParadaCarleton](https://discourse.julialang.org/u/ParadaCarleton)\
**Post date:** [October 25, 2023, 3:51am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/167 "2023-10-25T03:51:32Z")

</div>

> [@ChrisRackauckas](#):
>
> For reference to this thread, the issue was that the proposed methods for QMC sampling used in global sensitivity analysis proposed by Saltelli

That’s a different issue (and one David Metivier, not me, gets huge props for fully understanding and fixing). I’m talking about how up until the fall of 2022, several functions in QMC.jl were just flat-out wrong, returning incorrect results. Moreover, the package did not include even a _single_ test of correctness until then, relying exclusively on clearly-insufficient tests like verifying that the returned values were a `Matrix{Float64}` with correct dimensions. The lattice samplers were nondeterministic, and several functions returned out-of-bounds values.

I don’t bring this up because it’s representative of the Julia community; it was an utter nightmare of a package (put together in a rushed GSOC project with insufficient supervision). I have never encountered a package that bad since, except maybe a few dusty corners of Distributions.jl written back in 2014. But the fact that it happened at all in a SciML package tells me we need to raise our standards.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [October 25, 2023, 7:56am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/168 "2023-10-25T07:56:36Z")

</div>

> [@ChrisRackauckas](#):
>
> There are plans for interfaces that are being tested. A lot of ecosystems are locking down the interfaces more and more.

That’s good to know! I’d love to learn more about the approaches being tested, do you have some links on hand?

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [October 25, 2023, 9:12am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/169 "2023-10-25T09:12:24Z")

</div>

Well, here’s one:

> [@\[RFC\] GraphInterface.jl: An interface proposal for Graphs.jl version 2.0](https://discourse.julialang.org/t/rfc-graphinterface-jl-an-interface-proposal-for-graphs-jl-version-2-0/104518/28):
>
> I just wanna point out that while everyone is free to design their own interface, it is only valuable if there are lots of algorithms that can exploit it. At the moment, the main source of algorithms is Graphs.jl, and that seems unlikely to change. Given the lack of maintainer time compared to the size of the codebase, a complete algorithmic rewrite is out of the question. This means we need existing code to work with minor adaptations. In fact, rethinking the interface is already a major unde…

It seems the plan is for it to be an explicitly defined and tested interface with Interfaces.jl

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [October 25, 2023, 9:23am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/170 "2023-10-25T09:23:13Z")

</div>

> [@Raf](#):
>
> It seems the plan is for it to be an explicitly defined and tested interface with Interfaces.jl

Ah, I see - Interfaces.jl tries to be accommodating by allowing optionality in an interface; I don’t think that’s a good idea, because it makes requiring that “optional” part of an interface really difficult from the perspective of a consumer of that interface. That’s why I don’t have/allow that in [RequiredInterfaces.jl](https://github.com/Seelengrab/RequiredInterfaces.jl/issues/10). This is part of the issue when it comes to the informal interfaces of Base.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [October 25, 2023, 10:06am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/171 "2023-10-25T10:06:32Z")

</div>

> because it makes requiring that “optional” part of an interface really difficult from the perspective of a consumer of that interface

It doesn’t. [Interfaces.jl](https://github.com/rafaqz/Interfaces.jl) provides compile-time traits for all the individual options, consumers can just check what parts are implemented. That’s one of the central design goals of the package.

Most reasonably sized interfaces have optional components, there is no way around designing for it.

---

<div class="post-metadata">

**Author:** ![Sukera](https://avatars.discourse-cdn.com/v4/letter/s/ce7236/32.png) [@Sukera](https://discourse.julialang.org/u/Sukera)\
**Post date:** [October 25, 2023, 10:33am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/172 "2023-10-25T10:33:39Z")

</div>

My point is that no interface should ever have the notion of “optionality”; as soon as you’re talking about that, you have a larger interface that ought to have its own name, as well as being able to restrict dispatch on its own. Put another informal way, a smaller interface (i.e., without the optional parts) is a subtype of the larger interface (with the optional parts). Having optionality be expressed as a proper subtyping relationship means you can suddenly make use of a large body of type theory for static analysis, and it can (theoretically, at least) slot right into the existing type system. That’s much more difficult to do if it’s all part of one “interface type with optional parts”, because that more or less requires whole-program analysis to decide which parts of the interface are actually used (avoiding that, to me at least, is part of the problem this topic set out to discuss).

> [@Raf](#):
>
> Most reasonably sized interfaces have optional components, there is no way around designing for it.

I’d say most reasonably sized interfaces have some “layers” of guarantees & requirements; but that’s not the same as saying they have optionality. It’s perfectly feasibly (and, in my opinion, preferable) to split such an “optional interface” up into individual parts that can be combined on their own.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [October 25, 2023, 10:44am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/173 "2023-10-25T10:44:28Z")

</div>

Optionality is typically best expressed in terms of a derived, sub-interface. I.e. You could imagine that we have `ArrayInterface` and, then you could have `MutableArrayInterface <: ArrayInterface`, and `ResizeableArrayInterface <: MutableArrayInteface`, etc.

If something satisfied `ResizeableArrayInteface` then it automatically has to satsify `ArrayInterface`, but the converse is not true.

Typeclasses are the typical way things like this are formalized and expressed in other languages.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [October 25, 2023, 11:46am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/174 "2023-10-25T11:46:20Z")

</div>

Without multiple inheritance this ideal of layered interface is harder to achieve. That’s why I wanna go with Interfaces.jl for the Graphs.jl formalization.  
Here are some example of overlapping but not nested graph families:

- weighted vs unweighted
- directed vs undirected
- mutable vs immutable

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [October 25, 2023, 11:49am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/175 "2023-10-25T11:49:48Z")

</div>

Inheritance like that would be nice in some ways. The reason Interfaces.jl works like it does is In practice interfaces in julia are heterogeneous and patchy in a pretty fine-grained way.

See: [Document AbstractSet interface · Issue #34677 · JuliaLang/julia · GitHub](https://github.com/JuliaLang/julia/issues/34677)

You’re recommending defining a _lot_ of interfaces to say something is a `Set`

The question is how does a new `Set` type opt into the parts that it implements in a clean, simple way. I’m not at all sure Interfaces.jl has the best way, but you can do that with one line of code.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [October 25, 2023, 11:50am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/176 "2023-10-25T11:50:10Z")

</div>

You don’t need to use actual type inheritance for this relation though, that was just to illustrate the point. The sub-interfacing can be expressed in terms of traits quite easily, and give a very useful way of expressing add-ons to the interface.

If your graph interface treats “weighted vs unweighted”, “directed vs undirected”, and “mutable vs immutable” all using the same level of optionality that Interfaces.jl provides, that sounds like a very un-expressive interface solution that’d ultimately be kinda frustrating to use.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [October 25, 2023, 11:51am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/177 "2023-10-25T11:51:47Z")

</div>

I should also mention that inherited sub-interfaces are really the only way for a third party to express that they made something that’s related to, but a restricted-subset of the interface that you’re providing.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [October 25, 2023, 11:53am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/178 "2023-10-25T11:53:06Z")

</div>

Sub-interfaces will often inherit from multiple optional parts of an interface.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [October 25, 2023, 11:53am UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/179 "2023-10-25T11:53:50Z")

</div>

That’s just a further reason why you’d want to express sub-interfacing via traits.

E.g. something like (don’t take this code too seriously, it’s just illustrative)

```julia
@interface GraphInterface

@subinterface WeightedGraphInterface <: GraphInterface
@subinterface UnweightedGraphInterface <: GraphInterface

@subinterface DirectedGraphInterface <: GraphInterface
@subinterface UndirectedGraphInterface <: GraphInterface

@subinterface MutableGraphInterface <: GraphInterface
@subinterface ImmutableGraphInterface <: GraphInterface

```

and then some person would write

```julia
struct MyGraphType
    [...]
end
@satsisfies MyGraphType (MutableGraphInterface, DirectedGraphInterface)

```

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [October 25, 2023, 12:00pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/180 "2023-10-25T12:00:17Z")

</div>

But I think that’s the misunderstanding: Interfaces.jl does provide traits, including for subinterfaces with arbitrary combinations of optional components. It is just a wrapper to define and test these traits in a more user-friendly way

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [October 25, 2023, 12:03pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/181 "2023-10-25T12:03:34Z")

</div>

This is virtually equivalent to what Interfaces.jl allows, I’ll let @Raf do the translation in the package syntax but it looks almost as nice

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [October 25, 2023, 12:05pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/182 "2023-10-25T12:05:34Z")

</div>

Ah good, I just didn’t see anything about subinterfaces in the documentation, and that together with the fact that is supported optional stuff made it seem like it wouldn’t support this.

I’m not really sure why you’d have optional methods if you support arbitrary sub-interfacing.

---

<div class="post-metadata">

**Author:** ![Raf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/raf/32/3383_2.png) [@Raf](https://discourse.julialang.org/u/Raf)\
**Post date:** [October 25, 2023, 12:15pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/183 "2023-10-25T12:15:00Z")

</div>

It doesn’t support sub-interfacing exactly like that. The optional components aren’t inheritable, which is something I’ve been thinking about the last few weeks.

I think it can be done fairly easily by swapping out the Symbols currently used for abstract types like you define, that inherit from the parent interface.

The macro could write all the boilerplate so it stays a one-liner like it is currently. It used `Symbol` instead of abstract types initially just to keep things as simple as possible.

---

<div class="post-metadata">

**Author:** ![Mason](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mason/32/2423_2.png) [@Mason](https://discourse.julialang.org/u/Mason)\
**Post date:** [October 25, 2023, 12:24pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/184 "2023-10-25T12:24:46Z")

</div>

I think things like that would be especially important if an ecosystem grew up around one of these things with an interface. i.e., if we look at `Number` or `AbstractArray`, there’s a lot of packages out there that define subtypes of these that do special things, but there’s also packages out there that define abstract subtypes of them that other packages are then expected to further subtype.

These subtypes need to express different levels of sub-functionality. i.e. it’d be kinda bad IMO if someone had to submit a PR to change the inteface for `AbstractArray` in Base to add an optional flag for expressing some niche concept that their ecosystem is concerned with, like say geospatial data or something.

Instead, they’d ideally be able to just create a `subinterface` or whatever in in their package that inherits all the requirements of `AbstractArray` but also has further requirements for their use-case.

---

<div class="post-metadata">

**Author:** ![gdalle](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gdalle/32/224680_2.png) [@gdalle](https://discourse.julialang.org/u/gdalle)\
**Post date:** [October 25, 2023, 12:31pm UTC](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515/185 "2023-10-25T12:31:05Z")

</div>

That’s a very good point for modular, inheritable interfaces.

[Previous page](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515.md?page=6)

[Next page](https://discourse.julialang.org/t/did-julia-community-do-something-to-improve-its-correctness/102515.md?page=8)