How can interfaces work with symbolic-diff?

General Usage
1

In @MilesCranmer’s post about abstract typing, @ChrisRackauckas wrote

I didn’t want to take that thread off topic, so pulling this out. I’m interested to see @ChrisRackauckas mention function interfaces as a solution world splitting because my understanding from this post was that interfaces would mess up tracing for symbolic-diff (where Base.:(==)(::T, ::T)::T instead of returning Bool).

How can they be reconciled?

Edit: I understand how soft requirements like the various interface packages would work, but if it were a language feature to allow the compiler to be certain about a return value, then that seems like that strictness would be an issue.

Edit2: I also understand that there is a difference between isequal and ==, but I assume that there are functions that might have an interface defined that would preclude the normal tracing mechanisms.

Edit3: change references to auto diff to symbolic diff. Thanks @gdalle !

2

Just to clarify here: having == return a non-boolean is only necessary for symbolic differentiation (and other forms of tracing like sparsity detection), because the “numbers” involved there do not carry actual values around.
By contrast, automatic (or algorithmic) differentiation usually works by following the branch of code taken based on the input values, so == does yield a boolean. There might be other issues with strict interfaces but at least I think this one doesn’t apply for packages like ForwardDiff.jl or Mooncake.jl.

1 Like
3

That would give issues with tracing though, so SparseConnectivityTracer and Reactant would be two other cases that likely don’t treat that always as bool

julia> using SparseConnectivityTracer

julia> detector = TracerSparsityDetector()
TracerSparsityDetector()

julia> f(x) = [x[1]^2, @show(x[1] == x[2]) * 2 * x[1] * x[2]^2, sin(x[3])]
f (generic function with 1 method)

julia> x = rand(3)
3-element Vector{Float64}:
 0.1932411340661383
 0.4884474701278184
 0.09024014091785981

julia> jacobian_sparsity(f, x, detector)
x[1] == x[2] = SparseConnectivityTracer.GradientTracer{SparseConnectivityTracer.IndexSetGradientPattern{Int64, BitSet}}(SparseConnectivityTracer.IndexSetGradientPattern{Int64, BitSet}(BitSet([])), false)
3×3 SparseArrays.SparseMatrixCSC{Bool, Int64} with 4 stored entries:
 1  ⋅  ⋅
 1  1  ⋅
 ⋅  ⋅  1

But they clearly aren’t the only ones. There’s many others that you can think of too.

4

So is there a world where a Julia 2.0 could have enforced interfaces and tracing or do we always need interfaces to be warnings/documentation? Maybe you could use a Cassette-like approach to turn off interface checking for a call graph?

5

Well this is only an issue if Base.:(==)(::T, ::T)::Bool is the interface. Though, that can’t be true since Base.:(==)(::Missing, ::Missing)::Missing already breaks that interface… so this is really just a nice example that interfaces can be more difficult than you think. And that’s okay: all of these packages, even statistics packages with missing, would be fine if we accept Base.:(==)(::T, ::T)::T as the interface.

I would like a better interface for going into tracing mode. Currently the only nice interfaces to activate tracing require using (/abusing) dispatch, which effectively means you have to do @trace f(x,y) in front of any function you want to trace (where you then apply your tracer types in the macro and do the call).

If activating a abstract interpretation can be given a simpler interface then Symbolics, SCT, etc. could probably adopt that.

1 Like
6

Maybe CassetteOverlay.jl could be a way forward, but it looks a bit dead?

7

IMO this is the entire benefit of interfaces. They have to be opt-in. Then your symbolic type with a non-bool == would simply not opt-in. And every function that requires the interface would reject your type. Which is precisely what we want!

1 Like
8

I think Rust’s approach to interfaces is really well designed:

fn dump_numbers(out: &mut impl Write) -> io::Result<()> {
    for n in 1..=10 {
        writeln!(out, "{n}")?;
    }
    out.flush()
}

The “out: &mut impl Write” part says:

  • The function takes an argument out.
  • It modifies that variable (&mut), so you have to pass a mutable reference. The compiler checks this for you.
  • The variable needs to be of a type that implements (impl) the Write interface. The compiler also checks this.

Thus, only types that declare all necessary methods for the Write interface are allowed to be passed to that function.

I think Julia could do something like this and make it even cooler, because you could have multiple dispatch based on different levels of adherence to an interface :star_struck:

1 Like
9

100% agree. I think there’s just a lot of very narrow thinking that tends to go on with interfaces. I want all arrays to be mutable, so there for AbstractArray requires mutability. I want all == to emit Bool, so all == must emit Bool. But indeed BasicLogical or something meaning “I assert that these set of logical rules need to hold in this function” and then erroring on types for which it doesn’t, that is what would make them very useful and expansive, rather than just discarding everything someone feels is non-standard.

It’s maintained, though most of that stack is severely underdocumented.