How to create more helpful errors in nested interfaces

General Usage
1

Hi all, I am trying to throw more helpful errors in POMDPs.jl. This package has two nested interfaces. The explicit interface deals with explicit probability distributions; the generative interface samples from distributions that are never explicitly defined. If the explicit interface is implemented for a problem, functions from the generative interface will call explicit functions and sample from the distributions using rand.

The problem is that when parts of the interface are not implemented correctly, the MethodErrors are very misleading. Users often think they need to implement the explicit interface, which is very difficult or impossible for many problems. The main issue is here: https://github.com/JuliaPOMDP/POMDPs.jl/issues/260

Cartoon version

Below is a cartoon version of the problem. gen is a simplified version of the generative interface, and transition is a function from the explicit interface - it should return a distribution.

module POMDPs
    export POMDP, gen, transition, solve

    abstract type POMDP end

    function transition end

    gen(m, s, a) = rand(transition(m, s, a))

    function solve(m::POMDP)
        # in reality this does something much more complex, but uses gen
        sp = gen(m, 1, 1)
        policy = nothing
        return policy
    end
end

using Main.POMDPs

struct MyPOMDP <: POMDP end

solve(MyPOMDP()) # this throws a MethodError for transition - very confusing!!

Desired behavior

  • If transition is called from gen and transition is not implemented, an error saying roughly “no state transition model was specified, you can implement it with gen(MyPOMDP, s, a) or transition(MyPOMDP, s, a)” should be thrown.
  • If transition is called outside of gen, a MethodError should be thrown for transition
  • There can be no overhead when transition is implemented - these functions run in inner loops

Current best idea

My current best idea is to provide a default implementation for transition that analyzes the stack trace to see if it was called from within gen and then throws an error based on that.

This does not seem like a very good approach - does anyone have a better idea?

1 Like
2

How about defining a fallback for transition:

function transition(x::T,s,a) where {T<:POMDP}
    error("No state transition model was specified")
end
3

In that case, many users will mistakenly think they need to implement transition when they actually only need to implement gen.

(btw, the default MethodError is better than that anyways because it helpfully shows the closest matches)

4

What about

function gen(m, s, a)
    if !applicable(transition, m, s, a)
        error("no state transition model was specified, you can implement
               it with gen(MyPOMDP, s, a) or transition(MyPOMDP, s, a)")
    else
        transition(m, s, a)
    end
end

Ideally the compiler will optimize away the branch, so this has no cost.

5

Ideality and reality are often two different things though.

1 Like
6

You are right — this is not yet optimized away in the cases I checked.

7

Yeah @Tamas_Papp’s solution is what I’d like to do, and indeed what we have been doing by cheating and observing the method table in @generated functions, but I want to come clean and not cheat anymore!