That is true, and Gen has one of the most elegant interfaces for this. But it is the other way round from what I am thinking of: in Gen, the interface defines what a model needs to be able to do for the evaluators to work. It makes no assumptions about the structure; in fact, that’s the whole point. (The dynamic and static modelling languages are just two possible, but each in their own way restricted implementations of the GFI)
I’d like something where we do define the structure (thus having a kind of interface to it), but remain complely agnostic to what an evaluator might require.
Now, then, ideally, you could just implement a model-to-GFI transform and have the full machinery that Gen allows you to write available. But equally well you might ignore the GFI, check statically that a model does something restricted with only Gaussian Processes, and convert it to something that Stheno understands.
Quite that, yes. Universal PPLs have as their goal to let the user write down every model the language allows, and still be able to do inference on it. Of course at the “edges” of the space of reasonable programs, tradeoffs need to be made to still be able to do that. I’d like to get rid of the second part of that “and”. I want a format in which you can “write down” every possible model of a very large class, without a priori having to deal with the restrictions of inference. Then for each model, you can choose a suitable, perhaps specialized, backend that understands the fragment of the model language you used.
Probably the answer to the confusion I have caused is this: I come from a metaprogramming/analysis perspective, with interest in programming language design. I wanted variable names and dependencies to behave nicely, and primarily a closed, elegant language. Many PPL people probably come from an inference perspective, putting the language design problem second to that. “I want to write all the models” vs. “I want to do all the inference”. But I also try to close a bridge to the mostly theoretical, FP-based approaches of just formalizing probabilistic programs.
A bit like Julia IR. Representing an abstract and uninterpreted parametrized joint density function over its trace (as given through the unified name set) and return value, factorized syntactically into primitive statements and blocks.
But as I always say: pure phantasy as of yet :shrug: Soss surely comes closest to my ideas, I think.
I think symmetric monoidal categories with some extra stuff provide a good semantic framework for graphical models. After all, arrows are closely related to string diagrams.
