Hi,
The tl;dr of this post is basically how much hassle it is to depend on pre-compiled c artifacts v.s. just using an existing python wrapper for said artifacts.
I started the (perhaps foolish) endeavour to implement an ONNX exporter and while the main motivation was only have persistent storage of my own models so I can load them back into julia I couldn’t resist the challenge of also making them truly ONNX compatible in the sense that they can be read and used in any framework which supports ONNX (which I guess is not too much to ask from something which calls itself an ONNX exporter 
).
To my dismay, it turned out that using the data used in the testcases of the ONNX project was not enough to ensure compatibility.
As such, I’m now using ONNX runtime as a reference model where my testcases verify that the output from an exported model which is then imported in ONNX runtime is the same as the output from the original model.
Now to my real question: It was quite straighforward to use the python wrapper for ONNX runtime through PyCall and Conda. I do however see a few drawbacks with this:
- It is an extra dependency towards PyCall and Conda which I wouldn’t need save for the testcases. I have made them test-only dependencies but they are still dependencies
- I’m a bit worried that some poor unexpecting soul who is just trying to help me will get their python environment messed up when they run the testcases. From reading the PyCall and Conda documentation it seems like it would be quite messy to create a special python environment for the tests, but maybe I got that part wrong.
It seems like ONNX runtime provides assets with each release which seem to contain .so and .dll files which I think expose the inference API. The thing is that I don’t have much experience with working with dependencies like this.
In your experience, would this be something worth using or would it be better to just stick with the python wrapper?
My thinking is that I would add each tgz/zip file as an artifact and then use something like sys.isXXX to figure out which one to use. Is this how it is done or is there a better way?