Type stability with user input

Hi there!
I am writing a small parser, and I use multiple-dispatch to make Julia use the right parser function depending on the type of token entered by the user. Here is a mock-up tokenizer that uses the same idea as my code.

However, this is obviously not type-stable. Is there a way for me to have type-stable code while still being able to rely on the Julia type system? This has been a very useful approach so far because it allows me to write small methods that behave accordingly to the context of parsing.

Thanks!

Unfortunately, parsing is an inherently type unstable task.

If you think about it, what parsing is doing is turning an unstructured piece of data (e.g. a String or a Vector{UInt8}) into a structured piece of data, i.e. typed data. So inherently, you have a mapping of String to multiple different potential output types, the definition of type instability. You can alleviate it by hiding the type instability from the compiler by having a single type represent your parsed+typed token and more or less doing a version of dispatch yourself (either via branching or some other approach, e.g. with FunctionWrappers.jl). The cleanest version of this I’ve seen in julia so far is how parsing is done in the upcoming JuliaSyntax.

1 Like

Thanks, that was my intuition, but I wanted to be sure that I wasn’t missing something obvious. I’ll have a look at how JuliaSyntax does it.

The general approach is to have a primitive type encoding the various versions, with checking and accessor functions in ifs acting as “dispatch”. Keeps things type stable for the compiler and performant by being (effectively) just bitmasks & such.

1 Like