Came across this topic in algotrading subreddit, I was surprised not to see even one mention of Julia. It would be understanding if it’s a big fund as they usually have their own infrastructure in C++ wrapped in python but It kinda is interesting that retail has not yet adapted more into Julia.
I initially started with python(and programing in general) but after two months of running into problems of package/environment management I came to the conclusion that in the long run it is not going to be worth it. Looked around and it was obvious that Julia is probably the best fit for this field(An older article stating some reasoning). The only thing (as far as I can understand) you can’t do with Julia is the hard-real-time stuff like sub millisecond sensitive market-making or ultra fast HFT strategies which are out of reach for retail anyway.
There are problems as Julia is a newer language but my experience have been noticeably better than python (although I am yet to run full execution). I was curious anyone here have deployed their algos in Julia? How has been your experience?
I’m new to Julia from Python, and I have been playing with stock data in Julia lately (e.g. InvestorsExchange.jl), so I’ll chime in and give you my 2cents. Overall I’ve been enjoying using Julia, which is letting me learn a new language and build things at a lower level (e.g. building RollingTimeWindows.jl since DataFrames.jl didn’t come with that built-in the same way Pandas does in Python). I think Julia is definitely a solid and very enjoyable tool for trading with a decent fledgling data processing ecosystem. However, I don’t think there are enough reasons to leave Python to make Julia a good choice for most people, largely since by leaving Python you leave behind a ton of great tools (unless you go to R, which I’ve heard is comparable, but beyond Python/R, things get DIY fast in this space).
You cited environment management in Python being a reason to avoid the language/ecosystem, but if you get comfortable with a tool like Docker, you can solve that issue in production pretty soundly. I have worked several years developing in Python, and thanks to Conda and Docker, I spend relatively little time worrying about environment management even when I push things to production. I believe the batteries-not-included state of Julia’s ecosystem for trading is far more of a pain point than the pain I feel when dealing with Python environments. If you’re new to programming now, I think you might also come to agree with my perspective after you gain more experience building software (I certainly struggled a lot with things like version control and environment management when I started programming, but now I struggle with things like “why can’t I just do a ‘5m’ time-based rolling window in DataFrames.jl like I can in pandas?!” a lot more). If I had to guess, I’d say that the main reason why you don’t see a lot of people in that reddit thread complaining about Python being a bad tool for them is a boring one – that there is just nothing really nothing wrong with Python for most trading use-cases.
In case this leaves you wondering why I’m personally trying Julia for this type of programming even though I’m much more comfortable with Python and don’t have any reasons not to use Python, let me give my reasons for using Julia.
- I want to learn Julia for learning and fun
- I want to build my own medium-to-low-level components for learning, flexibility, and fun
- I want to work with raw trade and/or order book data, and Python can be a performance pain when you get that low level (though tools like numba bring some of the JIT-compiled-via-LLVM goodness of Julia to Python, so even performance isn’t always a great reason to ditch Python, but it certainly isn’t fun to figure out which things break numba’s JIT compilation and work around them)
Looking at these reasons, I’d say they do seem pretty personal and unlikely to be shared by many, and I wouldn’t recommend Julia over Python to anyone who also doesn’t share them.
I hope this perspective is helpful to you!
Thanks for the input.
The third reason you mentioned for why using Julia over python, I have came across it more often than not in twitter and with friends. When I asked why not just change the language their logic was they have years of infrastructure and they would rather deal with an annoying bug once or twice a month than building again from ground up in something else.
I’ve spoken with a few HFT firms who are using or planning to use Julia for their market simulators and machine learning. I only know of one group who was using Julia for execution of some medium frequency strategies.
I use Julia for most of my trading research and considered writing an allocation-free execution system in Julia, but it wasn’t practical, so I resorted to using C++.
I’d really like to see some work on the GC that could get pauses down to low single digit milliseconds. That would would make it a great solution for medium frequency execution. I’m sure it would also be useful in robotics and other areas well.
That is pretty interesting. What time-holding-period are you referring to medium frequency, few seconds to few minutes?
Have you seen this talk about simulation in robotics with Julia? They achieved millisecond loops in their design.
while GC certainly can use some improvement, I suspect this may not be harder than C++ overall because writing the whole application (for trading) C++ means you have to “work hard” manually everywhere. Considering https://www.researchgate.net/publication/331983442_Julia_for_robotics_simulation_and_real-time_control_in_a_high-level_programming_language was possible, I’d say if you can dive deep for the latency critical part of your code to make it not invoke GC when you don’t want it to, you can retain the pleasant high-level programming for most part and get good latency for the inner most logic
Holding periods of minutes to hours, but still needing to update orders quickly and reliably for good fills.
Yes, I could write everything from scratch. But it’s much easier to use highly optimized websocket and JSON libraries from C++, than to rewrite those for Julia with non-allocating functions, for example.
One thing I’m planning to try for a new project is to load some Julia functions on a background thread from C++ to do the heavy (but less latency sensitive) continuous optimizations.