I’m a little disappointed to see how little activity there is in the finance and economics domain, though perhaps this is just due to the lack of a “critical mass” of other people to discuss it’s use with.
So, I would like to get a handle on what kinds of things people are using Julia for within the finance and economics space. Obviously we have JuliaFin and a smattering of interesting cases like with BlackRock or the New York Federal Reserve, as well as the packages within the JuliaQuant sphere (particularly QuantLib). Unfortunately a lot of the JuliaQuant stuff has fallen into disrepair, so I mostly just want to hear the kinds of things that people are doing with Julia and economics/finance.
Stuff like:
What interesting problems have you solved that you believe are better implemented in Julia?
If you’re working with an institution of some sort, how has pickup of the language been?
What kinds of things do you think are missing in Julia for finance?
Any other notes or thoughts you have on the domain application for Julia would be helpful as well. Just wanted to hear from some other folks on this.
Continuous time problems boil down to differential equations which pretty much require looping. This is what brought me and many others to work on ODEs, PDEs, SDEs, etc. in Julia. There’s a lot of special tools to work with SDEs that are currently specific to Julia, and that’s a core topic of mathematical finance. Additionally, we are working hard towards better tooling on jump diffusions (we have some time-adapted jump solvers, we need the Platen style “regular jump” solvers) and for solving the weak Fokker-Plank forms. So I can’t say much in terms of the data analysis side of it but there’s a lot stuff for the simulation aspects, and I would even venture to say that you’ll find more simulation tooling than in other languages, but not a financial modeling package which makes use of all of it.
A possible explanation for this may be that a lot of the econ/finance calculations are about algorithms and numerical issues, neither of which are Julia specific, and the Julia specific part can be simplified considerably. I work on econ problems all the time, but when I ask questions here I simplify to an MWE which abstracts from the specific structure. I consider this beneficial, since the questions and answers apply to a wider audience.
Solving and estimating DSGE models (package private, will be made public soon), Bayesian estimation of structural models using indirect inference, solving RE models using global methods; and quite a bit of data work. But I try to abstract out the generic parts so the packages I make public are only about “econ problems” to the extent absolutely necessary. Eg ContinuousTransformations.jl implements the standard transformations one would use for both domain mapping in collocation (eg approximating a function using Chebyshev polynomials) and Bayesian analysis, but nothing says “econ” anywhere, since these problems are general.
100% among macroeconomists, but that is a small integer
Quite useful for solving RE models globally is the CompEcon.jl toolbox that has been ported from Matlab. For some more empirical work there are some crucial things that need to implemented in Julia before it can be used productively, i.e. automatic LaTeX-output of regression tables using custom Standard Errors (HAC, HC, etc.).
There is also the QuantEcon.jl toolbox and the accompanying online lectures. I am not sure how active development is there.
I am a PhD student and I try to use Julia whenever possible. My fellow students still use mainly Matlab. If more functionality from dynare was ported to Julia, that would be awesome.
My main area is corporate finance and financial engineering. I find Julia very useful in Monte Carlo derivatives pricing. Also, the Optim package is very useful for model calibration, I used it to estimate a Gram–Charlier foreign exchange option model, and it is faster than Matlab.
I guess for right now, most Econ-Fin Julia users are from academia. For the industry, I am not sure how many are using it. Last month, I went to Guggenheim in Chicago for a meeting. I talked to their fixed income team, but no one has heard of Julia. They told me they like Java and C#, and will use more F# in the future. It seems functional programming is very suitable for describing financial contracts.
There are a couple of users in Industrial Organization I’m aware of too. Tobias Salz has a paper which makes substantial use of JuMP. Garrett Hagemann’s paper was done in Julia, as was one of mine.
I’m TA-ing for PhD IO at UT right now and trying to convince the R and Matlab users that they should switch.
Less finance specific, but more econometrics and regression analysis. I am working on a JuliaEconometrics organization which will have a suite of functionality for econometrics packages. For example, EconUtils.jl has generalized within transformation, first-difference, between, and singleton-handling for fixed effects and panel data. Using those, the main package provides a few estimators such as fast multi-way fixed effects, 2SLS, random effects (Swamy-Arora harmonic mean), uses GLM or Optim for generalized method of moments, and a strong variance covariance estimators. Some of the functionalities are scatter across a few packages, but the ecosystem should make them coherent and everything will play nice with StatsBase/DataFrames/StatsModels. Should be released shortly after packages up to 0.7-rc.
I’ll echo what @Tamas_Papp said: I think there are more finance and economics people using Julia than you think. When anybody posts a question, the question is more technical, so it is not necessary to state the entire context. For example, I have encountered several “speed bumps” when trying to transition my SAS work to Julia, and have opened issues, asked questions, contributed code, etc. I don’t think I ever mentioned that I was working with CRSP + Compustat data because that generally just contributes noise.
I am currently the only one using Julia in my department, but once things stabilize a bit I think I could convert several Matlab users.
EDIT: I guess I didn’t really address all of the questions in the OP.
Recursive dynamic programs. Parameter estimation. Working with big data. These are things that I think Julia has a lot of potential to dramatically improve my own workflow, even if not all of the pieces are completed polished yet.
Like I said, I’m the only one so far. There is interest, but people are on the sidelines for now.
For typical finance research, the data manipulation that was traditionally done in SAS or Stata is still a bit tricky to do in Julia. Also, lots of researchers use built-in packages in SAS or Stata to do their econometrics, and that is still coming together on the Julia side.
In order for Julia to become mainstream in the enterprise, it needs to be 1.0. The psychological barrier of building software on something that’s constantly changing isn’t very appealing to many companies. I appreciate the core dev team are working very hard to put it together and I cannot stress enough how important it is.
I suppose a follow up question for you folks is whether there is anything we can do to make Julia a much more effective tool for industry. I personally would like to see more Julia out there, especially in my domain, but I don’t know how to make it a hospitable language.
The knee-jerk response is better packages, though I don’t know that that would be the best thing. Thoughts on that?
Among academic economists who care about fast-running code, the adoption is pretty good, I think. One area where julia can definitely make progress is the ease of data manipulation (for that, Stata is still king). Currently I do my data cleaning and linear regression work in Stata, and switch over to Julia when I estimate structural models and do simulations.
I’m dreaming of a world where julia solves that two-language problem. Ideally, there would be an implementation of a Stata-like language through macros for data manipulation. I’m thinking of something like
@manipulate myDataFrame {
// Stata-like code
drop if var1 == NA
summarize var2
bysort categorical1: egen var3 = mean(var1)
}
I can’t imagine that this would be too hard to do, and the benefit to me (and like-minded empirical economists) would be immense.
Oh, and support for seeing DataFrames in Atom/VS Code would be amazing too.
Take-up at my institution has been alright. One other economist (@floswald) is teaching it to grad students, and my feeling is that it’s slowly replacing Matlab for anything non-Dynare.
While I agree that there is a still gap in data manipulation, I prefer dplyr-like pipelines over a DSL. It’s easier to understand the logic, more flexible, and easier to debug.
At my company, there’s some concern about not being able to find Julia talents in the market.
I don’t completely disagree with that but as Julia’s syntax is so clean and easy to understand, any people that know python should have no problem picking up Julia.
Second, if the academia continue picking speed then we will be able to see people coming right out of school knowing how to build fast Julia programs and solving differential equations!
@tk3369, you have the dplyr / pipepline like syntax in both dataframesmeta.jl and Query.jl
#dataframesmeta
x_thread = @> begin
df
@transform(y = 10 * :x)
@where(:a .> 2)
@by(:b, meanX = mean(:x), meanY = mean(:y))
@orderby(:meanX)
@select(:meanX, :meanY, var = :b)
end
there is still some work in terms of efficiency to be done. see this PR on the Query.jl package. Achieving something as performant as R data.table will probably be always hard, but there are some big benefits to using a single language. @davidanthoff knows all about this (of course).
!