Is it worth to use Julia when you need several external non-Julia libraries?

Suppose that I want to develop a project, but several features that I need does not still exist in Julia and I must to use several R libraries (let’s say that about 60-70% of the logic of my project lies in the R libraries). Then, I would use Julia for the main logic, e.g. foor loops, use of data frames, etc.

Even so, Do you think that it is worth to use Julia for developing my project?

1 Like

Yes I definitely think so. 40-30% R code replaced by Julia will make a huge difference!
(assuming that 100% - 60-70% libraries makes 40-30% R)

2 Likes

I would be a little more conservative. If you are using Julia for speed, and you will call R functions inside your loops, maybe it is best to not make things more complex for a speed gain that will not happen.

However, if your plan is to slowly migrate the R code to Julia (by rewriting it, or by replacing by Julia libraries) then it can be worth it. Also, if you are using Julia just to use a sane language, or the main bottleneck of the code will be in pure Julia, it may also be worth it.

7 Likes

It also rather depends on the size of your project.
If the 40 percent of R code is a total of 40 lines, just rewrite it to Julia.

If it’s 40,000 línes, that will be more painful.

8 Likes

But even more to gain!

2 Likes

And maybe not painful at all, perhaps it is enlightening and fun?
I am coming from R to Julia, and that better describes it for me. On this way I skipped python :slight_smile:
(There are other paths I am on, e.g. .NET/C#, php… and from those I am still not heading to Julia, just for completeness)

3 Likes

Yes to this if those 40k lines are similar order of magnitude to Julia, but what makes Julia such a great ecosystem is that it’s so composable… So big R projects are almost always small R projects with big C/C++ projects underneath them (90% of the time installing R packages is spent running gcc).

Those big C projects are usually self contained, and they re solve many problems from the ground up.

If you write something similar in Julia, using Julian style, then you may pull the implementation of data structures from one package, and some generic algorithms to manipulate them from another package, and a parser from a third package, and graphical display from a fourth… the 40000 lines may become 7 Julia packages and 1500 lines of hand written Julia

so it depends.

3 Likes

Exactly, it depends of the size of R (or Python, etc) libraries that will be used in the project. In the enterprise, it is quite difficult to spent a lot of time rewriting libraries… it is a serious problem of productivity.

I’m a head data scientist and machine learning researcher of a private company and I’m anxious to migrate all my work to Julia. However, the main reason blocking me to do this is that I would need to integrate several non-Julia libraries into my Julia code. Then, I wonder the 2-language problem still persists for me.

Would I see the advantages of Julia speed if I only used Julia for programming the main logic of a project?

If all of the actual calculations are done by the R code then that is the limiting factor and the speed will probably be basically the same as R.

What kind of libraries do you need in R that are not available in Julia?

1 Like

For example some statistical libraries for computing reliability scores such as ICC, Kendall’ score, SEM. Also, libraries for conducting hypothesis testing with a lot non parametric test such as Friedman’s test, post hoc test like Hommer, Bergman Hommel, Shaffer, Finner’s tests, etc. Robust libraries for experimenting with a lot of machine learning algorithms like the ones included in sckiti-learn or caret libraries. Filters and algorithms for processing signals like the ones include in Scipy…

I avoid this stuff in favor of Bayesian methods so I don’t know what you’ll find here, I could say if you can recast these “named” tests in terms of some kind of simulation process, and code that simulation in Julia it’ll probably be fast.

But I’ll comment on this:

Where Julia is going to shine, for example GitHub - JuliaDSP/DSP.jl: Filter design, periodograms, window functions, and other digital signal processing functionality and JuliaHub

2 Likes

I think you should keep an open mind when it comes to how much “functionality does not exist in Julia”. Even if there is not a well-known package that can replace whatever package you are using in R, I would imagine there are plenty of tools here that can help you get something together. You might be surprised.

5 Likes

Just to sugggest you to use https://github.com/JuliaStats/HypothesisTests.jl for some statistical tests.

For machine learning, you can check https://github.com/alan-turing-institute/MLJ.jl. I expect to test it for some of my future ML works.

1 Like

Tx! I will take a look in these packages