I would like to follow the progress done on Julia with regards to slow compilation time.
I found the following list of work:
Is there a web page where I can see the milestones being completed or how each steps in the post above are being addressed?
I am currently the only one using Julia in my company. I use it sometimes for rapid prototyping. We are only allowed to use Python, C/C++ and Java in production. After review of the Julia language by IT and Management, they decided that it shouldn’t be use for any project but they allowed confined usage. Most of the reason they did not allow it are actually mentionned in the link above so I would like to be able to track the progress made on the language as I find it sad not to be able to use it in production.
The latest (as of 1 hour ago) is in the live State of Julia JuliaCon talk [edited to reflect the new location; thanks @rashid]. That’s of course not going to track progress going forward, but it can give you a sense of the state of affairs right now.
Individual work-items may appear on the Julia repo as issues and/or pull requests and/or labels; it depends on which specific aspects you’re interested in.
I would point out that compiler work is usually pretty inscrutable and it often takes months worth of obscure pull requests before the plan comes together and some externally visible changes manifests itself.
A relevant screenshoot from that important talk below, I can’t wait for 3) Array Optimizations, only after which I can completely migrate from Fortran.
-A lot of work remain to be done on the compiler and caching. I believe it is Jan Vitek who said in one of the video of JuliaCon that it was one of its biggest problem. (Correct me if I am wrong) PackageCompiler is nice, very useful and probably one the reason I still have hopes in the Julia language.
-I would really like to see the ability to compile a Julia script to a self contained executable. What is currently available attachs too much things to the “executable”.
-It would be great to see multithreading in Julia be as good as openmp. Correct me if I am wrong but something as simple as a “num_threads” does not exist in Julia.
-The ability to control how many asynchronous task are run at the same time would be nice. (I think I posted 2 questions on these topics on discourse, did not get an answer, also on github for the num_threads…someone just closed the issue without really answering it)
-Cxx only works for version of julia 1.1 to 1.3. It would be amazing to be able to tap into C++ 11 to 17 capabilities as well as libraries like Boost. R for example grew a lot because of Rcpp.
Hopefully with time it will get better and people will start using Julia. In the meantime what I notice is that Python is getting stronger in all the companies I work with a bit like Excel 30 years ago and today it is still difficult to get people moving from Excel to Python. In some companies (like Finance) I still see 80% of people working with Excel!!
Anyway I love the Julia language and hope to see it grow!
If the question is whether Julia’s scheduler is static and not dynamic as openmp can be, then the answer is yes. If you’re asking how to set the number of threads in Julia, that can only be done at startup time with the --threads (or -t) flag.
I’m not super familiar with Rcpp but I thought it had the same approach as CxxWrap.jl which is currently widely used to interface C++ libraries and isn’t limited to Julia up to v1.3, so I don’t understand the strong need for Cxx.jl (besides the fact it’s cool)
This is impossible to support given the semantics of the Julia task since the scheduler must accept all tasks to guarantee forward progress. If you already asked this in Discourse, I think it’d be better to link the question (or open a new one if not) so that the discussion can be continued in a more focused manner.
Yes only at startup is the problem whereas with openmp a specific number of threads depending on how long the loop is. Example, if the loop if only 100 elements it does not make really sense to have 10 threads working on it.
Regarding CxxWrap, I haven’t manage to make it work. Here is the link to my question:
Answering @DNF while I am at it. You do not need Rcpp to wrap C or C++ in R. R is not slow except if you write code like you do in C which nobody does. What is nice with Rcpp is that it creates an amazing bridge between R and C++. It gives you access to a lot of libraries like Boost, Armadillo, Eigen, QuantLib, intel threading building block…etc and this just by “decorating” your c++ file. It takes care of the linking and building for you. Instead of wasting your time focusing on coding you can actually focus on solving a given problem that has not been solved before.
Wrapping C++ is not only useful, it is key. I personnaly have no time to spend re-writting C/C++ code in Julia or re-inventing the wheel, not to mention the headache testing and maintaining it when I can leverage on what has been done by other over the years. There are things even Julia rely on which are written in C/C++ and I can bet that these are are not going anywhere like llvm and curl. Whether I like or not these C/C++ language have been around before I was born and I believe will still be around after my death…
@tkf , below is my question on discourse. Thank you for letting me know it is not possible.
Sorry if I am a little off topic here, but I ported many algorithms that were originally coded in FORTRAN (SGP4, NRLMSISE00 atmospheric model, JB2008 atmospheric model, IGRF, etc.). Some of them make heavy use of arrays. In all Julia implementations, and using MKL, the result was faster than it was obtained by FORTRAN code.
You lost me here. Do you or don’t you use Rcpp to wrap C++?
As for slowness, my impression is that R is slow, which is why you use it to call into libraries written in fast languages. And this works well if your problem is a square peg and the library is a square hole.
Anyway, the reason I asked is that most of the time when I’ve come across mentions of Rcpp, the examples refer to implementing all the code yourself (at least that is how I remember it). This is never needed in Julia.
Using it for calling into pre-existing code it seems more useful to me. It’s just not a question of performance.
To write c/c++ code and link it to R you don’t need Rcpp. If I need to use existing C++ library I do use Rcpp but again it is not necessary. Rcpp makes it a lot easier for you. There are actually a lot of R package that wrap C++ without using Rcpp. If you want to wrap a c++ library you have the option of directly doing it using the R C API (hard way) or use Rcpp which does a lot of the leg work for you ( easy way). Plus Rcpp is not the only option, you have other packages ( example: inline) which work well and are in a way similar to Cxx.jl. So when I do use Rcpp is to call most of the time existing C++ code or libraries. Hope it is clearer.
Cxx.jl is not only cool, but it is also strongly helpful for doing quick and dirty experiments with C++ libs. It may not be robust to wrap libraries in the long term, but it enables the REPL driven development with C++ that we like so much in Julia!
I even used to rely a lot in Cxx.jl for quick checks when developing in C++
I miss being able to quickly trying using some C++-only algorithms within my Julia code (I’m currently trying using cppyy through PyCall)
If someone else wanted to take over the maintenance of Cxx, I’m sure that @keno would be happy to provide guidance but the reality is that he is way too busy to do that these days. I don’t have a good sense of the amount of work required to make it work in current Julia versions but he may be able to provide an estimate.
(I’m currently trying using