Where can I follow compiler work progress?

If you use rss feeds, you can follow changes to that compiler webpage (or any discourse link) by adding .rss, eg:

Compiler work priorities

(Discourse is changing the link, you might need to right-click and copy the link to see it).

1 Like

As others have mentioned , there are some related labels in the https://github.com/JuliaLang/julia/labels:

And you can query the merged - related - pull requests:

3 Likes

The link doesn’t point to the state of julia talk which is here, but to the keynote.

1 Like

I would point out that compiler work is usually pretty inscrutable and it often takes months worth of obscure pull requests before the plan comes together and some externally visible changes manifests itself.

3 Likes

Everything in that post is done at this point.

3 Likes

A relevant screenshoot from that important talk below, I can’t wait for 3) Array Optimizations, only after which I can completely migrate from Fortran.

image

It would be helpful to learn what kind of array optimizations you are looking for, and what kind of applications those would enable.

-viral

1 Like

-A lot of work remain to be done on the compiler and caching. I believe it is Jan Vitek who said in one of the video of JuliaCon that it was one of its biggest problem. (Correct me if I am wrong) PackageCompiler is nice, very useful and probably one the reason I still have hopes in the Julia language.

-I would really like to see the ability to compile a Julia script to a self contained executable. What is currently available attachs too much things to the “executable”.

-It would be great to see multithreading in Julia be as good as openmp. Correct me if I am wrong but something as simple as a “num_threads” does not exist in Julia.

-The ability to control how many asynchronous task are run at the same time would be nice. (I think I posted 2 questions on these topics on discourse, did not get an answer, also on github for the num_threads…someone just closed the issue without really answering it)

-Cxx only works for version of julia 1.1 to 1.3. It would be amazing to be able to tap into C++ 11 to 17 capabilities as well as libraries like Boost. R for example grew a lot because of Rcpp.

Hopefully with time it will get better and people will start using Julia. In the meantime what I notice is that Python is getting stronger in all the companies I work with a bit like Excel 30 years ago and today it is still difficult to get people moving from Excel to Python. In some companies (like Finance) I still see 80% of people working with Excel!!

Anyway I love the Julia language and hope to see it grow!

Thank you for the work.

If the question is whether Julia’s scheduler is static and not dynamic as openmp can be, then the answer is yes. If you’re asking how to set the number of threads in Julia, that can only be done at startup time with the --threads (or -t) flag.

I’m not super familiar with Rcpp but I thought it had the same approach as CxxWrap.jl which is currently widely used to interface C++ libraries and isn’t limited to Julia up to v1.3, so I don’t understand the strong need for Cxx.jl (besides the fact it’s cool)

This is impossible to support given the semantics of the Julia task since the scheduler must accept all tasks to guarantee forward progress. If you already asked this in Discourse, I think it’d be better to link the question (or open a new one if not) so that the discussion can be continued in a more focused manner.

2 Likes

CxxWrap has effectively replaced Cxx as the de facto way to call C++ code.

Isn’t much of the reason for the success of Rcpp that it helps work around performance issues in R?

Wrapping c++ code is probably useful for other reasons as well, but Julia doesn’t normally need it for performance.

2 Likes

Here is the link for my threads question:

Yes only at startup is the problem whereas with openmp a specific number of threads depending on how long the loop is. Example, if the loop if only 100 elements it does not make really sense to have 10 threads working on it.

Regarding CxxWrap, I haven’t manage to make it work. Here is the link to my question:

Answering @DNF while I am at it. You do not need Rcpp to wrap C or C++ in R. R is not slow except if you write code like you do in C which nobody does. What is nice with Rcpp is that it creates an amazing bridge between R and C++. It gives you access to a lot of libraries like Boost, Armadillo, Eigen, QuantLib, intel threading building block…etc and this just by “decorating” your c++ file. It takes care of the linking and building for you. Instead of wasting your time focusing on coding you can actually focus on solving a given problem that has not been solved before.

Wrapping C++ is not only useful, it is key. I personnaly have no time to spend re-writting C/C++ code in Julia or re-inventing the wheel, not to mention the headache testing and maintaining it when I can leverage on what has been done by other over the years. There are things even Julia rely on which are written in C/C++ and I can bet that these are are not going anywhere like llvm and curl. Whether I like or not these C/C++ language have been around before I was born and I believe will still be around after my death…

@tkf , below is my question on discourse. Thank you for letting me know it is not possible.

Sorry if I am a little off topic here, but I ported many algorithms that were originally coded in FORTRAN (SGP4, NRLMSISE00 atmospheric model, JB2008 atmospheric model, IGRF, etc.). Some of them make heavy use of arrays. In all Julia implementations, and using MKL, the result was faster than it was obtained by FORTRAN code.

4 Likes

You lost me here. Do you or don’t you use Rcpp to wrap C++?

As for slowness, my impression is that R is slow, which is why you use it to call into libraries written in fast languages. And this works well if your problem is a square peg and the library is a square hole.

Anyway, the reason I asked is that most of the time when I’ve come across mentions of Rcpp, the examples refer to implementing all the code yourself (at least that is how I remember it). This is never needed in Julia.

Using it for calling into pre-existing code it seems more useful to me. It’s just not a question of performance.

To write c/c++ code and link it to R you don’t need Rcpp. If I need to use existing C++ library I do use Rcpp but again it is not necessary. Rcpp makes it a lot easier for you. There are actually a lot of R package that wrap C++ without using Rcpp. If you want to wrap a c++ library you have the option of directly doing it using the R C API (hard way) or use Rcpp which does a lot of the leg work for you ( easy way). Plus Rcpp is not the only option, you have other packages ( example: inline) which work well and are in a way similar to Cxx.jl. So when I do use Rcpp is to call most of the time existing C++ code or libraries. Hope it is clearer.

1 Like

Cxx.jl is not only cool, but it is also strongly helpful for doing quick and dirty experiments with C++ libs. It may not be robust to wrap libraries in the long term, but it enables the REPL driven development with C++ that we like so much in Julia!

I even used to rely a lot in Cxx.jl for quick checks when developing in C++ :slight_smile:

I miss being able to quickly trying using some C++-only algorithms within my Julia code :frowning: (I’m currently trying using cppyy through PyCall)

If someone else wanted to take over the maintenance of Cxx, I’m sure that @keno would be happy to provide guidance but the reality is that he is way too busy to do that these days. I don’t have a good sense of the amount of work required to make it work in current Julia versions but he may be able to provide an estimate.

1 Like

Cxx.jl has been maintained by @Gnimuc for quite some time (up until Julia 1.3). See Current Cxx.jl roadmap · Issue #486 · JuliaInterop/Cxx.jl · GitHub and Plans for Julia 1.7 · Issue #490 · JuliaInterop/Cxx.jl · GitHub for some relevant discussions.

1 Like

I’m currently working on a thin wrapper package for interfacing Clang( like LLVM.jl to llvm). The plan is to rebuild Cxx.jl on top of it.

14 Likes