# Where can I follow compiler work progress?

**URL:** <https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503>\
**Category:** Internals & Design\
**Tags:** question, cxx\
**Created:** [July 29, 2021, 3:43pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503 "2021-07-29T15:43:22Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![gitboy16](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gitboy16/32/24906_2.png) [@gitboy16](https://discourse.julialang.org/u/gitboy16)\
**Post date:** [July 29, 2021, 3:43pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/1 "2021-07-29T15:43:22Z")

</div>

Hi,

I would like to follow the progress done on Julia with regards to slow compilation time.  
I found the following list of work:

> [@Compiler work priorities](https://discourse.julialang.org/t/compiler-work-priorities/17623):
>
> In response to I thought I might write a little post about the rough priorities of the compiler team: Correctness finding and fixing compiler and inference bugs Multithreading [non-copying task stack switching](https://github.com/JuliaLang/julia/pull/13099) (done) [new PARTR parallel runtime](https://github.com/JuliaLang/julia/pull/22631)[locks for I/O operations](https://github.com/libuv/libuv/issues/1595) other thread safety Compile-time latency, aka “the time-to-first-plot problem” making compilation faster caching more things Compiler-related packages and tools [PackageCompiler](https://github.com/JuliaLang/PackageCompiler.jl) Debugger [Cxx](https://github.com/Keno/Cxx.jl) type checking/linting J…

Is there a web page where I can see the milestones being completed or how each steps in the post above are being addressed?

I am currently the only one using Julia in my company. I use it sometimes for rapid prototyping. We are only allowed to use Python, C/C++ and Java in production. After review of the Julia language by IT and Management, they decided that it shouldn’t be use for any project but they allowed confined usage. Most of the reason they did not allow it are actually mentionned in the link above so I would like to be able to track the progress made on the language as I find it sad not to be able to use it in production.

Thank you

---

<div class="post-metadata">

**Author:** ![mbauman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/mbauman/32/31082_2.png) [@mbauman](https://discourse.julialang.org/u/mbauman)\
**Post date:** [July 29, 2021, 4:05pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/2 "2021-07-29T16:05:09Z")

</div>

The latest (as of 1 hour ago) is in the [live State of Julia](https://www.youtube.com/watch?v=IlFVwabDh6Q) JuliaCon talk [edited to reflect the new location; thanks @rashid]. That’s of course not going to track progress going forward, but it can give you a sense of the state of affairs right now.

Individual work-items may appear on the Julia repo as issues and/or pull requests and/or labels; it depends on which specific aspects you’re interested in.

---

<div class="post-metadata">

**Author:** ![brett-mahar](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/brett-mahar/32/27671_2.png) [@brett-mahar](https://discourse.julialang.org/u/brett-mahar)\
**Post date:** [July 30, 2021, 12:02am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/3 "2021-07-30T00:02:14Z")

</div>

If you use rss feeds, you can follow changes to that compiler webpage (or any discourse link) by adding .rss, eg:

“[Compiler work priorities](https://discourse.julialang.org/t/compiler-work-priorities/17623.rss)”

(Discourse is changing the link, you might need to right-click and copy the link to see it).

---

<div class="post-metadata">

**Author:** ![ImreSamu](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/imresamu/32/20677_2.png) [@ImreSamu](https://discourse.julialang.org/u/ImreSamu)\
**Post date:** [July 30, 2021, 5:33am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/4 "2021-07-30T05:33:12Z")

</div>

> [@gitboy16](#):
>
> I would like to follow the progress done on Julia with regards to slow compilation time.

As others have mentioned , there are some related `labels` in the [https://github.com/JuliaLang/julia/labels](https://github.com/JuliaLang/julia/labels):

- [`latency`](https://github.com/JuliaLang/julia/labels/latency) - Compiler latency
- [`precompile`](https://github.com/JuliaLang/julia/labels/precompile) - Precompilation of modules
- [`codegen`](https://github.com/JuliaLang/julia/labels/codegen) - Generation of LLVM IR and native code
- …

And you can query the merged - related - pull requests:

- [`is:pr is:merged label:latency,precompile,codegen sort:created-desc`](https://github.com/JuliaLang/julia/pulls?q=is%3Apr+is%3Amerged+label%3Alatency%2Cprecompile%2Ccodegen+sort%3Acreated-desc)

---

<div class="post-metadata">

**Author:** ![rashid](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/rashid/32/9398_2.png) [@rashid](https://discourse.julialang.org/u/rashid)\
**Post date:** [July 30, 2021, 6:07am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/5 "2021-07-30T06:07:19Z")

</div>

The link doesn’t point to the state of julia talk which is [here](https://www.youtube.com/watch?v=IlFVwabDh6Q), but to the keynote.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [July 31, 2021, 4:05pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/6 "2021-07-31T16:05:42Z")

</div>

I would point out that compiler work is usually pretty inscrutable and it often takes months worth of obscure pull requests before the plan comes together and some externally visible changes manifests itself.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [July 31, 2021, 4:11pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/7 "2021-07-31T16:11:40Z")

</div>

Everything in that post is done at this point.

---

<div class="post-metadata">

**Author:** ![Seif\_Shebl](https://avatars.discourse-cdn.com/v4/letter/s/eada6e/32.png) [@Seif\_Shebl](https://discourse.julialang.org/u/Seif_Shebl)\
**Post date:** [July 31, 2021, 8:19pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/8 "2021-07-31T20:19:11Z")

</div>

A relevant screenshoot from [that important talk](https://www.youtube.com/watch?v=IlFVwabDh6Q) below, I can’t wait for **3) Array Optimizations**, only after which I can completely migrate from Fortran.

![image](https://global.discourse-cdn.com/julialang/original/3X/f/9/f9db6f57fbd22c6475fbd5b4626e094a2e47fe31.png)

---

<div class="post-metadata">

**Author:** ![viralbshah](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/viralbshah/32/54_2.png) [@viralbshah](https://discourse.julialang.org/u/viralbshah)\
**Post date:** [July 31, 2021, 9:42pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/9 "2021-07-31T21:42:07Z")

</div>

It would be helpful to learn what kind of array optimizations you are looking for, and what kind of applications those would enable.

-viral

---

<div class="post-metadata">

**Author:** ![gitboy16](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gitboy16/32/24906_2.png) [@gitboy16](https://discourse.julialang.org/u/gitboy16)\
**Post date:** [July 31, 2021, 11:03pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/10 "2021-07-31T23:03:53Z")

</div>

-A lot of work remain to be done on the compiler and caching. I believe it is Jan Vitek who said in one of the video of JuliaCon that it was one of its biggest problem. (Correct me if I am wrong) PackageCompiler is nice, very useful and probably one the reason I still have hopes in the Julia language.

-I would really like to see the ability to compile a Julia script to a self contained executable. What is currently available attachs too much things to the “executable”.

-It would be great to see multithreading in Julia be as good as openmp. Correct me if I am wrong but something as simple as a “num\_threads” does not exist in Julia.

-The ability to control how many asynchronous task are run at the same time would be nice. (I think I posted 2 questions on these topics on discourse, did not get an answer, also on github for the num\_threads…someone just closed the issue without really answering it)

-Cxx only works for version of julia 1.1 to 1.3. It would be amazing to be able to tap into C++ 11 to 17 capabilities as well as libraries like Boost. R for example grew a lot because of Rcpp.

Hopefully with time it will get better and people will start using Julia. In the meantime what I notice is that Python is getting stronger in all the companies I work with a bit like Excel 30 years ago and today it is still difficult to get people moving from Excel to Python. In some companies (like Finance) I still see 80% of people working with Excel!!

Anyway I love the Julia language and hope to see it grow!

Thank you for the work.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [August 1, 2021, 12:38am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/11 "2021-08-01T00:38:37Z")

</div>

> [@gitboy16](#):
>
> Correct me if I am wrong but something as simple as a “num\_threads” does not exist in Julia.

If the question is whether Julia’s scheduler is static and not dynamic as openmp can be, then the answer is yes. If you’re asking how to set the number of threads in Julia, that can only be done at startup time with the `--threads` (or `-t`) flag.

> [@gitboy16](#):
>
> R for example grew a lot because of Rcpp.

I’m not super familiar with Rcpp but I thought it had the same approach as [`CxxWrap.jl`](https://github.com/JuliaInterop/CxxWrap.jl) which is currently widely used to interface C++ libraries and isn’t limited to Julia up to v1.3, so I don’t understand the strong need for `Cxx.jl` (besides the fact it’s cool)

---

<div class="post-metadata">

**Author:** ![tkf](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/tkf/32/17635_2.png) [@tkf](https://discourse.julialang.org/u/tkf)\
**Post date:** [August 1, 2021, 12:54am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/12 "2021-08-01T00:54:47Z")

</div>

> [@gitboy16](#):
>
> The ability to control how many asynchronous task are run at the same time would be nice.

This is impossible to support given the semantics of the Julia task since the scheduler must accept all tasks to guarantee forward progress. If you already asked this in Discourse, I think it’d be better to link the question (or open a new one if not) so that the discussion can be continued in a more focused manner.

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [August 1, 2021, 2:22am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/13 "2021-08-01T02:22:20Z")

</div>

CxxWrap has effectively replaced Cxx as the de facto way to call C++ code.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 1, 2021, 5:19am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/14 "2021-08-01T05:19:53Z")

</div>

> [@gitboy16](#):
>
> R for example grew a lot because of Rcpp.

Isn’t much of the reason for the success of Rcpp that it helps work around performance issues in R?

Wrapping c++ code is probably useful for other reasons as well, but Julia doesn’t normally need it for performance.

---

<div class="post-metadata">

**Author:** ![gitboy16](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gitboy16/32/24906_2.png) [@gitboy16](https://discourse.julialang.org/u/gitboy16)\
**Post date:** [August 1, 2021, 2:14pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/15 "2021-08-01T14:14:21Z")

</div>

Here is the link for my threads question:

> [@Control number of threads](https://discourse.julialang.org/t/control-number-of-threads/62887):
>
> Hi, Let’s say i have started julia with JULIA\_NUM\_THREADS=4, however in one of my loop I only want to use 2 threads because it gives better performance. How do I do that so that Threads.@threads only use 2 threads? Thank you

Yes only at startup is the problem whereas with openmp a specific number of threads depending on how long the loop is. Example, if the loop if only 100 elements it does not make really sense to have 10 threads working on it.

Regarding CxxWrap, I haven’t manage to make it work. Here is the link to my question:

> [@CxxWrap on windows](https://discourse.julialang.org/t/cxxwrap-on-windows/65178):
>
> Hi, I am trying to use CxxWrap on windows. I am trying to reproduce the “hello world” on the github page. Questions: 1/ where is #include "jlcxx/jlcxx.hpp" ? 2/ how do I compile to a shared library? ( i have gcc) 3/ does anyone have a step by step tutorial for windows? The documentation seem a bit light for beginners. Thank you

Answering @DNF while I am at it. You do not need Rcpp to wrap C or C++ in R. R is not slow except if you write code like you do in C which nobody does. What is nice with Rcpp is that it creates an amazing bridge between R and C++. It gives you access to a lot of libraries like Boost, Armadillo, Eigen, QuantLib, intel threading building block…etc and this just by “decorating” your c++ file. It takes care of the linking and building for you. Instead of wasting your time focusing on coding you can actually focus on solving a given problem that has not been solved before.

Wrapping C++ is not only useful, it is key. I personnaly have no time to spend re-writting C/C++ code in Julia or re-inventing the wheel, not to mention the headache testing and maintaining it when I can leverage on what has been done by other over the years. There are things even Julia rely on which are written in C/C++ and I can bet that these are are not going anywhere like llvm and curl. Whether I like or not these C/C++ language have been around before I was born and I believe will still be around after my death…

@tkf , below is my question on discourse. Thank you for letting me know it is not possible.

> [@Async limit](https://discourse.julialang.org/t/async-limit/62732):
>
> Hi, When using @async is there a way to limit the number of processes/queries being run at the same time? Thank you Kind regards

---

<div class="post-metadata">

**Author:** ![Ronis\_BR](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/ronis_br/32/50999_2.png) [@Ronis\_BR](https://discourse.julialang.org/u/Ronis_BR)\
**Post date:** [August 1, 2021, 2:21pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/16 "2021-08-01T14:21:57Z")

</div>

Sorry if I am a little off topic here, but I ported many algorithms that were originally coded in FORTRAN (SGP4, NRLMSISE00 atmospheric model, JB2008 atmospheric model, IGRF, etc.). Some of them make heavy use of arrays. In all Julia implementations, and using MKL, the result was faster than it was obtained by FORTRAN code.

---

<div class="post-metadata">

**Author:** ![DNF](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dnf/32/10191_2.png) [@DNF](https://discourse.julialang.org/u/DNF)\
**Post date:** [August 1, 2021, 2:58pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/17 "2021-08-01T14:58:45Z")

</div>

> [@gitboy16](#):
>
> Answering @DNF while I am at it. You do not need Rcpp to wrap C or C++ in R. R is not slow except if you write code like you do in C which nobody does. What is nice with Rcpp is that it creates an amazing bridge between R and C++.

You lost me here. Do you or don’t you use Rcpp to wrap C++?

As for slowness, my impression is that R is slow, which is why you use it to call into libraries written in fast languages. And this works well if your problem is a square peg and the library is a square hole.

Anyway, the reason I asked is that most of the time when I’ve come across mentions of Rcpp, the examples refer to implementing all the code yourself (at least that is how I remember it). This is never needed in Julia.

Using it for calling into pre-existing code it seems more useful to me. It’s just not a question of performance.

---

<div class="post-metadata">

**Author:** ![gitboy16](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/gitboy16/32/24906_2.png) [@gitboy16](https://discourse.julialang.org/u/gitboy16)\
**Post date:** [August 1, 2021, 3:49pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/18 "2021-08-01T15:49:32Z")

</div>

To write c/c++ code and link it to R you don’t need Rcpp. If I need to use existing C++ library I do use Rcpp but again it is not necessary. Rcpp makes it a lot easier for you. There are actually a lot of R package that wrap C++ without using Rcpp. If you want to wrap a c++ library you have the option of directly doing it using the R C API (hard way) or use Rcpp which does a lot of the leg work for you ( easy way). Plus Rcpp is not the only option, you have other packages ( example: inline) which work well and are in a way similar to Cxx.jl. So when I do use Rcpp is to call most of the time existing C++ code or libraries. Hope it is clearer.

---

<div class="post-metadata">

**Author:** ![cdsousa](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cdsousa/32/215553_2.png) [@cdsousa](https://discourse.julialang.org/u/cdsousa)\
**Post date:** [August 4, 2021, 9:17am UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/19 "2021-08-04T09:17:22Z")

</div>

> [@giordano](#):
>
> [`CxxWrap.jl`](https://github.com/JuliaInterop/CxxWrap.jl) which is currently widely used to interface C++ libraries and isn’t limited to Julia up to v1.3, so I don’t understand the strong need for `Cxx.jl` (besides the fact it’s cool)

`Cxx.jl` is not only cool, but it is also strongly helpful for doing quick and dirty experiments with C++ libs. It may not be robust to wrap libraries in the long term, but it enables the REPL driven development with C++ that we like so much in Julia!

I even used to rely a lot in `Cxx.jl` for quick checks when developing in C++ 🙂

I miss being able to quickly trying using some `C++`-only algorithms within my Julia code ☹ (I’m currently trying using `cppyy` through `PyCall`)

---

<div class="post-metadata">

**Author:** ![StefanKarpinski](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/stefankarpinski/32/24_2.png) [@StefanKarpinski](https://discourse.julialang.org/u/StefanKarpinski)\
**Post date:** [August 4, 2021, 1:39pm UTC](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503/20 "2021-08-04T13:39:19Z")

</div>

If someone else wanted to take over the maintenance of Cxx, I’m sure that @keno would be happy to provide guidance but the reality is that he is way too busy to do that these days. I don’t have a good sense of the amount of work required to make it work in current Julia versions but he may be able to provide an estimate.

[Next page](https://discourse.julialang.org/t/where-can-i-follow-compiler-work-progress/65503.md?page=2)