That’s sum up near to 120 k$, half of the price of a decent julia debugger estimated by Stefan Karpinski. Not so bad.
I personally estimate that tickets for everywhere / anything conference is becoming very overpricey. But am totally ready to support good oss project thru fundraising.
Now that #22631 was closed and all the PRs that #22631 was refactored into were all merged I’m curious what is the status of PARTR parallel runtime? Does it mean it’s implemented now? Would really help to get an update on this. Thank you!
PARTR definitely exists within Julia, but that’s just the scheduler and adjustments to Tasks to make them able to run on other threads. Taking a look at the multithreading label on the Github repo should give you an idea of what’s still missing: Issues · JuliaLang/julia · GitHub
However, I’ve seen some examples of multithreading working for simple uses on #parallel in Slack and here on Discourse; it seems that, if what you’re doing doesn’t require I/O, regexes, and other not-currently-threadsafe things, it’ll mostly “just work”.
An update, since apparently some people are gatekeeping Julia adoption on the completion of these compiler work items: these items are all complete as envisioned at the time this was posted (and have been for a while). Additional details follow.
This is always ongoing, of course, but the compiler is very well debugged — the unreachable is quite hard to reach these days.
This was already pretty good in 1.3 and now in 1.7 we can say that it’s quite done. Moreover, the ecosystem has largely adjusted to the presence of multithreading as well, which is not a compiler work issue, but was very important. Multithreading is a very real and present thing in Julia and its packages now. It is widely and effectively used.
Compile-time latency, aka “the time-to-first-plot problem”
making compilation faster
This is done. The compiler got about and order of magnitude faster between Julia 1.0 and Julia 1.6. Depending on the speed of your system, time to first plot is now about 5-10 seconds, where it was originally 30 seconds to a minute or more.
This doesn’t mean that latency is solved because people are greedy and they don’t want to want to wait at all for their first plot. But it does mean that working on making the fully optimizing compiler faster is complete and it’s time to work on other approaches.
caching more things
The biggest change here is that precompilation is now done at package install time and in parallel using multiple threads. This helps a lot with the perception of latency when using packages. You can also generate custom sysimgs to eliminate all startup time, but more work needs to be done to make that tooling better and possibly generate sysimgs for each environment when Pkg does operations on the package versions in it. This is the least complete work item as more exploration needs to be done here.
Works quite well. The main problems at this point are that packages often do things that aren’t relocatable, which breaks this. But that’s an ecosystem issue, not a compiler issue. The tools exist to not do this — Artifacts, Scratch spaces, Preferences, DataSets — but packages need to use them.
Debugger
Works and is pretty smooth and featureful. Sometimes too slow because it’s interpreter-based but that was what was predicted by everyone involved. History: the previous debugger was fast but very complex and therefore flaky. There was a lot of pushback to have something simple and reliable — this is that simple, reliable debugger. Now we see why the earlier debugger took the approach it did.
Debugger has a compiled mode where it can run code fast but breakpoints don’t work. There’s probably something clever that can be done here to improve the perceived speed, but there’s no ongoing work on it and there won’t be unless someone takes it up as a project or decides to fund work on it.
There are a number of other debugging tools that have different approaches and properties, like Infiltrator, which lets you put a breakpoint in your code by using the @infiltrate macro at the point you want to stop.
CxxWrap has become the de facto way to wrap C++ code. Cxx was great but it’s a very complex approach that is highly intertwined with Julia’s internals and the author has not had time to maintain it. CxxWrap doesn’t have the fancy C++ REPL feature, but it’s straight forward and reliable and doesn’t depend on Julia internals to work. If someone does want to take on maintenance of Cxx, I’m sure that @keno would be happy to help get someone going with that — it’s a very cool project that would surely give someone a much deeper understanding of both C++ and Julia.
type checking/linting
The Julia VS Code extension has long included a good linter and JET does actual type checking of Julia code and more. Various independent parties have observed that JET seems much more capable and usable than similar tools in other dynamic languages due both to Julia’s type system and @aviatesk’s excellent work on the package. He and other members of the compiler team will continue to work on this functionality but it’s already state of the art.
Thanks for posting that summary @StefanKarpinski, it’s difficult to keep track of all the progress unless you’re a Julia nerd. I’d add that @atomic and friends are now exported for a more robust and general multithreading approach (AFAIU, I’m still trying to wrap my head around that myself)
The atomics work wasn’t even on this priority list, but yes, that’s been an important addition to the multithreading work that was finished recently. You can read about the design of atomics here and watch this lightning talk that Jameson gave about it last week:
The new feature in vscode, a gui that allows you to select modules to run compiled, largely solves this issue for me. The default choices are to avoid stepping into base and stdlib (except for some higher-order functions) and it makes the debugger only interpret user code. The end result is a rather snappy experience.
Sorry but not my experience. Doesn’t always stop at breakpoints and other times crashes where on REPL would work. Got better but still far from satisfactory.
I’m not experienced with compiling libcxxwrap but last time I tried it on 1.5 or 1.6 with Visual Studio and got errors which led me to that github issue. I wanted to use the Visual Studio path because it seemed more user friendly than Cmake.
You’re conflating two completely different things: compiling CxxWrap (one thing) and compiling Julia with MSVC (another thing). The second thing doesn’t work at the moment because no one has put the effort to fix the issues with MSVC, so if you use MSVC to build CxxWrap you’ll get a failure. But that’s not because CxxWrap itself doesn’t work (it does), but because you’re unable to build Julia with MSVC. People have no problems to build Julia/CxxWrap with different compilers (GCC or Clang, for example).
You are right. I should have been more specific and I apologize. As I said, I don’t know much about compiling stuff so I followed the documentation which stated that on Windows it’s easier to compile with MSVC. Sorry for hijacking the thread.
because one needs to compile