About Julia's development policy regarding Windows

General Usage
104

I’m not going to rehash again how we already test all registered packages for all Julia releases, and how community voting for packages is of little consequence on whether or not any particular work happens.

But it might be interesting to make something like this part of the annual Julia User & Developer Survey (organized by @ahclaster, I think). Along the lines of “which packages do you think are particularly central to the Julia ecosystem”. Nominations would happen during the feedback period and then everyone could vote in the actual survey. It would certainly be interesting data to have, and could inform some decisions on which packages to test with more scrutiny.

3 Likes
105

An actionable thing here could be to think about how to standardize performance benchmarks for a package, such that detection of performance regressions can be automated.

Thinking out loud for a little bit: To start, there has to be a way to specify a benchmark workload for a package. It could look similar to either test suites or precompile workloads, or something in between.

A complication with benchmarks is that you always need to compare two configurations—it’s not like tests where the pass/fail result can be decided uniquely for each combination of package version and Julia version. To make sure you’re only measuring changes due to Julia itself, you probably want to pin the package at a single version that’s compatible with both of the two Julia versions you’re comparing (this also ensures that the benchmark workload is identical, which is obviously important). In many cases, that version would have to be older than the current release, since new package versions often drop support for older Julia versions.

Next, if you want to create something like PkgBenchmarks.jl to complement PkgEval.jl, should that be opt-in (e.g., the community decides on 10 representative packages for which to monitor performance) or opt-out (like PkgEval, every package that defines a benchmark workload is included unless blacklisted). Should you only compare the PR/release candidate against the current release, or also against, say, the LTS version? (Seeing longer-term trends may prevent against some otherwise barely noticeable drift in the wrong direction.) And how big should the workloads typically be, i.e., what should the timeout be? Seconds, minutes, tens of minutes, hours?

Should the benchmarks focus on runtime performance only, or should they also measure precompilation time/load time/latency/TTFX? Should this be up to each package author or baked into the infrastructure somehow?

I’m not involved in Julia development or releases, but it certainly sounds to me like it would be useful to have a more systematic and automated way of monitoring actual performance impacts in the package ecosystem when making changes to Julia. However, someone has to do the work of setting it up and making decisions about all the finer points of how it’s going to work.

1 Like
106

Y’all really like wholecloth divining out what you think people should be doing and how that should work.

Throughout this thread I (and others) have been trying to describe the status quo. I’m really not trying to argue for or against any particular prescription — the world’s your oyster there — just that it’s helpful to first describe the status quo and work from there. Especially so when it’s other people doing the work.

Performance is checked for regressions automatedly and regularly. There does exist something like PkgBenchmarks that complements PkgEval — it’s called BaseBenchmarks.jl. It’s also run by nanosoldier. There’s an infrastructure to compare between commits and releases. Here’s an example. It is pretty focused micro-benchmarks, but there’s an open issue (and two work-in-progress PRs) trying to figure out how to test some packages and compilation time better. It’s hard, though, particularly since typical CI environments are quite noisy.

15 Likes
107

Howdy pardner! That’s me :index_pointing_up: I don’t think anyone has harped on the existence of PkgEval more than me here.

As you say, BaseBenchmarks.jl is focused on microbenchmarks, and the open issues/PRs on compilation. It doesn’t address benchmarks of representative package usage, which is on @uwestoehr’s wishlist. So in my last post, I allowed myself to think out loud about the possibilities and challenges of hypothetically automating that, and it looks like we arrive at the same conclusion:

No prescriptive intentions. Apologies for the missing citation of related work.

108

A discussion needs a basis. I think basis is that there is room for improvement. There are examples where things can be made better. And I started the discussion on how to do this.

I like discussions like this: after experiences and opinions are exchanged, the status quo is clear, people make proposals, the different proposals are evaluated, people think about them, refine the proposals (no matter if own or others), come up with new ones or retract their proposals later on. Every proposal is treated as something to consider, even when one might roll his eyes because others should not fear to make own proposals.
It is OK, and also normal to refuse a proposal as long as there is another or better one. It is however, not good to state a proposal is not necessary because nothing needs to be changed basically. This would imply there is nothing to improve.

Focusing on the status quo is understandable but it can end in blamings. In a company, a typical argument would be “There was an issue. Why do we have it at all and who can be blamed for it? - The person who implemented the status quo and those who did not change it already”. I had such discussions too often and they only lead to frustrations. Therefore the status quo is as it is, nobody is accused. The focus should be how to improve things, have a better QA AND also fun and a good feeling.

I think the basis of an OpenSource project is to appreciate every volunteer work that is done. Therefore nobody should feel attacked by discussions to change things. It is not about the current or past work, but the future.

Also, to encourage people to make proposals, the point who will do what work should be decided late in the proposal evaluation otherwise one can block himself. For example, for some it seems clear that the release is done by coders and coders are usually busy and fear to get more work. But a release manager can also really just be a manager, meaning no coder. Many companies work this way on purpose. The person or team to launch e.g. a medical engineering product is usually no engineer, no scientist, no doctor but e.g. a sales manager because his job is to connect the scientist with the marketing, the security paper work guys with the state officials, he has to organize a press conference, write or review the release announcement, use his contacts to the press to get the product mentioned there, organize that the social media and video channels will have content at the release date.
And software is not so different. From my experience doing a release was:

  • checking the list of new features on different computers (organizing others to do tests as well) the collecting bugs and open issues
  • labeling issues for importance and communicate important ones to the forum to get more persons working on them
  • contacting IT journalists, speak in advance with YouTubers
  • a lot of calls/meetings to teamup different people, translators, developers
  • writing and fine-tuning the announcement texts
  • encouraging in the forum people to test the new features and also the installation process
  • preparing graphics and animations to give YouTubers and bloggers something
  • updating the docs and assuring that the new features are well documented and, if possible, also already translated
  • …

Maybe there are persons around that are good at this and have fun to do such work.
For example my proposal include a teamup between the release team and the developers of Tier 1 packages - that is a task perfect for a manager.

Maybe I think out of the box. But why not :wink:. And in the end there is no hurry. I hope that other proposals are made and maybe there is also a discussion at the JuliaCon.

1 Like
109

A lot of that list seems appropriate for software applications like FreeCAD, not a programming language. To engage in meta-discussion for a bit, you ran into real and acknowledged shortcomings of the language (juliac) and the ecosystem (PackageCompiler) for something fairly significant (shared libraries and executables, especially of a reasonable size for integration). This has only been tolerable so far because a REPL workflow is well established for many usages (by other languages, Julia can’t take credit for this), and interop (Julia ↔ C/C++, Julia ↔ Python) has been usable even if less-than-ideal. But among the good or at least debatable suggestions, you made many demands that are too specifically informed by your FreeCAD development experience. Animations for YouTubers are just not nearly as useful for base Julia as they are for 3D modellers, for example, you’ll need to get into far more specific libraries e.g. Makie.jl for text and still images to become less efficient explanations than even simple animation loops, let alone videos with commentary and ads. It’s unfortunate that these topics ended up primarily explaining why many of the suggestions won’t improve development compared to what is done now, rather than the few suggestions that can make a difference (which they have in PackageCompiler, though the problem needs a whole lot more work to be resolved to anyone’s satisfaction).

5 Likes
110

I opened a new issue to track this idea.
Writing docs for Github-related things · Issue #5 · JuliaLang/devguide.julialang.org

Python has a separate website, Python Developer’s Guide, to document how to contribute to Python.

The devguide covers various aspects of the contribution process that you may come across: how to triage new issues, what the CI runtime environment looks like, how to become a core developer, the branching and release process, and so on.

The Github project JuliaLang/devguide.julialang.org is already set up, and there are some ideas but a lack of people actually writing something.

2 Likes
111

I made good experiences with StaticCompiler. There are some restrictions due static compilation, but there is much more possible than one might think of. I summarized some experiences here.

3 Likes
112

What you describe may work for some software projects, but note that Julia (and a lot of other FOSS communities) work in a different way.

In the Julia community, code almost always precedes discussion, let alone formal decision making processes (if applicable). Generally, if someone wants something done, they code up PR (or a new package), get feedback, refine it, let it rest for a while (when not critical), and refactor, merge, or close based on the experience.

This is because, as dominantly a FOSS project, we do not have the power to allocate a significant number of person-hours to a particular task. Proposals are cheap, but if no one wants to do the work, then it is unclear what happens. Seeing some code first at least clarifies that someone is interested in making it happen, and also gives an estimate about feasibility, outstanding issues, etc.

Generally, I would recommend that you spend some time in the Julia community before proposing radical changes to the development process. Consider the possibility that, as a newcomer, at this point you may not be in possession of all the information you need to improve things. If after a while you still think things should be done differently, you can get back to the issues. Note however that proposals from people who made solid contributions to the language are usually given more thought.

In the meantime, please report issues and make improvements, including PRs and changes to the homepage, wiki, etc.

12 Likes
113

A lot has been said in this discussion regarding PackageCompiler being or not being a “core” component. Different opinions were presented including the ones from Julia founders and core developers. I’ve been developing in Julia for more than 5 years and not considering myself a newbie. This is frustrating, however, to see statements that PackageCompiler is, in fact, not considered to be a core component despite being literally advertised on the very first page of Julia website. I would not even think that it is not a core component. It is being advertised on the website and on every JuliaCon with new features, etc. No matter what your definition of the “core” is it is simply not possible to deny that PackageCompiler is an essential part of Julia while being on the first page of the website.

image
image1920×1013 193 KB

It is also frustrating to see statements like

Julia is not ready for the kinds of standalone applications you (and I) are thinking of, and there exist better tools and ecosystems out there.

while the linked screenshot points to a video on YouTube dated 4 years ago. According to the video it was ready 4 years ago but apparently not good enough as for now?

I also do remember the sentiment of Julia solving “2 language problem” which still being advertised on every single JuliaCon, but really - not being able to compile and ship an application isn’t helping to solve the 2-language problem. Thats the whole point of the 2-language problem, being able to experiment but at the same time ship the code to production (which involves the compilation step).

juliac is a good direction, but in all honesty, while trying it out I don’t understand the benefits. It only compiles a subset of Julia, which can be statically analyzed (should have a static call graph). Why would I prefer that over C++, which has a very robust and established compilation toolchain? C++ has static multiple dispatch via function overloading and generic programming with templates. Now, juliac basically forces me to write code similar C++ in a different syntax with an experimental compiler that can randomly crash wiht no obvious reason. Why not putting the effort in improving the PackageCompiler instead of competing with established toolchains from other static languages? As I understand the --trim reduces the subset of possible programs in Julia even further. By quickly scanning the issues tracker it seems that even print a string is a problem for the --trim. Again, whats the benefit of using it over C++ or C or Rust which have very stable compilers for statically analyzable code?

I quite genuinely don’t understand the target audience of the juliac? It seems to be worse strictly than PackageCompiler? If the goal is to merge both in the long run, why to diverge in the first place? Why not simply put the --trim option in the PackageCompiler then and treat it as “core” component?

3 Likes
114

I’m not sure if you realize that you’re arguing with Stefan Karpkinski about how to best manage “his” own project…?

With all the respect to Stefan, you should not make such statements. It is not “his” own project. Julia is a community project where hundreds of people contribute. Multibillion companies depend on (like ASML). Many more smaller companies build their software stacks on top of Julia.

image
image1880×1138 161 KB

It is no longer “his” project, but a community effort and it is completely fine for a single community member to express frustration about how the process is going on and how it is being managed.

I really wish I found this discussion earlier, as it seemed to be old already and I didn’t have a chance to participate. Still have questions, for example this very statement from @StefanKarpinski

Core components of Julia are everything that ships with Julia itself. PackageCompiler does not ship with Julia, thus it is not a core component.

But isn’t it going to be shipped with Julia with the addition of App support in Pkg :: Juliacon 2024 :: pretalx ? On the JuliaCon I got the impression that it is going to be. Isn’t the apps feature directly interacting with PackageCompiler like here Apps · PackageCompiler? Are we getting yet another way of compiling apps from Pkg then that is not based on PackageCompiler?

I agree with sentiment from @uwestoehr that community should decide what is “core” and what is not. Doesn’t really matter if it ships or not with Julia. Again perfectly fine from a newbie to ask this kind of questions and starting the discussion.

For me personally, I would definitely include Pluto to the “core” for example. It is not included with Julia, but it is the most starred package in the Julia ecosystem and one of the reason why many people try Julia in the first place. Many students also use it while learning the language. Yet the core developers of Pluto (voluntarily) spend a lot of their time fixing the project when a new version of Julia is being released.

Another example of such a package which isn’t included in Julia, but which I personality would still consider to be essential and “core” is Revise. Revise isn’t even under JuliaLang org but I cannot imagine the language without Revise. Its the most core of the core of the language and is essential tool for any Julia developer. It would be completely insane to release a new version of Julia that breaks Revise or somehow has an significant regression and then say “yeah it’s not core component so it’s oke and can be fixed later”. The same level of attention can be of course given to PackageCompiler as to Revise.

11 Likes
115

Revise is indeed treated fundamentally differently from all other packages: it’s tested with Julia’s internal CI, alongside all the stdlibs that Julia ships with, on every single PR and commit. Does that make it a “core” package? I guess?

All other registered packages are tested both on-demand, periodically, and before every release. Yes, it’s possible for a Julia release to proceed if some registered package has failing tests — but those failing tests are investigated and diagnosed. The best way to ensure your favorite package is not missed in this process is by ensuring that its test suite is comprehensive, not flakey, and doesn’t depend upon Julia’s internals. It’s not uncommon for feature requests to Julia itself to come from the need for a stable API for a package to use.

To be clear: that’s never been an accepted rationale for breaking any registered package since Julia 1.0 was released.

7 Likes
116

How would this even work? Who gets to vote? How do we prevent ballot stuffing and gaming of the voting criterion by outside malicious actors? It was proposed (parenthetically) that “the community, (for example every discourse user with a certain trust level) can vote”. I guess this is not the worst voting criterion, but is this really a good criterion? I don’t think the algorithm that Discourse uses to assign trust levels is particularly great and certainly not something that should be baked into the governance of the project in such a fundamental way. I just perused the lists of level 4, 3 and 2 users and none of them seems like a sane delineation between who should be allowed to vote and who shouldn’t.

A much more plausible set of people would be people who have made a substantial number of contributions to the julia GitHub repo itself. But we already have a group for that: it’s the Julia committers group on GitHub. This is basically already who decides things like this. If you want me to, I can put the question to the committers channel:

Should PackageCompiler be considered a “core component” of Julia?

Decisions aren’t made by formal vote, but more by discussion and consensus. That’s a great form of decision making if you can make it work, and so far we have.

I think this makes the point I made previously quite well: everyone will have their own pet package that they think should be core. You use Pluto all the time, I never use it. I think it’s great, but I do not think Pluto working perfectly should be a criterion for making a Julia release.

As @mbauman said, Revise is treated specially. I guess that means we could label it as a “core package”?

Anyway, once again, this whole premise of “Tier 1 packages” is a bit flawed because we already do the thing that’s proposed for Tier 1 packages for every package…

21 Likes
117

This work would have to be done regardless of the release policy for Julia, so it is not clear how conditioning the Julia release on the work being completed would improve things. Moreover, what would be the rationale for making everyone else who does not depend on Pluto wait for the release?

4 Likes
118

I’m not going to try juliac until v1.12’s final release (and more importantly, its documentation), but from the talk given about juliac last year, I got the impression these sacrifices belong to the --trim option. From a few scattered discourse comments, I gathered that untrimmed juliac is essentially adapting PackageCompiler to a core feature of base Julia, so “improving the PackageCompiler” separately would just amount to the same work we’d instead have to install separately from another repo. If you also meant trimming without sacrifices by that, then that’s just impossible. For example, bounded AOT compilation just doesn’t allow for unbounded JIT-compilation of runtime dispatches, and if we’re on a system that accepts the latter, we don’t need trimming.

Seems obvious, but compiling Julia binaries, and finally with an option for practical sizes. A C/C++/Rust/etc developer who has no need for Julia code shouldn’t use juliac, even if juliac could magically be as mature as those languages and their tooling.

That does not involve PackageCompiler at all, despite the usage of the very common word “app.” Read 6. Apps · Pkg.jl for an overview and watch the more explanatory JuliaCon presentation you’re referencing (the main topic starts at 7:05 of the Youtube video, and a Q&A about static compilation at 26:18 really highlights the difference in approach).

1 Like
119

I tried using juliac in a real-world application and unfortunately ran into various issues, including random segmentation faults and general instability. While PackageCompiler has its own limitations and is quite slow, it worked more reliably in comparison. I also asked for help in the Julia Slack, where the need for a static call graph was emphasized. I hadn’t used the --trim option at that time, so it’s possible there was some miscommunication. I will double-check to clarify that.

EDIT: I think you’re right and static call graph is specific to the --trim option. That must have been misunderstanding in Slack thread where I asked for help, where I didn’t use the --trim option but got responses as if I used it.

The broader issue is the current lack of documentation — at least nothing that’s easy to find, even though Julia 1.12 is already in beta. Searching for “juliac” in the 1.13 documentation yields nothing. If I made any incorrect assumptions about juliac, I apologize — my feedback is based on hands-on experimentation, albeit without clear guidance. It’s good timing to explore and provide feedback, but in my case, everything I tried failed quite early.

PackageCompiler, by contrast, has much better documentation, which makes it easier to experiment with and adopt.

From what I’ve gathered, juliac currently lacks support for some advanced use cases — for instance, depot relocation. This is often needed in large projects with deeply nested dependencies. Even if you don’t use it explicitly, there’s a high chance that one of your dependencies will, potentially causing issues in juliac.

I also asked on Slack about providing a list of precompile statements, which I rely on since my application is designed with dynamic dispatch. I was told that this functionality isn’t supported in juliac yet — or at least it isn’t documented.

In this light, building a separate tool like juliac seems to introduce double work. PackageCompiler, while imperfect, is a working solution that has already addressed many complex real-world use cases over time. Recreating this maturity in a new tool is a long and difficult process. My concern is that, in the short term, juliac does not yet provide equivalent functionality. Investing more into improving and stabilizing PackageCompiler may be more efficient than starting over.

1 Like
120

That depends on the goals. The --trim option is very important for certain use cases and is not possible to implement in PackageCompiler (or any other package) since the trimming needs to happen in Julia’s core. Moreover, as I understand it, the trimming feature is, directly or indirectly, providing funding for Julia compiler developers in a way that polishing PackageCompiler wouldn’t.

But you are right that this is a longer term work and with juliac being marked as experimental I wouldn’t expect a polished experience in Julia 1.12.

5 Likes
121

Just to clarify — I don’t use Pluto either. I use VSCode for development. Pluto was simply meant as an example of how some packages become so widely used that they may deserve special consideration indeed or being a “core”, regardless of whether they are under JuliaLang or shipped with Julia. And indeed, there is already precedent for this in the case of Revise.

That was not my intention to focus on Pluto — it was just one illustrative example. The broader point is that packages with significant adoption could reasonably be treated as release blockers when regressions occur, even if they aren’t part of the standard Julia distribution or maintained under JuliaLang and even if not all people actually use it. And as you’ve confirmed Julia core team already does it so we are agreeing here.

What we disagree I think it that I always thought PackageCompiler that is a very critical component of Julia — similar in importance to Revise. This is what OP also wrote in his original message and this is how I found this topic originally, by googling and trying resolve problems with PackageCompiler.

It’s not shipped with Julia of course, but it’s featured on the Julia website and is regularly highlighted at JuliaCon. I guess I was also wrong it was a critical component of Julia? While not everyone may use it, it remains — as far as I know — the only viable way to bridge Julia into production workflows and help solve the two-language problem.

That said, it seems that issues with PackageCompiler are not currently being prioritized. Please correct me if I’m wrong, but my impression is that development focus is shifting toward juliac, particularly since juliac is intended to be shipped with Julia, whereas PackageCompiler is not. While juliac might offer long-term advantages, in the short term it doesn’t yet appear to be a full replacement — as I noted in my previous message.

If juliac is expected to eventually cover all the complex use cases that PackageCompiler supports today, it seems it would need to go through a similar development process. Perhaps it can do so more efficiently, but I genuinely don’t know — especially given the current lack of documentation, which makes it difficult to properly evaluate or compare the two tools.

I’ve also seen some comments suggesting that juliac and PackageCompiler are conceptually related or even intended to converge. If that’s the case, it would be great to avoid divergence from the start — for example, by having PackageCompiler.create_app(..., trim = true) internally delegate to juliac, with appropriate documentation of the behavior and limitations. That would help provide a smoother transition path and preserve the value of existing tooling.

122

I do believe it should be possible to implement in PackageCompiler. Similar to Revise which is very deeply integrated with Julia’s core and many features needed for Revise to work properly are actually implemented in Julia. Same can be done to support advanced options within PackageCompiler. The easiest form of implementation at the first stage would be PackageCompiler simply calling juliac under the hood with appropriate command line arguments.

123

Ah, I think I understand. PackageCompiler.jl isn’t a compiler. Heck, juliac.jl isn’t a compiler. The compiler here truly is julia itself. These two things are more or less (respectively) complicated wrappers that expose easier high-level workflows around calling julia --output-* and maybe other args and gathering artifacts and packages and precompiles and using a C linker and such.

The juliac script is easier to see because it’s only ~200 lines of code. The question is where those 200 lines of code should live. You can actually see that it uses some code copied from PackageCompiler.

So, yeah, there’s not been a huge amount of movement on PackageCompiler.jl itself. But look at all the issues and PRs in Julia itself that have mentioned PackageCompiler: GitHub · Where software is built

9 Likes