About Julia's development policy regarding Windows

[uwestoehr]

@Benny

I would vote against tying PackageCompiler releases to Julia releases

You can replace PackageCompiler with any other package. My opinion is that if the community decides e.g. that FluxML should be a core component, why not?

Also, every project has to do PR and advertise itself. I found Julia via YouTube. And before starting a big new project, I inform myself. I watched some videos about Julia and there several features were advertised. In my case I decided for Julia because of FluxML and also because of Julia’s statistics and graphics capabilities.

There are different users and in my opinion the community is best in sorting out the strength of a project as it is the sum of different users.
Strengths are naturally advertised. I mean, Julia advertises all the time that it is faster than e.g. Python. That is fair and sensible. And if the community decides that statistics should be a core component, then some statistics packages can be promoted to be e.g. "Tier 1 "packages. Tier 1 means for example their compatibility with a new Julia version is tested before a beta release. “Tier 2” could be packages tested after beta but before release candidates. And of course the Tier classifications can be changed.

These are just some ideas on how the community could participate.

[Eben60]

@uwestoehr

My opinion is that if the community decides e.g. that FluxML should be a core component, why not?

Maybe because it is not the community who pays the core Julia developers?

[goerz]

@uwestoehr

My opinion is that if the community decides e.g. that FluxML should be a core component, why not?

And then what? Who is going to do the work to maintain FluxML as a “tier 1” project? Are you saying we can have a poll where a majority of votes determines what the core maintainers have to spend their time on? Quite frankly, that is absurd.

The people who do the work get to decide what the priorities are, nobody else. If you want to shift priorities, get involved in doing the work!

[Benny]

@uwestoehr

My opinion is that if the community decides e.g. that FluxML should be a core component, why not?

Putting aside that the community has never and probably will never decide that particular move is a good idea, feature bloat is a very bad thing, especially for such a foundational dependency. You would force everyone who uses Julia to install FluxML code, likely even load it on startup if a Base feature relies on it. You could also bog down development if both projects are forced to share releases. In reality, the community has already complained about the standard library for these reasons, and the current work is to make them more like independent packages, even if not entirely. This isn’t an isolated tendency; the Julia ecosystem is full of Core packages separated into lightweight dependencies, and other language ecosystems do so as well (SciPy → NumPy).

I don’t want to just shoot down that idea, I do see where you’re coming from. I don’t like that PackageCompiler is more vulnerable to Julia changes or that it doesn’t have as much staff as it warrants. Much of that has to do with PackageCompiler being more bloated than alternatives so not many users bother to venture beyond loading a julia process, and I don’t like that either. I also don’t think integrating the development and releases of PackageCompiler with that of Julia will solve any of this. I know I’m in good company there because people are bothering to develop juliac, an actual core feature.

[uwestoehr]

@goerz

I’m not sure if you realize that you’re arguing with Stefan Karpkinski about how to best manage “his” own project…?

Yes, I did no know him until now. However, I made my points and also proposals. I appreciate that he joined the discussion.

Personally I was once in the governing body of FreeCAD. But then I changed my job and had to deliver real-life projects in time and within cost frames and that way I got a different perspective and realized that it is good that we have in FreeCAD active forum users who “just use”, don’t deal with code. No matter when they joined the project or what their role is, they bring in new perspectives. YouTubers can be as important as coders as their viewers can give you unfiltered feedback. I remember a feature I was super proud of, that cost me much spare time and after a YouTuber presented it I read in the comments the feedback and was shocked. His viewers had different needs than what I read in our forum. Well, only a fraction becomes active in a forum. I hold the feature back, changed, polished and it was worth that journey.

I mention this as I think it is fair to make proposals about the Julia policy despite I am a newbie. Take my perspective as an impulse not as blaming, insisting or the like.

[uwestoehr]

@Benny

You would force everyone who uses Julia to install FluxML code

No, you misunderstand my proposal.
Imagine my proposal would be adopted and there are Tier 1 packages. That only means that before a beta is released the release team has to team up with the maintainers of these packages to verify their packages work in combination with Julia. In most cases the packages will have to adapt their code but this way also give direct feedback about issues.
Same with Tier 2, just that then the pre-RC phase is used to test these packages.

The final release is the same as done now, the tiering it is only about the testing process.

[Benny]

@uwestoehr

No, you misunderstand my proposal. Imagine my proposal would be adopted and there are Tier 1 packages

Acknowledged, thanks for clarifying. “Core component” would normally be interpreted the way I did, this term is better.

@uwestoehr

before a beta is released the release team has to team up with the maintainers of these packages to verify their packages work in combination with Julia.

That’s the purpose of betas. The PackageCompiler issues stemmed from an unexpected hole in testing and insufficient staff to fully resolve it in a timely manner. Moving the developer’s tests into alpha testing or before would not have helped, in fact more beta testers, even if more casual user than third-party package developer, might have caught it sooner. That’s broadly why I argued for more accessible betas, though it was more about making juliac feasible for anyone to try.

[StefanKarpinski]

@uwestoehr

No, you misunderstand my proposal.
Imagine my proposal would be adopted and there are Tier 1 packages. That only means that before a beta is released the release team has to team up with the maintainers of these packages to verify their packages work in combination with Julia

As has been said several times: we already do this for every single registered package, before even reaching the point of making an alpha. Every single new package test failure is investigated. By your definition, every registered open source package is already Tier 1.

[jules]

@uwestoehr

it is good that we have in FreeCAD active forum users who “just use”, don’t deal with code. No matter when they joined the project or what their role is, they bring in new perspectives. YouTubers can be as important as coders as their viewers can give you unfiltered feedback. I remember a feature I was super proud of, that cost me much spare time and after a YouTuber presented it I read in the comments the feedback and was shocked. His viewers had different needs than what I read in our forum. Well, only a fraction becomes active in a forum. I hold the feature back, changed, polished and it was worth that journey.

Before, you said that the community should decide certain things, and then you also gave this example.

I wanted to add that there’s a big difference between decisions and actions in open source development. Decisions don’t really mean much on their own, I’ve seen it many times that there was a clear outcome of some discussion either here or in github issues, but still nothing happened.

Things happen when people are motivated enough to get moving. In open source, due to lack of funding, that often means intrinsic motivation. But community decisions don’t necessarily create intrinsic motivation of developers. In general, you want fewer hurdles to contributing so that you make best use of the motivation that people have.

While “all features should work on windows at merge time” is a good goal in that it serves a wide community, the fact is that most of our core developers don’t use windows, so it creates additional friction for them if they have to consider Windows while working on experimental features. I’d rather they keep developing quickly with a narrower focus and once good ideas emerge, others can step in and help with the chores that are necessary to make things work well on all systems.

In your example, you held a feature back because you were “shocked” by feedback, that’s intrinsic motivation, even if negative. Not because you were told “we held a poll on YouTube and consensus decided that you now have to implement this and that before you can release your feature”.

[DNF]

@uwestoehr

My opinion is that if the community decides e.g. that FluxML should be a core component, why not?

You’ve advocated community decision making in several posts, but I’ve not been able figure out exactly what you mean. As far as I can tell, the community already does decide, and if FluxML is not a core component, it is because the community did not decide to make it one. The community decides by the process of being open to anyone who wants to participate and contribute. There are also open fora (like this) where anyone can voice their opinions.

Some posters have argued against your suggestions concerning community decision making, but perhaps they are simply making guesses about what you mean, and arguing against those guesses. Could you clarify what you actually mean when you say “the community decides”?

@uwestoehr

These are just some ideas on how the community could participate.

Your ideas seem mostly to revolve around what the community could decide, not how decisions would be reached.

@uwestoehr

I remember a feature I was super proud of, that cost me much spare time and after a YouTuber presented it I read in the comments the feedback and was shocked. His viewers had different needs than what I read in our forum. Well, only a fraction becomes active in a forum. I hold the feature back, changed, polished and it was worth that journey.

Isn’t this an example of you making a decision based on user feedback, because you became convinced that they made good points? If those youtube commenters had the authority to direct your work, even against your own best judgement, and you disagreed with the decisions, would you be content with that?

Or perhaps I misunderstood what you meant. That’s why I think the discussion could benefit from some clarification of your terms.

[paulmelis]

@StefanKarpinski

I’m having a harder time with the others. For example, you seem to want us to consider PackageCompiler a “core component” of Julia. It isn’t and we don’t want to do that. In the future, once juliac is no longer experimental, we will likely fold the functionality of PackageCompiler into juliac and then that will be a core component since we ship it with Julia. Until then, it’s not. I suppose we can more officially document what is or isn’t a “core component”, but this is the first time this issue has come up in the 15 years I’ve been working on this, so I’m not sure one person’s misunderstanding warrants that. You thought it was, there was a big discussion, now you know that it’s not.

This one seems to stem from some of the bugs encountered in PackageCompiler usage with the current stable Julia 1.11.x version, especially the lockup with multi-threaded use. I think the underlying issue is a difference in perceived maturity/stability of a package like PackageCompiler, versus its actual state, together with it being hosted under the JuliaLang organization on github.

This leads to certain expectations, which might be best to explicitly manage up front in e.g. the PackageCompiler readme. The current “PackageCompiler is a Julia package” (nor does it mention of being part of the stdlib) should already signal it’s not an integrated part of Julia, but the concept “package” is a bit vague in certain respects with Julia, as e.g. Pkg is also a package yet fully integrated.

[jishnub]

@StefanKarpinski

we already do this for every single registered package, before even reaching the point of making an alpha. Every single new package test failure is investigated

I deeply appreciate the efforts the julia team has taken into setting up this infrastructure, and their efforts at keeping track of nanosoldier reports. However, it’s still worth noting that 1300 packages are being skipped in the daily package evaluation runs, and runs for 700 other packages are being killed (often the tests are timing out). It might be worth looking into if this number might be brought down. Needless to say, this has to be a community effort.

[giordano]

@jishnub

1300 packages are being skipped

I’m moderately sure the vast majority of them are JLLs which don’t have tests to start with (downstream packages would exercise them anyway), and the rest are listed in PkgEval.jl/Packages.toml at ff3294147c090023b6b491fcd025500abdec8b82 · JuliaCI/PkgEval.jl · GitHub with an explanation for their exclusion (many if them require proprietary third-party software, MATLAB is one such case, or specific hardware, like CUDA.jl and AMDGPU.jl, or specific environment, like AWS). There isn’t too much to investigate here.

[Sukera]

@giordano

the rest are listed in PkgEval.jl/Packages.toml at ff3294147c090023b6b491fcd025500abdec8b82 · JuliaCI/PkgEval.jl · GitHub with an explanation for their exclusion

It’s interesting that in this thread numerous people say that PackageCompiler is “not official” and “just a package”, yet simultaneously considered by your link (and thus PkgEval!) to be so important to never ever be blacklisted:

Packages.toml

It’s even in a special list to give it more time to execute! If that’s not a sign that PackageCompiler is not “just a package”, I don’t know what is.

[goerz]

It’s a list of “important” packages. That doesn’t mean it’s part of Julia. Nobody would think that DataFrames is part of the Julia language, even though it’s obviously an important package in the ecosystem.

Maybe the difference with PackageCompiler is that PackageCompiler is very technical and tied to Julia’s internals. So it’s something that could (and maybe should) be part of Julia. And the Julia developers agree, which is why they’re working on making it part of Julia, in the form of juliac.

Also, if anything, the OP wants the core developers to treat PackageCompiler as “special”… and it turns out they do! So, everybody should be happy

[lmiq]

@giordano

, and the rest are listed in

Wouldn’t it be useful to have a standard mechanism to mark packages as deprecated? It would help this tests, among other things. For instance, there could be a flag in the Project.toml file indicating that in the latest release of the package. I’m imagining something like:

name = "MyPackage"
uuid = "e29189f1-7114-4dbd-93d0-c5673a921a58"
authors = ["..."]
version = "2.9.1"
deprecated = true

It would then be very easy to “undeprecate” a package, by releasing a new patch version removing the flag.

[giordano]

I feel like people are reading too much into the “officiality” of the list. I have no direct involvement in the whole PkgEval business but my understanding of an outsider is that the “deprecated” list refers to packages which are broken because they peered into Julia’s internals (many of them are related to compiler stuff), which were never guaranteed to be stable, and as a matter of fact these packages are unmaintained and no one is going to fix them and as such they’re bound to always fail in all PkgEval runs, so what’s the point of keeping testing them? The list I linked is more a pragmatic one, than an “official” one.

If you’re suggesting to an official deprecation mechanism in the package manager, that’s a totally unrelated topic, PkgEval shouldn’t care too much about that, I personally (and, again, I have no relation to the PkgEval infrastructure, so my opinion counts as much as anyone else) think it’d be worthwhile to still test “deprecated but otherwise valid/functioning” packages, to increase coverage.

[affans]

I think everyone is appreciative of your proposal here, but frankly it needs a bit more work.

A robust way to test software is through unit tests (i.e. test-driven development). When a Julia update is released, packages are tested against the update (as Stefan mentioned). In the case of PackageCompiler, it was seemingly missing the relevant tests that lead to your issue. If you believe that PackageCompiler is a tier 1 project, perhaps you could volunteer your time to write the missing tests. It certainly can’t be expected that core Julia devs or the volunteer package authors drop everything to write missing tests because a new version of Julia is coming out. By that logic, the progress of Julia would grind to a halt as everyone would just be writing tests for edge-cases all day long.

[uwestoehr]

Many thanks for the discussion and the input!

Allow me to sum up the issue from my perspective:

• as it is there is room fro improvement, I think that many agree
• it is not about blaming, but to improve the situation by making proposals, think about the proposals and then maybe at the next JuliaCon to discuss it and then implement something
• I mentioned some issues as example not to work and fix them particularly but as an example to trigger a discussion about the development policy.

Here is my proposal refined, having in mind to use the community:

• the community, (for example every discourse user with a certain trust level) can vote for packages to be in a Tier group. He has as many votes as the Julia team lead decided there should be e.g. Tier 1 packages.

• If for example there can be 10 Tier 1 packages, everybody has 10 votes, the top 10 voted packages become Tier 1, the next 10 packages Tier 2, there is no Tier 3 or below.

• The result is a list of e.g. Tier 1 and Tier 2 packages, that are for the community so important that they should be tested before a release candidate in a way that

  • all package functionalities work on the 3 major OSes, MacOS, Linux and Windows
  • the performance is not worse (execution speed, compile time etc.) except there is a technical reason. In that case the reason will be given in the release notes

• The Julia project lead can define how they handle Tier 1 and Tier 2 packages. For example if Tier 1 should be tested before beta, or after etc.

• Every year there is a new vote because things might change. That does not mean that packages are no longer respected, only that it won’t be a teamup with the release team and the package maintainers.

What would be the benefit?:

• Julia and and package developers are not the same, so there will be a teamup before the release to quickly/directly exchange test results, feedback, regressions, different user perspectives etc.
• the release announcements get more info. For example what is still experimental and why, what is the plan until the release, what regressions are known and the developers are aware of under the different OSes

Back to the examples I gave, this would improve the situation:

• the Julia project lead get feedback what is important for the community, thus it can steer the Julia development in that direction if they want
• new experimental features like --trim are directly in the release notes declared as experimental, and there would be the info, that it is known that it does not yet work under Windows. And there will also the info that it is listed as new feature because the development sees good chances to get it production-ready for the final release. Or the announcement can also be that --trim will not be production-ready for reasons also given in the announcement.
• If e.g. PackageCompiler, Pkg, Flux etc. are in Tier 1, then there cannot be a final Julia release before these packages works well with that new Julia under the major OSes. For the community counts that Julia + the desired packages work well together. For the example PackageCompiler its 60% performance loss with Julia 1.11 under Windows would either lead to another release candidate of Julia in which that issue is addressed in Julia code. Or if only code changes in PackageCompiler are necessary, the final Julia release would be hold back until the issue is resolved in the package.

---

This is my proposal. Maybe some will be taken out of it, maybe not. I made it to be constructive because complaining is easy, improving sometimes difficult. But to improve things, there must be different proposals/ideas/options that can be discussed and evaluated.