That test is broken. There’s probably already an issue open for it. Three solutions are 1) disable the test for Project.toml format. 2) Make the changes that appear in the error report every time it is reported. 3) Look in Aqua for the code for this test. There is a tool (in Pkg I think) that does this formatting. Run this tool manually or in a script.
Is it broken though? I think it’s just complaining about the lack of alphabetical order, which (in my experience) is ensured by default? Did OP edit the Project.toml manually to get Aqua at the bottom?
Sometimes you have to edit a Project.toml by hand rather than using add and remove, for instance to manage a test environment (the newer method is still experimental and is buggier). Furthermore, the manual explicitly allows manually updating compat entries. There is no requirement for the entries in a section to be alphabetized.
If the entries are not alphabetized, I get:
julia> Aqua.analyze_project_toml_formatting(GraphsExt)
1-element Vector{Aqua.LazyTestResult}:
⟪result: 😭 FAILED: /home/lapeyre/myrepos/julia_repos/GraphsExt/Project.toml
Running `Pkg.resolve` on `/home/lapeyre/julia_repos/GraphsExt/Project.toml` will change the content.
But running resolve does not change the content. Deleting Manifest.toml and running resolve does not change the content. Project.toml is left untouched and the Aqua test will fail again with the same message.
Using the Pkg.compat command will alphabetize the file. But Aqua will also complain about the extras section, and I don’t know how to edit that except by hand.
Yes, sometimes the package manager alphabetizes Project.toml.
But Aqua is enforcing something that is neither documented to be nor intended to be invariant, and the error message that it gives is incorrect.
That is so weird! What did Aqua complain about in your case?
I agree that editing the extras section must be done by hand if you don’t trust the “new” test/Project.toml system (and it’s quite a bummer).
It would definitely be worth opening an issue on Aqua.jl, I don’t think it exists yet. FYI there are plenty of other bugs in the interaction with Pkg.jl:
If you try it, you might not find it weird at all. You can construct a valid Project.toml file, in accordance with the documentation, and it will function well with all of the Julia tools. Except Aqua, which will complain if its not alphabetized. Take an alphabetized file and switch the order of any two entries in compat or in extras. This mimics what you could do when constructing the Project.toml file. You can use add, then fill in compat by hand. Then do resolve, instantiate whatever. You are in a completely functional and legit state, but Aqua will complain. And as I said, it will claim (it doesn’t follow strictly logically from the error message, but close enough) that resolve will fix it, which is not true.
I poked around Aqua before posting and saw one or two of your links. I have seen this discussed before more explicitly. I don’t recall on which forum, GH, slack, etc.
EDIT: Also, to expand on what I wrote above, Aqua in fact uses Pkg.TOML plus a little hand rolled code to alphabetize the file. Its a handful of lines. You could organize this in your local workflow if you want to enforce this invariant. And if you trust that what Aqua is doing will not get out of sync with what add, remove, etc. (but not resolve!) do.
I don’t know what else you could do… add Example, rm Example maybe? That’s a bit silly. Maybe there’s something better. If I want this Aqua test enabled I use the Aqua diff output to realphabetize by hand. But, honestly given the state of Pkg, Aqua, and the TOML formatting tools, the best thing is to disable this test. And by “state of” I don’t mean to imply that they are deficient, it’s just that they don’t require or claim to require that the file be alphabetized, they just do it sometimes.
You could decide to enforce it as an additional style requirement, beyond what the standard Julia tools require. But then you’d want some reasonably automatic tooling to reformat it and document why you are doing it.
IIRC, if you have CompatHelper running in your repo, the compat section will get alphabetized for you at some point. So it’s arguably a good idea to start it out that way, to minimize diff noise.
But I also ran into this issue got a confusing error message when I first started using Aqua.
