So it seems that ARPACK (briefly known as IterativeEigensolvers) has been removed from julia. I haven’t seen any discussion on this other than some comments on the slack so I thought I’d make a thread here.
Personally, I was a little annoyed that ARPACK was removed from Base and so in julia 0.7 if I wanted to do some basic linear algebra involving eigenvalues and eigenvectors I’d have to do using IterativeEigensolvers to load in the functions from the standard library.
Now, IterativeEigensolvers and ARPACK have been removed from the standard library and are now just packages that one can download if they so choose. The rationale seems to be that this will make installing julia faster and easier by removing a FORTRAN dependancy.
To me, this seems like a decision that merits some community discussion. It seems to put up more barriers between users and their intended usage. I see julia as primarily a language for numerical computing and I think linear algebra related functionality is central to that and should be as accessible as reasonably possible.
I’m interested to know what others think of this? Am I overreacting?
Edit: As was pointed out on Slack,
you don’t need IterativeEigensolvers/ARPACK to do computations on eigen(values|vectors) for dense arrays
Yes. Users of this functionality (e.g. me) will benefit from this since Arpack.jl can now be developed independently of Julia, and get bugfixes and features out faster. For non-users this is an obvious improvement since they don’t have to depend on this external library at all.
Taking the contrary position, this is a net negative for any environments that require code/author vetting before use, as it means that the codebase is now being maintained by a new/different organization/set of actors.
From a paperwork perspective alone, this was a very expensive decision, though I understand why it was made.
IMHO splitting it off is a good thing, because it makes development more nimble, as @fredrikekre says.
I’m involved in the Microbenchmarks. When the microbenchmark code was split off, I was at first a little disappointed to lose my avenue for contributing to the main Julia source tree. But all of a sudden it was a lot easier to track issues, file PRs, and get them accepted. And after a while I was granted ability to merge PRs. So now I can contrubute a lot more, and the pace of development has improved. It’s a good thing
I think it’s a good thing, but it badly needs to be offset by some mechanism to make it more convenient to use, because remembering to install 10 packages and add 10 lines of using to .juliarc every time you change computers is not fun. Perhaps metapackages that would install a few “recommended” packages and put appropriate using lines to .juliarc?
My understanding is that with the new Pkg, you just create “projects” for, well, projects, which take care of dependencies in a reproducible and standardized manner.
The idea is to create a SparseLinearAlgebra package that will pull in a lot of stuff such that users won’t have to load ten package and remember what ARPACK is.
@Tamas_Papp This is less about projects and dependencies than about suddenly having to know what the eigenvalues of random matrices are and running the REPL with eigs(sprandn(10,10,.1)). If I’m thinking about something else and just want to do a quick check, I don’t want to have to google three pages to remember what packages and what invocations I have to use to get it working. Of course there is always a way to things in the proper manner, but these ways are usually ignored by users in favor of the first answer that pops up on google
However, I think it’s wrong to throw in the basic SparseArray stuff, such as SparseMatrixCSC with the linear algebra packages.
Lots of people depend on the structures for their compactness, but don’t ever use linear algebra.
Please reconsider moving the basic structures to a 3rd party package.
you don’t need to if there are proper metapackages. Just have a meta package Sparse that pulls in all other packages that you want using Revise.jl. You even then just need a single using statement.
The good thing is: You can be the one creating and maintaining this metapackage. And you can do so now.
@anon94023334: Why don’t you just say what your restriction is, so that one can help you? Are you not allowed to write to the home directory? The best thing then is to make a julia distribution, where the package dir is in the Julia dir. Then you can prepare you julia distribution with all the packages that you want and then distribute that within you organization. The other way is to build a custom system image where all packages that you require are in.
Whatever this means, it does not fit to the wish that Julia is scalable. You seem to think that Base code has a higher quality but there is no reason why that should be the case. The concept being applied here is modularization and it is really required for a scalable environment.
While this will probably be brought up in each and every post about this, I’m pretty sure it’s because Seth is in a very sensitive environment where he’s not allowed to use software just ecause he trusts it. Quite literally he will have to submit a statement about the software he wants to use, and then some department will go through that software, look at the authors (who are they, what software do they develop, are they involved in things that might make them a security risk) and the actual code itself. Say your package re-distributes a binary (as in the ARPACK case, BLAS, …), then they have to make sure that the distribution channels are safe, where is the source code for the binaries? Who compiles them?
I think as long as it is in stdlib, it’s fine (for Seth and other places that have similar vetting requirements) and I feel there will be a lot of advantages to moving things at least out to stdlib.
@pkofod: These requirements are in contradiction with having a modular Julia, where not only the code is modular but (most importantly) the development is modular. It is also in contradiction to the entire Pkg system which encourages modularity.
I’m not arguing for or against anything (though if I have to reveal my preference, I’d personally prefer a super slim Base), I’m trying to explain why some particular Julia users might be cut off from using Julia completely from such decisions. Few are subject to such requirements, so few might realize where Seth and others are coming from.
Sure I get that. But within such sensitive environments it would be really risky using such a moving target which julia currently is. Stdlib will certainly change further in 1.x and this will be no fun in such an environment.
Regarding the „10 using statements“ in juliarc. Would it be possible (and meaningful for certain users?) to have „using stdlib“ which would load all stdlib packages? After all that was implicitly the case in 0.6 (Distributed, Random, LinAlg, Core, Dates, …)
Or, the windows installer (I do not know much about common procedures of the other OS‘) could ask: which stdlibs do you want to have always loaded (these would then be put into juliarc either as using or import)…
@bernhard: I think this is an excellent idea which is inline with what I describe as “metapackages” above. I would encourage that you open an issue on github to request this, or even better, make a pull request. It would require integration of
