How can we evangelize Julia if we cannot point new users to a good list of all available registered packages?
I am using various versions of Julia on Mac and Linux platforms, and am willing to run list generators and upload to a site, if that’s all we need to do. There should be a site where one can choose the Julia version, OS platform, and get a simple long list like PackageEvaluator produced, with links to the github home pages. If we can get a few other volunteers to run the generators on other platforms (e.g. MS Windows), it should be straightforward to keep a list updated, perhaps weekly.
I don’t quite see how the list of all available package is useful to a new user—it’s like drinking from a firehose. A small, curated list would be ideal, and one can always ask on the forum, too. I would say that the packages I would confidently refer a new user to are not more than a hundred, maybe less.
Also, the lists you link to covered all registered packages — with the new Pkg framework, unregistered packages can be used quite easily, too.
Even currently abandonned projects are the result of effort put in by volunteers. I appreciate that you feel strongly about this (even though I am not sure I understand why), and it is fine if you want to revive one of these projects, or start a new one, but please consider using a different tone.
Thanks for your reply. I edited my top post to change the tone as you suggested.
For me, the long comprehensive list is best. I like to be able to search on a term, e.g. “protobuf” or “gpu” and find all the registered packages working on various things. Curated lists are fine too, but no substitute for a regularly-updated comprehensive one, IMO.
On Github, seaching for protobuf language:Julia works fine. Given that most open source packages on Julia are on Github, this gives you fair coverage; but Gitlab has similar functionality.
Even better: Just add a jl to your search term. Since Julia packages are named with a trailing .jl by convention, simply searching for protobuf.jl or gpu jl works great.
When I started to play around with julia, I scrolled through several times the list of packages at PackageEvaluator. I think when you don’t search for something, but you are interested in what is possible with the language, it is useful to have a complete list.
And I also agree that a smaller curated list would be ideal to show the state-of-the-art of the ecosystem.
using DependenciesParser
function lookingfor(str::String)
found = Array{String, 1}()
for pkg in DependenciesParser.data
if occursin(str, pkg)
push!(found, pkg)
end
end
return found
end
In general, the appropiate solution in my opinion is to have package metadata in the Project.toml, such as author, license, tags, description, etc. That way one can filter and discover packages based on the metadata (e.g., permissive license with maintainer with x reputation that does y). Hopefully, that gets incorporated soon with the new Registrator and full transitioned Pkg era. Other tools like a centralized review system or recommender systems would be nice too.
I totally agree with @ronubi here, and this if my major complaint about the whole Julia thing these days.
It is very very odd for pkg.julialang.org to be outdated. Many people access that and think that the language is just dead. I had to tell people about juliaobserver.com, which is a nice site, but sounds like not official to me, since it’s not under julialang.org . And if it’s offline, than it is useless.
It is also very odd to suggest users to write code to search for packages. How the hell can an outside user find out on his own that DependenciesParser exists and you can use it to search for packages?
Regarding searching for packages on github, I know many people who wants to use Julia but doesn’t even know that github exists. If we want outsiders to get to know Julia, it must exists one straightforward way to look for packages. Many people want to check if there is a package for something before even installing Julia. And I would guess that if we want the user base to increase, we must make it easy for non-developers (the data-scientists that are not developers that share code on github) to find out what is available in the Julia ecosystem.
Or, alternatively, just document how to search for packages on Github, Gitlab, etc. Maybe provide a simple website that does this in one step.
There is an important distinction between:
all Julia packages,
all registered Julia packages,
registered Julia packages which were selected as stable/useful (“curated list”),
above plus some security assurances thrown in (eg if you report malicious code, it will be taken down).
Most package lists maintained by various languages fall between 2 and 4 somewhere. But also some older languages have a single centralized package registry, either by design or de facto.
The advantage of Github and similar is that they may have a lot of readily available information that should aid the user in picking a package (outstanding issues, contribution stats, stars). It is not clear that duplicating this is a good investment of effort.
Nor is the idea that Julia has to be evangelized. Use it because it solves YOUR problem, then share your solution with others. Finding ways to inject that into every conversation is IMO counterproductive.
There are two items in the recent JuliaTeam vision blog post that will solve this problem: Centrally hosted documentation as well as search for docs and code.
A public instance of this should be out very soon.
Thanks. This bullet point on Search is informative:
Search [in progress]. Currently search and discovery of packages is a serious pain point in the Julia ecosystem. JuliaTeam will provide integrated search of documentation and code for all packages. This will let you find the package that does what you need, whether it’s a public open source package or a private package that your organization uses—they’ll all be searchable in a single place. Code search will match the capabilities of the Google Code Search of old.
And I wasn’t aware of the METADATA.jl (old way) vs General (new way) registry issue, which probably has a lot to do with why pkg.julialang.org hasn’t been updated.
@Tamas_Papp, while metadata is not within the scope of Registrator in general, this needs to happen first in order for METADATA/Attobot/REQUIRE to be drop so people can submit Project.toml and not have these bare-generated. Registrator does gets a bit into the maintainer-like with the user permissions to tag a release. Another issue is to find the universe of packages. In my own research I do this, but I usually care about production ready maintained documented packages for instance. These are mostly from the subset that is registered with a main registry (CRAN/CTAN/PyPI/General). The way to check those is to crawl the registry. However, due to the current state of things a few things can happen, a package might not be compatible with that Julia version, its repository might have been deleted or turned private, it might have been deprecated, etc. I usually run a script for JuliaEcosystem that uses Pkg to assess whether a package is installable (package and all dependencies compatible and available). The next step is to check whether it is working as intended which means to run the test matrix à la CI/PkgEval. The plan for new-era registries is for forking the release tag and hosting a copy making it immutable (i.e., author forfeiting right to take it down similar to a journal submission and such).
@felipenoris PackageEvaluator has been outdated since Julia v0.7/v1.0 came out (never supported beyond v0.6). The new project is NewPkgEval, but it is not run CRON and used only for assessment of breaking changes in the language to check what in the ecosystem (registered packages) might break. I have my packages CRON with Travis on the release version (decentralized), but hopefully a centralized system comes up at the registry level. PkgEval as well as JuliaObserver are contributed tools not affiliated/supported by the core language or JuliaComputing.
