ANN: JuliaHub — explore, run, scale

I’m pleased to announce JuliaHub, a service offered by Julia Computing for exploring the Julia ecosystem, running your Julia code easily in the cloud and scaling it to more and larger machines. The current iteration of the site focuses on the exploration aspect of this vision:

  • Home: see popular packages, new packages, and new package versions
  • Packages: search all registered public Julia packages by name and keywords
  • Doc Search: search the documentation of all registered packages in one place
  • Code Search: search the entire repo of all registered packages by regex or by semantic code search (i.e. it understands the structure of Julia code)

Users can log into JuliaHub as well, which currently doesn’t offer much additional functionality (you can download the current JuliaPro version and register packages via a web UI, which allows registering packages that aren’t hosted on GitHub), but in the future we will support running compute jobs on the cloud simply and easily from inside your favorite local Julia client.

Give JuliaHub a try and if you encounter any issues or have ideas for how to improve the service, please file issues here.

Stefan Karpinski
CTO, Julia Computing

63 Likes

This is pretty cool. Would it be possible to tweak the search results a bit though? Right now when you search “Graph”, the first result is the mostly-abandoned Graphs.jl package which is in JuliaAttic. LightGraphs isn’t even in the first page of results (though packages that support LightGraphs, like GraphIO, are).

If the intended use here is for exploration of packages within a domain (particularly if this is tailored to new users), I don’t think we should be highlighting packages in the attic. How to get the most relevant package names up top is a different (and presumably more difficult) matter.

2 Likes

I had actually already opened an issue internally about that when you mentioned it to me and @pfitzseb has a fix for the package search ranking that improves this, which should be deployed fairly soon. In short, the package search did not previously take package popularity into account, considering only fuzzy name match ranking. It will now include log(stars) in the ranking, which leads to LightGraphs ranking second in that search; Graphs.jl still ranks higher since it has a lot of stars and matches the text perfectly. In the future, once we have install data from package severs, we’ll be able to use that instead of (or in combination with) stars, as a better metric of popularity.

8 Likes

Do we want to be directing new users to packages in JuliaAttic, though?

JuliaAttic is just an org like any other and we don’t want to special case it. I tested Graphs.jl — it installs, loads and passes tests on Julia 1.4, so there’s no legitimate reason to ding it in search. If a huge number of people were using and installing some package in JuliaAttic, then we would want to list it first. Of course, we would also want to consider finding a better org for it, but that’s a separate issue. Once we can use current usage data to measure real popularity right. now, then historical stars won’t matter anymore.

3 Likes

I tried the package-search functionality and, searching for “integral” or “integration” or “quadrature”, neither QuadGK nor HCubature nor Cubature showed up. Is there something I should be doing differently?

No. Package search currently only takes the package name and organization/owner into account.After the fix @StefanKarpinski mentioned above is deployed we’ll also take the package description into account. You can however already filter by the “integration” tag, which will indeed list the three packages you mention.

2 Likes

Thanks a ton for this, it’s extremely useful! Could you say more on the semantic search (how it differs from words/regex)?

1 Like

We parse all Julia code we can get our hands on so that you can search for e.g. macro definitions for enter:

7 Likes

Neither Gaston.jl nor Gnuplot.jl are listed under the plotting tag. What is the process to tag a package so it gets recognized by JuliaHub?

It picks up the tags in the Github repo.

See this:

versus

6 Likes

Got it, thanks!

Once more regarding Gaston.jl: it seems like JuliaHub is hosting its own copy of the documentation, except that it stripped all the plots, making it much less useful.

Why not just link to the official docs?

We’re generating docs ourselves for a few reasons:

  1. It’s hard to find a link to a package’s docs.
  2. Package authors don’t need to worry about building/deploying docs themselves if we do it for them.
  3. We can still provide searchable docs even if a package doesn’t have Documenter set up.

Click the small “info” icon to access our logs (in Gaston’s case here) to figure out why something went wrong while building docs. If it’s something out of our control (e.g. a package requiring a program that’s not present in our build environment – gnuplot in Gaston’s case), you can add that package to


and we’ll link to your own deployed documentation.
3 Likes

Thanks for the information! I’ve already submitted a PR.

2 Likes

Ah excellent! I did look for a symbolic search option but didn’t notice the “quote” button (or maybe I didn’t think it was related). It’s easy to miss… What about making it more obvious with a button that says “Symbolic search” or similar?

1 Like

One of the things this should enable eventually is generating PDFs for package docs as well.

-viral

5 Likes

I think we should separate “Repo Search” from “Code Search” where the former is regex search of entire repos and the latter is semantic search of Julia code only.

2 Likes

@avik I’ve added tags (“topics”) to Gaston on github (two days ago), but it’s still not listed under “plotting” in JuliaHub. If you have a chance, could you please look into this?

image

It would be cool if we could read every package docs inside JuliaHub instead of being redirected, so that the user got a single consistent UI. I feel that I lose a lot of time jumping from github page to github page, having to get used to different webpage themes. I don’t know if that is even feasible though.

Nonetheless, JuliaHub is an awesome tool!

Should be updated soon, I think.

Don’t most packages just use the default Documenter.jl UI?