Julia sure looks like it needs a version of Hoogle

I’m new to Julia. Finding functions seems more difficult than I’d hoped.

Smalltalk lets me find references to Classes and implementors of method selector Symbols. I can also inspect instances of a class and browse the network of code attached.

Haskell has Hoogle which allows me to search Haskell libraries, external to my current environment, by type signature: FilePath → String. As you can see, it works pretty well.

As much as I love Smalltalk, I prefer the Haskell facility.

I tried to find a type conversion function for PosixPath -> AbstractString. People answered from experience and memory. Which is helpful, but not a general solution.

Julia has a lot of wonderful type information. it seems a shame not to use it in search.

Here’s what the interpreter/compiler (?) is trying to do when I try to convert a ::FilePathsBase.PosixPath to a String so I can use it as the second argument to download()

MethodError: no method matching String(::PosixPath)
Closest candidates are:
  String(::String) at boot.jl:350
  String(::Vector{UInt8}) at strings/string.jl:53
  String(::Symbol) at strings/string.jl:83
  ...

I have an answer to my specific conversion exercise. I want a solution to how do I find all the functions which transform a particular signature of types.

How and where do I propose such a thing - if it doesn’t already exist?

1 Like

A proof of concept might start with downloading the General Registry and using methodswith.

Julia functions are often so generic and polymorphic that it’s hard to determine function input-output without running type inference (and sometimes not even that is enough). You can get an enormous amount of hits, because people often write code that is completely generic, with only an inkling of an idea about what input/output it will take and produce.

When I replied to your question I didn’t just rely on memory and experience, I used the perspective shift I recommended. Don’t ask about types, ask about functionality. You want to convert? Search for convert! That’s pretty straightforward.

I don’t oppose your suggestion, but I’m not sure if it will be quite as useful as in Haskell, where it seems like input/output types of functions are more common and available.

2 Likes

To build on this, some sort of conglomeration of methodswith, @which, apropos, ?(x, y)TAB completes methods accepting x, y by timholy · Pull Request #38791 · JuliaLang/julia · GitHub and JuliaHub (which itself is a great resource) removes the need to go to memory for everything.

As @DNF mentioned though, something like Hoogle really relies on pervasive, constrained type declarations. In contrast, writing Julia is more like writing Template Haskell: many declarations can’t be evaluated without lugging the compiler around with you, and the potential surface area for instantiations is unbounded. That’s not to say such a system wouldn’t be useful, but it’d take a heck of a lot more effort and careful design than Hoogle because of all the aforementioned reasons.

3 Likes

Haskell has a a lot of inferred typing. But it search seems to work to some reasonable degree I cant argue the specifics.

I don’t think convert is a good example, it was just what I was looking for at the time. Without knowing - as you do, the particular vocabulary, I wasted a huge amount of time trying to do a trivial operation.

Both Google and Hoogle get a lot of hits but they rank them.

How does hoogle rank them? I’m having a hard time thinking of a meaningful ranking.

OK Thanks - what I needed to know. I dabbled a little in Haskell - I don’t know enough to be dangerous.

I am really so incredibly unproductive if I waste hours looking for such a trivial function, in what I’d is a fairly high-traffic area. I’ve got much more difficult stuff to find.

ZipFile.ReadableFile in its documentation has seek(z,0) which I cannot for the life of me find how to access. Once I read the stream, it is at its end. I cant re-read it, without seek(z,0) which0 is the only allowed argument value. I’d have called that reset

All the answers to my question came from people who already knew the answer, along with many hopeful attempts.

I haven’t found any way of browsing Julia source. Was string(::PosixPath) → ::String something I could have found?

By closeness of match I think… Relaxing the types somehow. Like Google I don’t know how it works.
My example of a naive search, by a beginner:

FilePath → String.

Search answers included some generic Typeclass parameters too a=>a -> String I think. It’s a few years since I did the lessons.

I really liked this dynamic book format:
Using Typeclasses

Julia does know the answer to your question:

julia> @code_typed string(PosixPath())
CodeInfo(
1 ─ %1 = Core.getfield(xs, 1)::PosixPath
│   %2 = invoke Base.print_to_string(%1::PosixPath)::String
└──      return %2
) => String

Is it possible to query the type of string(x::PosixPath) without actually instantiating PosixPath?

1 Like

Generally you can’t search by return types because that requires the compiler get involved (this is why I drew the Template Haskell comparison). However, the existing tools can get you partway there:

julia> methodswith(PosixPath; supertypes=true)
...
[110] print(io::IO, fp::AbstractPath) in FilePathsBase at /home/brianc/.julia/packages/FilePathsBase
...

Of course this requires knowing that string calls print under the hood, but that’s an issue in a Hoogle-like system as well.

I think one actionable insight from this thread is that packages ought to provide a full API reference in their docs. For example, searching for “print” in the FilePathsBase docs turns up nothing. It’s unfortunate that Documenter.jl does not provide this by default. GitHub - MichaelHatherly/Publish.jl: A universal document authoring package for Julia. kind of does, but hasn’t been adopted very broadly. As one HN commenter mentioned (and I mostly agree with), having auto-generated docs even for parts of the API the maintainers haven’t written more curated documentation for is really nice in other language ecosystems, especially if e.g. externally defined methods are being overloaded.

1 Like

Getting the compiler involved may be possible though. I wonder if JET can do it API · JET.jl

2 Likes

Thank-you all so much!

Complete documentation of the Libraries concerned would have left me in a much better place. Should I raise a PR?

Do I have to raise a PR on a library to ask a specific question?

Maybe there needs to be some sort of request for review process on docs.

Was there any way of discovering string() was the method I needed? I assume it was already in my activated environment.

There’s nothing wrong with making discoverability part of a language/environment design :slight_smile:

  1. Find a list of methods that take PosixPath instances. I’m not seeing string in methodswith(PosixPath, supertypes=true), don’t know why.
  2. For each one, check its return type on a PosixPath instance. julia> code_typed(string, PosixPath())[1][2] returns String.
  3. Filter to ones that return AbstractString subtypes.
1 Like

In fairness, string is a bit of a special function. This should be documented somewhere but string is defined for every type. It’s always possible to get a string representation of a struct because julia needs to be able to print every struct.

It probably doesn’t show up in docuementation because it is kind of knowledge, which definitely isn’t fair. I think this information should be added to the String documentation.

I wouldn’t add to the documented contract of stringification functions, IMHO they are messed up and Julia needs the flexibility to fix them without breaking contract.

Publish looks pretty interesting

Most environments do something similar with Strings.
In the Smalltalk we used to build we generally had two variants which usually delegated to one.

There were methods with selector: #printString, implemented as #printOn: aStream which gave a fairly standard printable representation of an object. What we really liked was a printString which when you fed it to the compiler, returned an object which was at least equal to the original. so if you printed a Point 1@2 it printed itself as 1@2 which was its literal.

it was easy to find though Point allImplementors: #printString which would give all implementations up the hierarchy.

I digress…

I think the Julia developers are aware that method discovery could use some work. But as many have mentioned in this thread, there is work underway to improve this tooling.

1 Like

Sure - thanks… I can see from some of the tools mentioned that people care. It takes a lot of effort.
I am really enjoying Julia.

1 Like