I want to understand how the GLM package works by looking at the source code.
I used the package manager’s develop command to get the code.
I asked for a local install, thinking it would go to my local directory. But instead it’s a few layers down from my Windows “home” directory (~/.julia/…). Is there a way to put it under my project directory? Even if there is, is that a bad idea?
The package makes extensive use of code from other packages. The package manager brought them in automatically, but apparently as binary packages. Which don’t help much if I want to see the source. I could chase through the references by hand and get a develop version of each, but is there are more automatic way to do so?
It may also matter that my intention differs some from that which the package manager develop is designed. The latter is about getting and modifying a package; I only want to browse the source.
I thought once I had the source code juno would help me with things like lists of types and functions, quick jumping to definitions, references or calls, but the documentation for juno doesn’t seem to mention such features. There is a “go to definition” shortcut which sounded promising, but in my limited attempts it didn’t do anything. Possibly this is caused by 1 or 2 (more likely 2–it obviously can’t show the source if there is no source available), or by the fact that I hadn’t loaded any of the modules since I just wanted to browse the source. What tools can be helpful in understanding and navigating source code from several modules?
Not sure what you mean by “asked for a local install”. If you deved the package, where it ends up is controlled by the JULIA_PKG_DEVDIR environment variable (see 3. Managing Packages · Pkg.jl). If you added the package, where it ends up is controlled by the DEPOT_PATH Julia global variable and the JULIA_DEPOT_PATH environment variable (see 9. Glossary · Pkg.jl). Or you can manually clone and then dev the local path like @PetrKryslUCSD suggested.
There’s not really such a thing as a binary Julia package, at least not in this sense. The source code for package dependencies is always downloaded; it’s just a question of finding where it is. One way to do so is as follows:
julia> using SIMD # for example
julia> pathof(SIMD)
"/Users/tkoolen/.julia/packages/SIMD/nqizO/src/SIMD.jl"
Note that the the nqizO part of the directory is an implementation detail (used to distinguish between different versions of the same package) and that you shouldn’t edit the source code of packages in .julia/packages. But you can still view the code.
In addition, there are a bunch of tools for exploring source code (all available without an IDE):
I would use the Pkg facility for installing a local development copy.
Enter into the Pkg prompt ]
Create a new Project directory (it cannot be named GLM):
generate myGLM
activate myGLM
develop --local GLM.jl
Then open a terminal window and enter the directory ~/myGM
In the dev directory you will find a git clone of GLM.jl
If you want to make changes to the code and test them out locally this is the way to do it.
Thanks to you and the others for very helpful info.
By local install I meant I used the --local option. The docs say the option sends installs into the dev directory of the current project. I thought that meant the current working directory, but it didn’t create a dev subdirectory there. Do I somehow need to set up a project? What is a project?
No, it means the current Project™ - which is minimally a folder with a Project.toml file (I think). But you need to ]activate it in order for you to be working in the project. When you start Julia, the project you’re working in is 1.x. See here
You can also ] activate . to start a project in the current working directory.
Reading the fine manual seems to be a bit of a 2-pass experience.
The basic description of pkg starts with an explicit, though informal and partial, description of environment, and then drops in project with no explanation half-way down, in the context of mentioning “your project directory”. The same material appears as section 2 of the full Pkg manual, but that manual doesn’t get around to projects until section 4. Section 3 also contains passing references to projects. Projects are defined (if that’s not too strong a word) in the glossary, section 8. Additional details appear in the discussion of code loading in the main julia manual.
This is all in the context of the intro to the Pkg manual which states
Pkg is designed around “environments”: independent sets of packages that can be local to an individual project or shared and selected by name.
So while reading sections 2 and 3 I thought I was reading about manipulations of environments, not projects. But apparently pkg is also the project manager.
I think the docs could be clearer, though of course that’s easier to say than to do.
I can’t resist an old joke about 2-pass algorithms:
An old lady gets on a bus and sits down next to a kid. She turns to him and asks “I’ve never taken this route. Could you help me get off at Clark road?”
Kid: “Sure. Just watch where I get off, and leave at the stop before that.”
On fuller reading and consideration I’m beginning to suspect the reason the documentation is hard to follow is that the things being documented are a bit of a tangle, with packages, projects and environments all getting in each others’ business.
Another way to say this is that their definitions overlap. I feel like there’s a post by Stefan somewhere that explains it real it well, but the way I think about it is:
A project is a folder with a Project.toml file and optionally a Manifest.toml.
Packages are a special subset of projects that contain code meant to be used by others, usually invoked by using.
An environment is what you get in a Julia session when you activate a project.
Not sure if these definitions are consonant with what’s in the docs, but this is how I see it and I think I have a reasonable handle on it (now, after a fair but if confusion starting out).
I originally had that as one of the first things in the documentation since I generally want to understand what terms mean before anything else, but it seems that people found it too technical and now it’s buried kind of deep in the Pkg manual.
Packages and projects are explicitly overlapping: a package is a kind of project (an application is the other kind). Environments are distinct; each project has an environment associated with it but there are other kinds of environments too, which is necessary since you don’t want to have to create a project for any set of packages you might want to use together.
That’s certainly helpful, but I have recollection of a discourse or slack post. Then again, I did some digging on discourse and couldn’t find it, so maybe I’m mistaken.
I haven’t found a use case for Applications yet. Is there something explicit inside Project.toml that let’s you know whether a project is an Application or a Package?
You need src/ProjectName.jl and name + uuid to using ProjectName if thats what you meant. Of course, a package environment is also often used as an application.
But if you are not using ProjectName, what’s the benefit/purpose of having a Project.toml with name = "ProjectName" and an UUID?
I also don’t quite understand what applications have to do with the package infrastructure. There is a single mention of the concept in the glossary, but otherwise it is not used, and I have not seen any examples.
Of course I can imagine an application that has its code in a package, and then has some file with
or something like that as the main entry point. But even in that case, the concept of an application seems orthogonal to the whole Pkg infrastructure — an application is a program (script), that may or may not use a package, but otherwise it is just like any other piece of runnable code.