Distributing own julia scripts

New to Julia
1

Suppose one is getting started with Julia, but has made a series of scripts to pass on to someone else, that has just installed julia for the first time on its machine.

Without having to specify which Packages are to be used to make your scripts work, and without say to him nothing on packages he should install to be able to run the scripts … because he is more beginner than you, for example … can you pass to him a “needs.jl” script that will import all the packages you need?

Is such an approach correct, if you say to your friend “write in command line julia.exe needs.jl” to be sure he will have all package installed in his machine?

For example a script of this type, I mean:

println("\n\nThis script will add the necessary packages\nfor the operations, please wait ..\n\n")

import Pkg

Pkg.add("NameOfFirstPackageOneNeed")
Pkg.add("NameOfSecondPackageOneNeed")
...
Pkg.add("NameOfLastPackageOneNeed")

Thanks for replies.

2

Don’t do this. It is very convoluted, and can break easily if any of those packages has a release that breaks that API.

Put your code in a project, with a Project.toml, and ask your friend to pkg> add it directly from the URL. Then have a runtime shell wrapper script which calls julia with that as the active project:

julia --project=@. main.jl
13 Likes
3

@Tamas_Papp, Can you clarify how this would work without git? Let’s say you share with them a dropbox folder (I know, I know) or email them a .zip file.

4

Unzip the zip file. Start julia with julia --project=/path/to/unzipped/folder or just cd to the folder and start with --project, run import Pkg; Pkg.instantiate(). Done.

4 Likes
5

If you really want to have the script automatically install dependencies, you can do something like the following at the top of your script:

try
    @eval using MyDependency
catch
    @warn "Unable to import MyDependency. Instantiating project."
    Pkg.instantiate()
    @eval using MyDependency
end

This is somewhat brittle and probably a bad idea though.

6

This will break if there is an incompatible version of MyDependency that happens to be available.

If someone insists on a single, self-contained script, it could call Pkg.instantiate() every time it is run. This is a small overhead, but may be acceptable in some contexts.

The alternative is

but it is somewhat more heavyweight than a script.

7

It would have to be a pretty long-running script for Pkg.instantiate to be small overhead unfortunately.

julia> @time Pkg.instantiate()
  7.608510 seconds (12.10 M allocations: 566.187 MiB, 5.94% gc time)

But you’ve still convinced me that try-wrapped using is highly brittle rather than “somewhat”.

I suppose if you really wanted to make something self-contained without using PackageCompiler, you could always vendor all dependencies in and just alter DEPOT_PATH at the top of your script. That’s a lot of overhead just to avoid having someone run Pkg.instantiate() though.

8

Try running it for the second time, it should be much faster. That said, this is not something I would do.

The key problem is that “standalone scripts” are not really a meaningful concept in Julia, as extra information is needed to resolve “simple” code loading like

using SomePackage

Depending on the environment, it may not even load the same package (ie different UUID). Casually running a script without this information may just work, or may break, in an fashion that is essentially random (because it depends on global state).

1 Like
9

The 7.6 seconds was the second time running it (in a fresh Julia session).

10

Are you using 1.4 or 1.3?

11

Pkg.instantiate() tends to be slow on Windows. Probably due to the antivirus’ intrusive snooping in all the git files behind the scenes.

12

I would put your code, the Project.toml, and the Manifest.toml in a zip file, send it to your friend, and make sure your “main.jl” script begins with

(only on 1.3 does this code work)

using Pkg
pkg"activate ."
pkg"instantiate"

I give some examples of this in a blog post here A Newcomers Guide To The Julia Package Manager | Andrew’s Programming Blog .

It’s unfortunate that instantiate seems to be slow on Windows. Is it really that much slower?

5 Likes
13

1.3. Does 1.4 have improvements in this dimension?

14

just cd to the folder and start with --project , run import Pkg; Pkg.instantiate() . Done.

A related question:

I am working in an airgapped computer network. I have been trying to develop a package that can be used by all the computers on the network. I would like to have all computers on the network be able to import the package from a git repository on one specific computer.
How do I do this?
I’ve been under the impression that, from any computer, I would add the line
push!(LOAD_PATH, "\\\\ComputernamePath\\drive\\collection_of_repos\\my_repo")
to the julia startup script (or run it at the top of a file), then run Pkg.install(my_repo). This doesn’t work.
Frankly, I have no idea how to do this despite substantial time searching.

Any thoughts?

15

Do you mean “develop” rather than “install”? Then this should work:

Pkg.develop(PackageSpec(path="\\\\ComputernamePath\\drive\\collection_of_repos\\my_repo"))?

What do you get?

16

Julia can’t load code from a git repository directly (using http/git protocols), it has to be in the filesystem.

I would consider either a local registry that mirrors packages, or a DEPOT_PATH pointing to a location on a network drive.

17

You might be interested in using DrWatson, which automates the process of making your code-base a project, see here: Project Setup · DrWatson

18

Would you please elaborate on how to implement both of your potential solutions: (1) create a local registry that mirrors packages; (2) push!(DEPOT_PATH, “location_on_network_drive”).

The workflow I we are currently using (please tell me if there is a better way):
(1) we have a bare repository on one computer.
(2) to develop the repository, each computer has a local repository that pulls from/pushes to the bare repository.
(3) When we have a version for a “package” we are developing, we pull from the bare repository to a single non-bare repository in a folder on one computer containing all packages we are developing (I put “package” in scare quotes because I think I have the appropriate directory structure, .toml file, and such for this module for it to be a valid package, but I frankly don’t know what I am doing).
(4) we would like for all other computers on the network to be able to import this “package”, not for development, but for using on different projects.

A related question:
Because of our airgap, we are currently spending a lot of time manually installing packages on each computer. Is there a way to mirror all julia packages to a computer on the network and have julia installations on each computer look to this mirror for pkg.install(“…”)? This would save us a lot of time.

Thanks so much!

Thank you!

  • I should add that if Atom’s project folder is the “package”'s folder, then the following is in fact working:
    ] activate .
    using packagename
19

Please look at the docs and earlier discussions, eg

https://julialang.github.io/Pkg.jl/v1/registries/

20

Why not consider the Julia Computing tool which is made just for this situation?
https://juliacomputing.com/products/juliateam

disclosure - I have never used Julia Team.