Where to store custom packages?

dear all,

I am writing some scripts containing some functions that I could use in the future. Where shall I save the file? Can I simply load the script with using?

Thank you

Store it wherever you feel like, β€œinstall” it (i.e. make it available to using) by Pkg.developing the path to it.

4 Likes

Hi again Fredrik.
I tried to develop my own package from my working directory, but it does not work:

(v1.1) pkg> develop C:/Users/sindre.abrahamsen/juliafiler
[ Info: Assigning UUID df5de59a-cb49-5d19-96de-9b3899e6645c to juliafiler
 Resolving package versions...
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Project.toml`
 [no changes]
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Manifest.toml`
 [no changes]

julia> using MyModule
ERROR: ArgumentError: Package MyModule not found in current path:
- Run `import Pkg; Pkg.add("MyModule")` to install the MyModule package.

julia> import Pkg

(v1.1) pkg> add MyModule
ERROR: The following package names could not be resolved:
 * MyModule (not found in project, manifest or registry)
Please specify by known `name=uuid`.

Whats the content of C:/Users/sindre.abrahamsen/juliafiler? Pkg expects it to be a package, e.g. something with a Project.toml and a src/MyModule.jl file which defines the MyModule.

Here is an example:

$ tree /tmp/local-packages/
/tmp/local-packages/
β”œβ”€β”€ LocalPackage
β”‚   β”œβ”€β”€ Project.toml
β”‚   └── src
β”‚       └── LocalPackage.jl

4 directories, 4 files

$ julia -q --project=$(mktemp -d)
(tmp.ABVpKGjmuJ) pkg> develop /tmp/local-packages/LocalPackage
 Resolving package versions...
  Updating `/tmp/tmp.ABVpKGjmuJ/Project.toml`
  [6b6a5c98] + LocalPackage v0.1.0 [`/tmp/local-packages/LocalPackage`]
  Updating `/tmp/tmp.ABVpKGjmuJ/Manifest.toml`
  [6b6a5c98] + LocalPackage v0.1.0 [`/tmp/local-packages/LocalPackage`]

(tmp.ABVpKGjmuJ) pkg> st
    Status `/tmp/tmp.ABVpKGjmuJ/Project.toml`
  [6b6a5c98] LocalPackage v0.1.0 [`/tmp/local-packages/LocalPackage`]

julia> using LocalPackageA
[ Info: Precompiling LocalPackageA [6b6a5c98-9e56-11e9-0c7c-6bf820af5e25]

julia> LocalPackageA.greet()
Hello World!

I made a new folder

julia> pwd()
"C:\\Users\\sindre.abrahamsen\\juliamodules"

I developed the directory:

(v1.1) pkg> develop C:/Users/sindre.abrahamsen/juliamodules
[ Info: Assigning UUID 7f3e4f07-f9d7-511a-90fe-184c4798bfcf to juliamodules
 Resolving package versions...
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Project.toml`
  [7f3e4f07] + juliamodules v0.0.0 [`C:/Users/sindre.abrahamsen/juliamodules`]
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Manifest.toml`
  [7f3e4f07] + juliamodules v0.0.0 [`C:/Users/sindre.abrahamsen/juliamodules`]

I did a generate MyModule to create the Project.toml and src folder. Then I tried to add the package:

(v1.1) pkg> add MyModule
[ Info: resolving package specifier `MyModule` as a directory at `C:\Users\sindre.abrahamsen\juliamodules\MyModule`.
   Cloning git-repo `MyModule`
ERROR: Git repository not found at 'MyModule'
1 Like

using MyModule does not work. Both with and without the preceeding dot .MyModule.
Below is the status of the packages. As you can see, the juΓΈiamodules is included after i did develop

(v1.1) pkg> status
    Status `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Project.toml`
  [c52e3926] Atom v0.8.7
  [7073ff75] IJulia v1.18.1
  [e5e0dc1b] Juno v0.7.0
  [5fb14364] OhMyREPL v0.5.1
  [91a5bcdd] Plots v0.25.2
  [295af30f] Revise v2.1.6
β†’ [7f3e4f07] juliamodules v0.0.0 [`C:/Users/sindre.abrahamsen/juliamodules`]
β”Œ Warning: Some packages (indicated with a red arrow) are not downloaded, use `instantiate` to instantiate the current environment
β”” @ Pkg.Display C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Pkg\src\Display.jl:220

You have to do it the other way around, files have to exist at the time of develop. Also, it seems like you are trying to develop a folder where you keep multiple packages, you should develop the package folder itself.

3 Likes

Thanks! Now everything works! Even Revise works as expected!

Sorry but, as a novice, I did not understand. The modules I am building are working in progress, so I would like to load them with using MyModule but without having to install it every time I change something. Since I understand that Pkg add uses official packages stored in GitHub, how can I install my own packages, which contain the modules? From the question of Tilstandsrommet looks to me that they need to be in a specific folder, for instance .julia\environments\v1.1\Project.toml. Can I create a symbolic link there to a more convinient folder, for instance ~/myScripts?

No, store them wherever you want, and make them available to using with Pkg.develop, see Where to store custom packages?

1 Like

I got this:

julia> Pkg.develop(PackageSpec(path="/home/gigiux/Documents/scripts/SelMa.jl"))
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `/home/gigiux/Documents/scripts/SelMa.jl`
ERROR: failed to clone from /home/gigiux/Documents/scripts/SelMa.jl, error: GitError(Code:ERROR, Class:Net, unsupported URL protocol)
Stacktrace:
 [1] pkgerror(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:120
 [2] #clone#2(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GitTools.jl:107
 [3] clone at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GitTools.jl:88 [inlined]
 [4] (::getfield(Pkg.Types, Symbol("##26#29")){Bool,Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1}})(::LibGit2.CachedCredentials) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:547
 [5] shred!(::getfield(Pkg.Types, Symbol("##26#29")){Bool,Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1}}, ::LibGit2.CachedCredentials) at ./secretbuffer.jl:184
 [6] #add_or_develop#13(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:513
 [7] #add_or_develop at ./none:0 [inlined]
 [8] #add_or_develop#12 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:29[inlined]
 [9] #add_or_develop at ./none:0 [inlined]
 [10] #add_or_develop#10 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:27 [inlined]
 [11] #add_or_develop at ./none:0 [inlined]
 [12] #develop#19 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:70 [inlined]
 [13] develop(::Pkg.Types.PackageSpec) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:70
 [14] top-level scope at none:0

julia>

This:

looks like the same misunderstanding as

Now I see! got it, thank you.

Is there a way to keep the source for multiple modules in a single directory? We have a large number of modules and relocating each one to its own subdirectory would be rather painful.

AFAIK not. The loader code assumes that each package gets its own directory.

You could organize them into submodules of a module (it may be equally or more painful, though :wink:).

Note that this is an uncommon setup β€” most people run version control, which is much more convenient in separate directories. If all those packages operate as a whole, then possibly they are one package, otherwise they should live in separate directories.