Where to store custom packages?

Luigi_Marongiu · July 3, 2019, 12:55pm

dear all,

I am writing some scripts containing some functions that I could use in the future. Where shall I save the file? Can I simply load the script with using?

Thank you

fredrikekre · July 3, 2019, 1:46pm

Store it wherever you feel like, “install” it (i.e. make it available to using) by Pkg.developing the path to it.

Tilstandsrommet · July 4, 2019, 12:16pm

Hi again Fredrik.
I tried to develop my own package from my working directory, but it does not work:

(v1.1) pkg> develop C:/Users/sindre.abrahamsen/juliafiler
[ Info: Assigning UUID df5de59a-cb49-5d19-96de-9b3899e6645c to juliafiler
 Resolving package versions...
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Project.toml`
 [no changes]
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Manifest.toml`
 [no changes]

julia> using MyModule
ERROR: ArgumentError: Package MyModule not found in current path:
- Run `import Pkg; Pkg.add("MyModule")` to install the MyModule package.

julia> import Pkg

(v1.1) pkg> add MyModule
ERROR: The following package names could not be resolved:
 * MyModule (not found in project, manifest or registry)
Please specify by known `name=uuid`.

fredrikekre · July 4, 2019, 12:26pm

Whats the content of C:/Users/sindre.abrahamsen/juliafiler? Pkg expects it to be a package, e.g. something with a Project.toml and a src/MyModule.jl file which defines the MyModule.

Here is an example:

$ tree /tmp/local-packages/
/tmp/local-packages/
├── LocalPackage
│   ├── Project.toml
│   └── src
│       └── LocalPackage.jl

4 directories, 4 files

$ julia -q --project=$(mktemp -d)
(tmp.ABVpKGjmuJ) pkg> develop /tmp/local-packages/LocalPackage
 Resolving package versions...
  Updating `/tmp/tmp.ABVpKGjmuJ/Project.toml`
  [6b6a5c98] + LocalPackage v0.1.0 [`/tmp/local-packages/LocalPackage`]
  Updating `/tmp/tmp.ABVpKGjmuJ/Manifest.toml`
  [6b6a5c98] + LocalPackage v0.1.0 [`/tmp/local-packages/LocalPackage`]

(tmp.ABVpKGjmuJ) pkg> st
    Status `/tmp/tmp.ABVpKGjmuJ/Project.toml`
  [6b6a5c98] LocalPackage v0.1.0 [`/tmp/local-packages/LocalPackage`]

julia> using LocalPackageA
[ Info: Precompiling LocalPackageA [6b6a5c98-9e56-11e9-0c7c-6bf820af5e25]

julia> LocalPackageA.greet()
Hello World!

Tilstandsrommet · July 4, 2019, 12:59pm

I made a new folder

julia> pwd()
"C:\\Users\\sindre.abrahamsen\\juliamodules"

I developed the directory:

(v1.1) pkg> develop C:/Users/sindre.abrahamsen/juliamodules
[ Info: Assigning UUID 7f3e4f07-f9d7-511a-90fe-184c4798bfcf to juliamodules
 Resolving package versions...
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Project.toml`
  [7f3e4f07] + juliamodules v0.0.0 [`C:/Users/sindre.abrahamsen/juliamodules`]
  Updating `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Manifest.toml`
  [7f3e4f07] + juliamodules v0.0.0 [`C:/Users/sindre.abrahamsen/juliamodules`]

I did a generate MyModule to create the Project.toml and src folder. Then I tried to add the package:

(v1.1) pkg> add MyModule
[ Info: resolving package specifier `MyModule` as a directory at `C:\Users\sindre.abrahamsen\juliamodules\MyModule`.
   Cloning git-repo `MyModule`
ERROR: Git repository not found at 'MyModule'

Tilstandsrommet · July 4, 2019, 1:12pm

using MyModule does not work. Both with and without the preceeding dot .MyModule.
Below is the status of the packages. As you can see, the juøiamodules is included after i did develop

(v1.1) pkg> status
    Status `C:\Users\sindre.abrahamsen\.julia\environments\v1.1\Project.toml`
  [c52e3926] Atom v0.8.7
  [7073ff75] IJulia v1.18.1
  [e5e0dc1b] Juno v0.7.0
  [5fb14364] OhMyREPL v0.5.1
  [91a5bcdd] Plots v0.25.2
  [295af30f] Revise v2.1.6
→ [7f3e4f07] juliamodules v0.0.0 [`C:/Users/sindre.abrahamsen/juliamodules`]
┌ Warning: Some packages (indicated with a red arrow) are not downloaded, use `instantiate` to instantiate the current environment
└ @ Pkg.Display C:\cygwin\home\Administrator\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.1\Pkg\src\Display.jl:220

fredrikekre · July 4, 2019, 1:21pm

You have to do it the other way around, files have to exist at the time of develop. Also, it seems like you are trying to develop a folder where you keep multiple packages, you should develop the package folder itself.

Tilstandsrommet · July 5, 2019, 8:03am

Thanks! Now everything works! Even Revise works as expected!

Luigi_Marongiu · July 9, 2019, 10:13am

Sorry but, as a novice, I did not understand. The modules I am building are working in progress, so I would like to load them with using MyModule but without having to install it every time I change something. Since I understand that Pkg add uses official packages stored in GitHub, how can I install my own packages, which contain the modules? From the question of Tilstandsrommet looks to me that they need to be in a specific folder, for instance .julia\environments\v1.1\Project.toml. Can I create a symbolic link there to a more convinient folder, for instance ~/myScripts?

fredrikekre · July 9, 2019, 10:50am

No, store them wherever you want, and make them available to using with Pkg.develop, see Where to store custom packages? - #2 by fredrikekre

Luigi_Marongiu · July 9, 2019, 12:30pm

I got this:

julia> Pkg.develop(PackageSpec(path="/home/gigiux/Documents/scripts/SelMa.jl"))
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
   Cloning git-repo `/home/gigiux/Documents/scripts/SelMa.jl`
ERROR: failed to clone from /home/gigiux/Documents/scripts/SelMa.jl, error: GitError(Code:ERROR, Class:Net, unsupported URL protocol)
Stacktrace:
 [1] pkgerror(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:120
 [2] #clone#2(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::String, ::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GitTools.jl:107
 [3] clone at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/GitTools.jl:88 [inlined]
 [4] (::getfield(Pkg.Types, Symbol("##26#29")){Bool,Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1}})(::LibGit2.CachedCredentials) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:547
 [5] shred!(::getfield(Pkg.Types, Symbol("##26#29")){Bool,Pkg.Types.Context,Array{Pkg.Types.PackageSpec,1}}, ::LibGit2.CachedCredentials) at ./secretbuffer.jl:184
 [6] #add_or_develop#13(::Symbol, ::Bool, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/Types.jl:513
 [7] #add_or_develop at ./none:0 [inlined]
 [8] #add_or_develop#12 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:29[inlined]
 [9] #add_or_develop at ./none:0 [inlined]
 [10] #add_or_develop#10 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:27 [inlined]
 [11] #add_or_develop at ./none:0 [inlined]
 [12] #develop#19 at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:70 [inlined]
 [13] develop(::Pkg.Types.PackageSpec) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.0/Pkg/src/API.jl:70
 [14] top-level scope at none:0

julia>

fredrikekre · July 9, 2019, 12:36pm

This:

looks like the same misunderstanding as

Luigi_Marongiu · July 9, 2019, 12:48pm

Now I see! got it, thank you.

dhazeghi · July 9, 2019, 5:32pm

Is there a way to keep the source for multiple modules in a single directory? We have a large number of modules and relocating each one to its own subdirectory would be rather painful.

Tamas_Papp · July 10, 2019, 5:25am

AFAIK not. The loader code assumes that each package gets its own directory.

You could organize them into submodules of a module (it may be equally or more painful, though ).

Note that this is an uncommon setup — most people run version control, which is much more convenient in separate directories. If all those packages operate as a whole, then possibly they are one package, otherwise they should live in separate directories.

Topic		Replies	Views
How to add package from a folder (not registered) into the the project? General Usage question	10	924	November 14, 2021
Developing and using my own code General Usage	2	840	December 6, 2018
Recommended setup for developing packages General Usage package , development	19	2196	January 19, 2018
Package Development New to Julia question , package	4	540	July 17, 2021
JULIA_PKGDIR and Julia-0.7 General Usage	17	1954	January 30, 2019

Where to store custom packages?

Related topics