I am able to add and use this package without issue, but if anyone else at my company tries to use it, they get this error. How can I permit anyone with access to the network drive to use my packages?
Input:
Pkg.add(path="S:\\Julia\\Packages\\KM620")
Output:
ERROR: GitError(Code:EOWNER, Class:Config, repository path 'S:/Julia/Packages/KM620/' is not owned by current user)
Google says:
Git now checks if the owner of a repository directory matches the user executing Git commands. If they do not match, Git raises this error to prevent potential security vulnerabilities where a malicious actor could exploit a repository owned by another user.
You can explicitly tell Git to trust a specific repository path by adding it to the
safe.directoryconfiguration.git config --global --add safe.directory "/path/to/your/repository"
but my users will (likely) not have Git installed on their Windows systems.
Is there a way to bypass this security check from my end?
Not sure how to avoid this from your side, but two alternatives come to mind right now:
develop work for your use case? (I’m not sure: Does Pkg.add actually create a copy in the user depot when used with a local path? develop happily handles local packages also when they are not git repos, so it might work here.)Pkg.develop(path="<path/to/package>")
I agree with Sevi; a LocalRegistry and/or LocalPkgServer might be good options for you to consider.
Pkg.develop did work! It seems considerably slower though since now the code must run directly off the network drive.
I will need to figure out LocalRegistry.jl eventually, but I’ve yet to bite that bullet. I’ve looked through the documentation, and it’s all over my head. Seems a lot of work to just share a package.
What’s the difference?
Ah yes, so it seems like add really creates a local copy (the docstring from pkg>?add mentions the restriction that the package has to be a git repo). Using a shared directory directly is anyhow not a good idea since updates to the code will propagate to every user immediately. A hacky workaround would be that users first locally copy the package and then run develop on it 
It could be automated with a small Julia function or shell script though.
I haven’t used LocalPkgServer myself, but it’s also not required for LocalRegistry to work. This is my (limited) understanding of these two tools:
Pkg.add or similar, Julia will pull the latest version of this remote registry to the local machine and look up what the existing versions of the package are and where to find them (including dependencies, etc.). It’s really just a directory with files for each packages containing URLs and git commit information for each registered version and their dependencies/compat information. The repo will look like the one here, just that there are only the packages you registered locally: GitHub - JuliaRegistries/General: The official registry of general Julia packagesadding. In principle it’s not required if I understand correctly. When installing/updating/… a package, Pkg will look in the registry for the most up-to-date information, and then just take the package code from the URL (e.g. Github) stored in the registry. The package server somehow helps streamlining this process for potentially many different storage locations (Github, Gitlab, …) and reduce traffic over the network by caching stuff etc. This issue contains a comprehensive write-up (that I didn’t read entirely ) Proposal: Pkg & Storage Protocols · Issue #1377 · JuliaLang/Pkg.jl · GitHub But Julia can just find packages at standard URLs that point to git repos without a package server.The point is that Pkg operations work most smoothly when there is a registry holding all the relevant information about the packages – plus you get versioning of your packages for free. So LocalRegistry.jl lets you create such a private git repo holding this information and anyone who you give access to it can run Pkg.Registry.add(url="...") once to permanently add this registry to their Julia depot and afterwards they will (in principle) not notice anymore that a package comes from the General registry or from your local one.
That’s the big constraint though, that your registry needs to reside in some location reachable with an URL and with git support. In our case, we have a local GitLab installation that hosts this repository.
The actual package code then needs to reside in (potentially different) locations, but I’m pretty sure it also needs to be a git repo with corresponding URLs. If you just want to install a package from the local registry, Pkg.add should just work™, but if you want to register a new (version of a) package, you need to also have write access to the remote git repo. LocalRegistry.jl will take care of updating the git repo with the registry in a consistent way (through the function LocalRegistry.register which you need to call from the machine from which you want to register the current code to the registry.
My current workaround for this is that I keep the master development code on my local machine and a private GitHub repository. Then I just git clone and git pull to the shared server to release a new version.
I’m confused about the requirements and best setup options for me. I’m the only person doing any coding here, so we don’t have any infrastructure around these things. Can I have a public GitHub registry that points to local (private) server packages? I don’t want to have to set up GitHub accounts and ssh keys for everyone. Or should I set it all up on the network drive despite the recommendation not to?
“The recommended way to manage local packages and a local registry is by using a web-based git hosting service such as GitHub, GitLab, or Bitbucket - either public or private or in-house. However, it is also possible to do this on a shared or network filesystem, or on a single computer.”
Ah, I wasn’t aware that shared filesystems are also supported. But yes, you can also have the registry completely public and have it point to private locations (then everyone could add the registry, but when they try to install packages from that registry, they will be prompted for some kind of authentication – and note that the URLs/paths/… of your private packages will be visible in the public git repo).
I think setting everything up on the shared file system might be the most straightforward for your current situation, if you can ensure that the paths to the packages and the registry are identical on all machines of your users. But you could just try it with a dummy package.
The doc page* that you quoted shows a nice example: I just tried these to set up a registry on a shared Nextcloud folder and it seems to work fine. I am on Mac though, so I’m not sure what pitfalls Windows holds in store.
But I think the biggest issue will be local paths. When you register a package with LocalRegistry.jl and put everything on the shared filesystem, you set the package git repo’s upstream remote location to a file://... URL that will then be “hard-coded” into the registry when you register the package. Every machine that wants to install this package will get the same URL and it needs to point to the same folder. The same holds true for the registry itself.
So if all users can find the package under "S:\\Julia\\Packages\\KM620" then the URL file:///s:/Julia/Packages/KM620 should resolve fine. Or if you have a host named server-name in the local network, you could write file://server-name/Julia/Packages/... etc.
But if for example user A has the package under S:\\Julia\\Packages\\KM620 and user B under T:\\Julia\\Packages\\KM620 it will not work.
* Shout out to @GunnarFarneback for creating this package and also the nice examples in the docs!
Doesn’t that make this an issue again?
Where would I develop the package? Can you git push to folders?
Is there a way to still keep a copy on GitHub as a backup (even if the master lives on the file server)?