Upload new package to github

ufechner7 · March 9, 2021, 4:02am

I created a new package locally using PkgTemplates.

It looks like this:

ufechner@TUD277255:~/repos/Rubi$ tree -L 1
.
├── deps
├── doc
├── input
├── LICENSE
├── Manifest.toml
├── Project.toml
├── README.md
├── src
└── test

Now I want to upload it to github. Do I have to rename the folder name to Rubi.jl first?

And if so, why does PkgTempletes not allow to create it with this ending in the first place?

affans · March 9, 2021, 4:19am

The name of the folder on your local machine does not have to match your github repository. You can indeed call your online github repository Rubi.jl. There are two ways to develop a package on github:

Create an empty repository on github first. You may call this Rubi.jl. Clone this empty repository in your local workstation and run PkgTemplates (or copy paste that structure it already created). Then use git push to push everything to the remote repo.
Create the package first using PkgTemplates like you did and then create your repository on github. However, to make an existing folder into a git-tracked folder, there are a few steps you have to take, which are all a simple Google search away. The general gist is that you first initialize the folder as a git repository (i.e. git init) and then “connect” this local folder to your online github repository by running adding the fetch/pull remote links.

I generally prefer the first way as it tends to be quite easier. If you are a git beginner, know that the learning slope is quite steep.

ufechner7 · March 9, 2021, 5:06am

Well, I am using github for 8 years and created probably 100 repositories so far, but this is the first time that I need a repository that has a different name than the local folder.

Anyway, what you explained is neither documented in the Julia documentation nor in the Pkg documentation nor in the PkgTemplates documentation.

So I consider this to be at least documentation bug. I created an issue: Documentation how to push a package to github is missing · Issue #2427 · JuliaLang/Pkg.jl · GitHub

But thank you for your explanations, it works for me now.

affans · March 9, 2021, 5:44am

I suppose you don’t have to quote me on that, but as long as the folder is initialized for git, the name of the folder is irrelevant. I’ll google it just to be sure.

tbeason · March 9, 2021, 6:03am

That’s correct. Folder can be called whatever you want. The default is just for them to be the same.

sylvaticus · March 9, 2021, 6:53am

You may find intetesting this tutorial related to package creation, where the first method of @affans is exploited.

In general the official documentation doesn’t include all the github, creating package documentation, continuous integration, and all the other essential bits of info you need to create a package.
I suppose it is intentional, as strictly speacking, it doesn’t refer to the language itself, but I wonder if there should be an “official” documentation for all these bits of info that remain essential to build, and maitain, a Julia package.

ufechner7 · March 9, 2021, 8:20am

The tutorial you linked is indeed very useful and detailed.

lmiq · June 8, 2021, 12:45am

I keep struggling to find the answer to this each time. Now, inspired by the above answers, I arrived to these minimal steps:

Create the repository on github. For example https://github.com/lmiq/MyPackage.jl. When creating it, add one file, for example the README.md (the content will be discarded).
Create a package locally with PkgTemplates, using:

using PkgTemplates
tpl = Template(user="lmiq")
tpl("MyPackage")

This will create the .julia/dev/MyPackage directory with the content inside.

Clone the github repository somewhere else:

cd ~/Downloads
git clone https://github.com/lmiq/MyPackage.jl

Remove the .git directory from the .julia/dev/MyPackage folder and replace it with the .git from Downloads/MyPackage.jl:

rm -rf ~/.julia/dev/MyPackage/.git
cp -r ~/Downloads/MyPackage/.git ~/.julia/dev/MyPackage

Push the package to the repository:

cd ~/.julia/dev/MyPackage
git add -A
git commit -m "first commit"
git push

You can safely delete the ~/Downloads/MyPackage.jl directory.

ptoche · June 8, 2021, 1:55am

I too create the repo on github first and pull it to my local drive into an empty directory with the name I want (if my package is called MyUtils.jl my directory is usually more descriptive like home-github-myutils or work-github-myutils). I copy a LICENCE file, Manifest.toml, Project.toml from some previous package I have. I erase most of the content: Inside the Manifest.toml file I keep only the first line (# This file is machine-generated - editing it directly is not advised), then I edit the Project.toml with the basic information and, the crucial step, change the UUID with:

import UUIDs
UUIDs.uuid4()

For some reason I find this quicker than using a template creator. But YMMV of course.

And I usually use the Github app so it’s all pointy-clicky and no terminal commands, unless I have to.

tbeason · June 8, 2021, 2:37am

Those seem somewhat convoluted. Create the repo on Github but leave it EMPTY. No readme, no license. Use whatever method you like to locally create your package (shoutout to @Tamas_Papp for my preferred method GitHub - tpapp/PkgSkeleton.jl: Generate Julia package skeletons using a simple template system). Then just point your local folder to the online repo and push.

git commit -a -m "of course you have to have a first commit"
git remote add origin <repo link html or ssh>
git push -u origin <branch> #likely main or master

lmiq · June 8, 2021, 8:40am

What does this mean exactly?

I tried things like that, but always ended with some error (repo exists, not authorized…)

I really was expecting that something like you mention would come out.

For example:

Created the github repo:

Created the local package:

julia> using PkgTemplates

julia> tpl = Template(user="lmiq");

julia> tpl("MyPackage")
[ Info: Running prehooks
[ Info: Running hooks
  Activating environment at `~/.julia/dev/MyPackage/Project.toml`
    Updating registry at `~/.julia/registries/General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
  No Changes to `~/.julia/dev/MyPackage/Project.toml`
  No Changes to `~/.julia/dev/MyPackage/Manifest.toml`
Precompiling project...
  1 dependency successfully precompiled in 2 seconds
  Activating environment at `~/.julia/environments/v1.6/Project.toml`
[ Info: Running posthooks
[ Info: New package is at /home/leandro/.julia/dev/MyPackage

Now I try to commit to the github repo:

~/.julia/dev/MyPackage% git commit -a -m "first commit"
On branch master
nothing to commit, working tree clean
~/.julia/dev/MyPackage% git remote add origin https://github.com/lmiq/MyPackage
fatal: remote origin already exists.

I never figured out how to pass this, except by removing the .git directory with another one from the repo, as I mentioned above.

GunnarFarneback · June 8, 2021, 11:51am

After doing the same steps as you in Julia and going to the package directory I get:

$ git log
commit d81fcbc9b7afc5c94ca5ffe6d482ec56ccd8d820 (HEAD -> master)
Author: Gunnar Farnebäck <x@y>
Date:   Tue Jun 8 13:32:12 2021 +0200

    Files generated by PkgTemplates
    
    PkgTemplates version: 0.7.16

commit 06a3076c99646cd285ced70af94aa08ea615a03b
Author: Gunnar Farnebäck <x@y>
Date:   Tue Jun 8 13:32:07 2021 +0200

    Initial commit

The commits are already there.

$ git remote
origin

PkgTemplates has set up a remote called origin.

$ git remote get-url origin
https://github.com/lmiq/MyPackage.jl

This doesn’t exactly match the Github repository you created. Either rename your repository or change the remote:

$ git remote set-url origin https://github.com/lmiq/MyPackage.git

Now try to push:

$ git push
fatal: The current branch master has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin master

Still need to tell what branch to push.

$ git push --set-upstream origin master
Username for 'https://github.com':

Well, that’s as far as I can go as it’s not my repository.

Skoffer · June 8, 2021, 12:03pm

Why would you want to add a new origin if existing is perfectly fine?

Anyway, git has it’s own means to resolve almost any git related issues. For example, you can always check which remotes you have now with git remote -v command

➜  git remote -v
origin  git@github.com:Arkoniak/Telegram.jl.git (fetch)
origin  git@github.com:Arkoniak/Telegram.jl.git (push)

and you can change existing remote to anything you like with git remote set-url command

➜   git remote set-url origin git@github.com:Arkoniak/MyPkg.jl.git
➜   git remote -v
origin  git@github.com:Arkoniak/MyPkg.jl.git (fetch)
origin  git@github.com:Arkoniak/MyPkg.jl.git (push)

And you can always drop unnecessary remotes with git remote remove command

➜   git remote remove origin
➜   git remote -v # empty output

So, if you are using PkgTemplates.jl all you have to do is create package, cd to the corresponding directory and do git push origin master (or whatever is default name of the main branch nowadays).

You do not even need to add/commit, as far as I remember, since PkgTemplates.jl do initial commit by itself.

lmiq · June 8, 2021, 12:08pm

@GunnarFarneback , thanks. I see. I think I was always hitting a mixture of two things: 1) Getting confused on what is the “package name” and what is the “repository name”, and not knowing what was the “repository name” that PkgTemplates was creating (didn’t know about the git remote get-url origin command).

Conclusion:

The simplest path:

Create the empty github repo, i. e. http://github.com/lmiq/MyPackage.jl (create it with .jl at the end).
Use package templates with:

using PkgTemplates
tpl = Template(user="lmiq")
tpl("MyPackage")

Navigate to ~/.julia/dev/MyPackage, and do:

git push --set-upstream origin master

Thanks for the tips. I don’t really want anything… I just don’t understand what is going on.

ufechner7 · June 8, 2021, 12:09pm

This is not going to work because PkgTemplates.jl creates names without .jl at the end, but for github you need it.

Skoffer · June 8, 2021, 12:09pm

Not quite get it, since it is a working setup. Can you elaborate please? What exactly is not going to work?

ufechner7 · June 8, 2021, 12:12pm

The command

git push --set-upstream origin master

will fail.

Skoffer · June 8, 2021, 12:16pm

Sorry, I still do not understand. I’ve used hundreds of time and it never failed. I guess there is some misunderstanding here.

Ah, it seems you wanted to answer @GunnarFarneback reply.

lmiq · June 8, 2021, 12:21pm

Actually that was the confusing part. PkgTemplates creates a directory .julia/dev/MyPackage without the .jl, but the git repository that is associated there does contain the .jl, such that the remote git repository must also have the .jl. If the remote github repo has .jl that command will work.

tbeason · June 8, 2021, 5:27pm

Ah I did not know that PkgTemplates.jl creates the github repo for you. So that makes it even easier! Should be no troubles. My method above will work just fine if you use Pkg or PkgSkeleton.jl or some other method that does not create the github repo for you. If it is already created of course you don’t need another!