Package publishing problems (failed)

New to Julia
1

I’d like to report back on my recent (last night) experience with package publishing - ultimately the happy path failed.

1/ per the docs I’ve used the hub binary – hub create is a great way of setting up the remote repo. However, I suspect hub has a problem handling repos ending up with a .jl prefix. Notice that in the output, the repo is created without the .jl postfix:

$ hub create
github.com username: essenciary
github.com password for essenciary (never stored):
origin	https://github.com/essenciary/IssueReporter.jl.git (fetch)
origin	https://github.com/essenciary/IssueReporter.jl.git (push)
created repository: essenciary/IssueReporter

Obviously, pushing to it will fail:

$ git push origin master
remote: Repository not found.
fatal: repository 'https://github.com/essenciary/IssueReporter.jl.git/' not found

Thus updating the origin was necessary:

$ git remote set-url origin https://github.com/essenciary/IssueReporter.git

2/ PkgDev.register worked, but PkgDev.publish did not:

julia> PkgDev.register("IssueReporter")
INFO: Registering IssueReporter at https://github.com/essenciary/IssueReporter.git
INFO: Committing METADATA for IssueReporter

julia> PkgDev.publish()
ERROR: There are no METADATA changes to publish
Stacktrace:
 [1] (::PkgDev.Entry.##4#9{String,Dict{String,Array{String,1}},String})(::Base.LibGit2.GitRepo) at /Users/adrian/.julia/v0.6/PkgDev/src/entry.jl:72
 [2] with(::PkgDev.Entry.##4#9{String,Dict{String,Array{String,1}},String}, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:608
 [3] publish(::String, ::String) at /Users/adrian/.julia/v0.6/PkgDev/src/entry.jl:64
 [4] publish() at /Users/adrian/.julia/v0.6/PkgDev/src/PkgDev.jl:u7533:

3/ PkgDev.tag worked, but again, PkgDev.publish did not:

julia> PkgDev.tag("IssueReporter")
INFO: Tagging IssueReporter v0.0.1
INFO: Committing METADATA for IssueReporter

julia> PkgDev.publish()
ERROR: SystemError: opening file /Users/adrian/.julia/v0.6/METADATA/BSON/versions/0.1.0/sha1: No such file or directory
Stacktrace:
 [1] #systemerror#44 at ./error.jl:64 [inlined]
 [2] systemerror(::String, ::Bool) at ./error.jl:64
 [3] open(::String, ::Bool, ::Bool, ::Bool, ::Bool, ::Bool) at ./iostream.jl:104
 [4] open(::Base.#readstring, ::String) at ./iostream.jl:150
 [5] readchomp at ./io.jl:491 [inlined]
 [6] (::PkgDev.Entry.##4#9{String,Dict{String,Array{String,1}},String})(::Base.LibGit2.GitRepo) at /Users/adrian/.julia/v0.6/PkgDev/src/entry.jl:79
 [7] with(::PkgDev.Entry.##4#9{String,Dict{String,Array{String,1}},String}, ::Base.LibGit2.GitRepo) at ./libgit2/types.jl:608
 [8] publish(::String, ::String) at /Users/adrian/.julia/v0.6/PkgDev/src/entry.jl:64
 [9] publish() at /Users/adrian/.julia/v0.6/PkgDev/src/PkgDev.jl:75

So unless there’s an easy fix, I presume I’m stuck with trying the manual approach.

Here’s some debug info:

julia> MetadataTools.get_pkg("IssueReporter")
IssueReporter   https://github.com/essenciary/IssueReporter.git
  0.0.1,f37dce,julia 0.6,GitHub,MetadataTools,URIParser

julia> versioninfo()
Julia Version 0.6.2
Commit d386e40c17 (2017-12-13 18:08 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin14.5.0)
  CPU: Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, skylake)
2

I would suggest using AttoBot https://github.com/attobot/attobot. It makes registering and tagging packages much easier. Just install it to your repo and tag a release. Done.

3 Likes
3

Thanks, Kristoffer, I’ll give it a try and report back on the progress. It would be great to identify and officially recommend a better way. Unfortunately every publishing step, following the official doc, resulted in an error so far.

4

Attobot seems to have worked. I’ve also opened an issue with hub.

5

I think the doc is a bit outdated, I made a PR about it a while ago:

https://github.com/JuliaLang/julia/pull/25163

6

The master docs recommend attobot.

7

@Tamas_Papp ah, ok, I’ve seen that. I guess it depends on how each person understands it. I didn’t read that as the official way. It’s just a tip and as I have limited experience with GitHub bot integrations, I decided to skip it. Also I prefer not to do things via the GitHub website but rather stay in the Julia REPL. If that’s the best practice as of recently, I’d expect the documentation to move away the parts about PkgDev.register, PkgDev.publish, and PkgDev.tag and elaborate on the Attobot approach. If that is the case, I’m happy to do it.

Anyway, it’s good to know that Attobot works - the whole thing is part of a write-up on creating and publishing Julia packages and I was quite disappointed by the failure. I was reluctant to go into manual git wrangling as it would’ve been outside the scope of the chapter, which is targeting beginners. I’m adding in a section on
Attobot now. In the end, it’s all for the better as it’s a more exciting process. :slight_smile:

@jonathanBieler I also tried tagging and then publishing again, but that failed as well. Not sure if the previous call to PkgDev.publish broke it, but the tagging seems to have worked, per my output.

8

imho: while attobot is quite nice, the REPL way should work as described. Not everything is hosted on github.

1 Like
9

Good point

10

Note that it is not very difficult to just do what PkgDev does manually (push .julia/v0.x/METADATA/to local METADATA and create a PR)

Of course, PkgDev should work as well, but like with everything, there is a cost-benefit ratio to consider.

but with limited there is always a effort to gain rat

1 Like
11

Following up on the repo naming issue, after checking with the hub team, I believe that the problem is with PkgDev:

julia> PkgDev.generate("TestPkg", "MIT")
INFO: Initializing TestPkg repo: /Users/adrian/.julia/v0.6/TestPkg      # repo is initialized as "TestPkg"
INFO: Origin: https://github.com/essenciary/TestPkg.jl.git              # origin is set to "TestPkg.jl"
INFO: Generating LICENSE.md
INFO: Generating README.md
INFO: Generating src/TestPkg.jl
INFO: Generating test/runtests.jl
INFO: Generating REQUIRE
INFO: Generating .gitignore
INFO: Generating .travis.yml
INFO: Generating appveyor.yml
INFO: Generating .codecov.yml
INFO: Committing TestPkg generated files

The fact that hub create adds a remote repo called “TestPkg”, just like the name of the local repo, seems consistent to me.

Another issue is that it’s not clear how to address the problem. My initial solution was to update the origin (instead or renaming the repo to add the .jl extension) – but repos which don’t end with a .jl extension break Attobot :face_with_raised_eyebrow:

Suggested fix: PkgDev.generate should consistently append .jl to both the repo name and the origin.

###Update 1
Also tried creating a package which includes the .jl suffix in its name – but PkgDev removes that from the repo’s name, ending up with the same situation:

julia> PkgDev.generate("TestPkgTwo.jl", "MIT")
INFO: Initializing TestPkgTwo repo: /Users/adrian/.julia/v0.6/TestPkgTwo
INFO: Origin: https://github.com/essenciary/TestPkgTwo.jl.git
INFO: Generating LICENSE.md
INFO: Generating README.md
INFO: Generating src/TestPkgTwo.jl
INFO: Generating test/runtests.jl
INFO: Generating REQUIRE
INFO: Generating .gitignore
INFO: Generating .travis.yml
INFO: Generating appveyor.yml
INFO: Generating .codecov.yml
INFO: Committing TestPkgTwo generated files
12

I was under the impression that this is how it should work; you create the repos ending in .jl (so that software is easier to find on the net), and locally this is omitted.

(I always end up changing the location of origin anyway, to use the git protocol instead of https).

13

Well, I don’t know what to make of it. Attobot requires the .jl suffix while PkgDev strips it off.

14

Sometimes when one obsesses too much about a problem, one misses the most obvious solution:
$ hub create TestPkg.jl

All that’s needed is passing the name of the repo with the .jl extension this time. So I’m guessing the only issue with the documentation is that the convention is not spelled clearly:

  • no .jl locally (or if used, will be stripped)
  • do use .jl remotely (required by Attobot & co)