# Dropping Github history from a published package

**URL:** <https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172>\
**Category:** Package Management\
**Tags:** question\
**Created:** [November 27, 2021, 3:47pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172 "2021-11-27T15:47:00Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![goedman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goedman/32/217_2.png) [@goedman](https://discourse.julialang.org/u/goedman)\
**Post date:** [November 27, 2021, 3:47pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/1 "2021-11-27T15:47:00Z")

</div>

Someone kindly [pointed out to me](https://github.com/StatisticalRethinkingJulia/StatisticalRethinking.jl/issues/145) that my package [StatisticalRethinking.jl](https://github.com/StatisticalRethinkingJulia/StatisticalRethinking.jl) is severely bloated. It contains more than 1Gb of garbage in the .git/objects/pack directory (multiple copies of a few pretty old iJulia notebook .ipynb files I used in v1, many other v1 notebooks are no longer present).

I have tried to remove those files and rewrite the Github history (basically following many of the recipes available online to do this), to no avail. The last resort would be to delete the repo, create a new one and make the current content of the repo the initial commit (using the new .git subdirectory).

Before doing that I wanted to check if there are any other suggestions. Losing the history is not a major concern “content wise” but I’m not sure about the JuliaHub tools being able to continue with the next version (v4.4.2). I don’t think there are other published packages that depend on StatisticalRethinking.jl except a few StatisticalRethinking Github org Julia projects.

---

<div class="post-metadata">

**Author:** ![PetrKryslUCSD](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/petrkryslucsd/32/215825_2.png) [@PetrKryslUCSD](https://discourse.julialang.org/u/PetrKryslUCSD)\
**Post date:** [November 27, 2021, 5:32pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/2 "2021-11-27T17:32:35Z")

</div>

Refer to [Git - Maintenance and Data Recovery](https://git-scm.com/book/en/v2/Git-Internals-Maintenance-and-Data-Recovery) (section Removing Objects). That might be of some help.

---

<div class="post-metadata">

**Author:** ![giordano](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/giordano/32/2166_2.png) [@giordano](https://discourse.julialang.org/u/giordano)\
**Post date:** [November 27, 2021, 10:29pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/3 "2021-11-27T22:29:46Z")

</div>

Note that if you rewrite the history by changing the content of release revisions, which would affect their git-tree-sha1, you’d be making your own package uninstallable (if that’s registered)

---

<div class="post-metadata">

**Author:** ![goedman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goedman/32/217_2.png) [@goedman](https://discourse.julialang.org/u/goedman)\
**Post date:** [November 27, 2021, 10:56pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/4 "2021-11-27T22:56:39Z")

</div>

Thanks Petr and Mosè!

Mosè:

Have you seen an example of someone merging a history-free new version of a package? Or is there simply no remedy for this?` `

Petr:

I’d attempted the Git book’s approach but after the rewriting of the git history the file was corrupted. That is when I went online and tried SO suggestions (which are all variations on what the book suggested).

I think it is the rewrite step which does complete after several minutes that corrupts the file (it already warns about bugginess):

```julia
rob@Rob-16-MBP-2 StatisticalRethinking % git filter-branch -f --index-filter \
  'git rm --ignore-unmatch --cached notebooks/03/clip-02-05.ipynb' -- e0ec1390^..   
WARNING: git-filter-branch has a glut of gotchas generating mangled history
	 rewrites. Hit Ctrl-C before proceeding to abort, then use an
	 alternative filtering tool such as 'git filter-repo'
	 (https://github.com/newren/git-filter-repo/) instead. See the
	 filter-branch manual page for more details; to squelch this warning,
	 set FILTER_BRANCH_SQUELCH_WARNING=1.
Proceeding with filter-branch...

```

---

<div class="post-metadata">

**Author:** ![goedman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goedman/32/217_2.png) [@goedman](https://discourse.julialang.org/u/goedman)\
**Post date:** [November 28, 2021, 3:19am UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/5 "2021-11-28T03:19:34Z")

</div>

@giordano @dilumaluthge

Maybe the simplest solution is to move the contents of StatisticalRethinking.jl to a new package StatisticalRethinkingBase.jl (as since StatisticalRethinking.jl v2 the package is only intended to support a number of Julia projects).

Over time StatisticalRethinking.jl will become a general intro for the other packages in the StatisticalRethinking Github organization.

---

<div class="post-metadata">

**Author:** ![John\_Gibson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/john_gibson/32/5321_2.png) [@John\_Gibson](https://discourse.julialang.org/u/John_Gibson)\
**Post date:** [November 28, 2021, 3:29am UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/6 "2021-11-28T03:29:37Z")

</div>

I remember that DIfferentialEquations.jl went through something similar years ago due to large PDFs in documentation, or something like that. Maybe @ChrisRackauckas could advise on the best approach.

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 28, 2021, 7:47am UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/7 "2021-11-28T07:47:45Z")

</div>

The best approach is to just never make a repo have that problem 😅. However, note that these days the repo is not downloaded by users, it only sends the release version, and so you don’t really need to worry about the repo’s history for the package usage.

If you want to fix the history for other reasons, well you could do a BFG repo clean and do a PR to General fixing all of the SHAs

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [November 28, 2021, 8:01am UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/8 "2021-11-28T08:01:18Z")

</div>

> [@ChrisRackauckas](#):
>
> do a PR to General fixing all of the SHAs

We would not accept that PR. We do not accept PRs that change the tree hashes.

---

<div class="post-metadata">

**Author:** ![dilumaluthge](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/dilumaluthge/32/29283_2.png) [@dilumaluthge](https://discourse.julialang.org/u/dilumaluthge)\
**Post date:** [November 28, 2021, 8:04am UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/9 "2021-11-28T08:04:26Z")

</div>

Probably the easiest thing for you to do here is to make a new package named `StatisticalRethinking2.jl` that contains only the files you want.

Then, add a deprecation notice to `StatisticalRethinking.jl` and archive the repository.

It’s not uncommon in the Julia ecosystem for packages to be deprecated and archived in favor of a new package.

---

<div class="post-metadata">

**Author:** ![kristoffer.carlsson](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/kristoffer.carlsson/32/22_2.png) [@kristoffer.carlsson](https://discourse.julialang.org/u/kristoffer.carlsson)\
**Post date:** [November 28, 2021, 10:00am UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/10 "2021-11-28T10:00:53Z")

</div>

> [@giordano](#):
>
> Note that if you rewrite the history by changing the content of release revisions,

This doesn’t really make sense, you cannot (by definition) change the content of a release (because the release is addressed by the content itself). As long as the content exists somewhere in the repo you are fine. Keeping the content available is as easy as e.g. having a tag for each released version. At that point you can do whatever you want with the history of the master branch.

---

<div class="post-metadata">

**Author:** ![cormullion](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/cormullion/32/49131_2.png) [@cormullion](https://discourse.julialang.org/u/cormullion)\
**Post date:** [November 28, 2021, 11:39am UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/11 "2021-11-28T11:39:45Z")

</div>

Wouldn’t your repo lose all its stars though?

(asking for a friend… 🙂 )

---

<div class="post-metadata">

**Author:** ![goedman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goedman/32/217_2.png) [@goedman](https://discourse.julialang.org/u/goedman)\
**Post date:** [November 28, 2021, 2:08pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/12 "2021-11-28T14:08:24Z")

</div>

Thanks everybody for the very helpful comments. I’ll do what I indicated above (and similar to what @dilumaluthge suggests).

StatisticalRethinking.jl will remain around as an anchoring point and overall README. This will preserve the stars (@cormullion).

As I did with Stan.jl for the StanJulia Github organization, it will have no additional functions (these will all go to StatisticalRethinkingBase.jl), just GitHub organization type docs and StatisticalRethinking.jl will continue to be used for overall testing (e.g. functionality comparisons between Stan and Turing or showing new options such as the recently released ParetoSmooth,jl, AxisKeys.jl and DimensionalData.jl packages).

As @ChrisRackauckas pointed out (and I didn’t know), the size issue is only a problem if the package is dev-ed.

Thanks again all!

---

<div class="post-metadata">

**Author:** ![ChrisRackauckas](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/chrisrackauckas/32/77_2.png) [@ChrisRackauckas](https://discourse.julialang.org/u/ChrisRackauckas)\
**Post date:** [November 28, 2021, 2:45pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/13 "2021-11-28T14:45:33Z")

</div>

> [@dilumaluthge](#):
>
> We would not accept that PR. We do not accept PRs that change the tree hashes.

That’s probably a good policy 😅 .

---

<div class="post-metadata">

**Author:** ![lmiq](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/lmiq/32/18314_2.png) [@lmiq](https://discourse.julialang.org/u/lmiq)\
**Post date:** [November 28, 2021, 3:20pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/14 "2021-11-28T15:20:13Z")

</div>

Is there in `dev` something equivalent to the `depth` git option, to avoid downloading the full history?

---

<div class="post-metadata">

**Author:** ![goedman](https://sea2.discourse-cdn.com/julialang/user_avatar/discourse.julialang.org/goedman/32/217_2.png) [@goedman](https://discourse.julialang.org/u/goedman)\
**Post date:** [November 28, 2021, 7:35pm UTC](https://discourse.julialang.org/t/dropping-github-history-from-a-published-package/72172/15 "2021-11-28T19:35:12Z")

</div>

I believe if you do

```julia
git clone --depth 10 https://github.com/StatisticalRethinkingJulia/StatisticalRethinking.jl

```

Julia does not recognize the package. I think that matches what Kristoffer said above?