'.git' folder is bloated with 1 Go of data !

[samusz]

My branch is up to date
'

julia du -h StatisticalRethinking.jl
28K StatisticalRethinking.jl/.git/hooks
0 StatisticalRethinking.jl/.git/info
0 StatisticalRethinking.jl/.git/logs/refs/heads
0 StatisticalRethinking.jl/.git/logs/refs/remotes/origin
0 StatisticalRethinking.jl/.git/logs/refs/remotes
0 StatisticalRethinking.jl/.git/logs/refs
0 StatisticalRethinking.jl/.git/logs
0 StatisticalRethinking.jl/.git/objects/info
1.2G StatisticalRethinking.jl/.git/objects/pack
1.2G StatisticalRethinking.jl/.git/objects
0 StatisticalRethinking.jl/.git/refs/heads
0 StatisticalRethinking.jl/.git/refs/remotes/origin
0 StatisticalRethinking.jl/.git/refs/remotes
0 StatisticalRethinking.jl/.git/refs/tags
0 StatisticalRethinking.jl/.git/refs
1.2G StatisticalRethinking.jl/.git
0 StatisticalRethinking.jl/chapters/00
16K StatisticalRethinking.jl/chapters/02
16K StatisticalRethinking.jl/chapters/03
44K StatisticalRethinking.jl/chapters/04
8.0K StatisticalRethinking.jl/chapters/05
12K StatisticalRethinking.jl/chapters/09
96K StatisticalRethinking.jl/chapters
1.8M StatisticalRethinking.jl/data
16K StatisticalRethinking.jl/docs/src
24K StatisticalRethinking.jl/docs
140K StatisticalRethinking.jl/notebooks/00
1.5M StatisticalRethinking.jl/notebooks/02
3.1M StatisticalRethinking.jl/notebooks/03
12M StatisticalRethinking.jl/notebooks/04
960K StatisticalRethinking.jl/notebooks/05
128K StatisticalRethinking.jl/notebooks/09
18M StatisticalRethinking.jl/notebooks
0 StatisticalRethinking.jl/scripts/00
16K StatisticalRethinking.jl/scripts/02
20K StatisticalRethinking.jl/scripts/03
96K StatisticalRethinking.jl/scripts/04
8.0K StatisticalRethinking.jl/scripts/05
16K StatisticalRethinking.jl/scripts/09
156K StatisticalRethinking.jl/scripts
0 StatisticalRethinking.jl/src/quaps
16K StatisticalRethinking.jl/src
16K StatisticalRethinking.jl/test
1.2G StatisticalRethinking.jl /0.7s
➜ julia cd StatisticalRethinking.jl/'

So for about 4 Mo of data the .git is 1 Go
Some branches needs squashing ?

[goedman]

Thanks for pointing this out and I see that as well. What I don't know is how to safely fix it.

Maybe git rebase --autosquash master?

[goedman]

The bloating is due to a bunch of (pretty) old .ipynb files which I used in version 1. Why specifically 8 or 9 versions of one notebook and one or 2 other notebooks are still present is unclear to me.

I've tried in many ways to remove these files and/or shorten the history to e.g. only contain v4+ related files, but to no avail.

Not sure what other options are available. At least one suggestion I came across is deleting the Github repo, create a new repo with the same name and commit the contents of the current master.

[samusz]

I am not git nor gihub expert so can't really give safe advices. If I understand well what I've read, git squash, deleting branches or rebase could be a solution.

• https://www.reddit.com/r/git/comments/215h58/how_to_shrink_large_repository/ but very old,
• git magic book, section on rewritting history

HTH

[goedman]

Thanks for your suggestions!

It might be difficult to go the route of rewriting history as Mosè pointed out. But nevertheless I'm glad you raised the issue! I pretty much have the package all the time in my dev directory so I hardly ever download it.

Anyway, what I think I'm going to do is to move the contents of StatisticalRethinking.jl to a new directory probably called StatisticalRethinkingBase.jl (or SRBase.jl) and use the current version more as a general intro to the StatisticalRethinking GitHub organization.

[goedman]

After asking for some feedback on Julia discourse I'll implement the above solution.

The large size issue only occurs if you dev the package, it does not happen if you add the package.

As soon as StatisticalRethinkingBase.jl is published I'll let you know and close this issue.

Thanks again for your help!

[goedman]

Hi @samusz ,

Given that I now have a better understanding of the (somewhat limited, only on initial dev-ing) impact of this issue I might hold off a little longer with creating StatisticalRethinkingBase.jl. Such a change will have consequences for many of the notebooks in the Turing and Stan Julia projects that use StatisticalRethinking.jl. But if it really becomes/remains a major issue for you or others, please ping (@goedman) me. For that reason (and in case others run into it), we should probably leave this issue open.