I learn a lot after some advice from thoughtbot.

Update 2019-11-19: slides for a presentation on this performance mystery

Update 2019-11-29: Both HTML traces are available on Google Drive: 1, 2

Dumpster-diving?

Thoughtbot apparently started a new series on finding tricks in folks’ Dotfiles. As you may know by now, I maintain my own over on GitHub. I peruse others for fun and often learn a lot.

Well, in the course of adjusting their git-branches alias, I found a new trick.

g for git

I have long had alias g='git' in my bash files, along with the appropriate completion definition. But I saw that thoughtbot had a g function which does a git-status if no arguments are provided! This is genius, says I, and I convert.

Aside: In doing so, I forgot to unalias g prior to reload my files. That caused the function definition g() { ... } to expand to git() { ... }—worse, since that definition calls git, I had made every invocation of git (including that of my PS1 prompt) a recursive stack overflow.

Lesson: prefer functions if the complexity can grow.

More performance problems

Anyways, as you may remember from earlier, my Dotfiles repo is big. At time of writing:

λ g cstat count
1707
λ du -hd1 .git
 16K    .git/gitweb
1,4M    .git/objects
  0B    .git/rr-cache
8,0K    .git/info
 84K    .git/logs
 24K    .git/hooks
 12K    .git/refs
 40M    .git/modules
 42M    .git

So the sucker is, well, large (and note how big .git/modules is—we’ll be coming back to that).

Testing out my new g function, I found that git status -sb was taking half a second to complete, where on other repos it ran 10-100x faster. What gives?

Naïvely, I tried git gc (even with --aggressive, which git admits you probably don’t need). This shaved off 6MB to get the numbers you see now. But the performance didn’t waver.

Thinking back to those numbers (and a trace of git code I’d accidentally done recently, wherein there were calls to submodule code), I remembered this little gem:

λ g submodule | wc -l
      47

A quick experiment revealed the issue. After issuing git submodule deinit --all, speeds were back to normal. Having a lot of vim plugins is catching up to me…

Long-story short, turning off submodule stuff in the status brings me back down to the right time. You can do this with a flag on status, but I ended up doing

λ g config diff.ignoresubmodules all

So only my Dotfiles, where the submodules are a problem, ignores them.

Lesson: if your repo has a lot of submodules, you can speed up some operations with this local configuration. But be sure to check git submodule summary every now and then, just to be sure. In my workflow, I almost always only make changes by updating the submodules from upstream, so this can be less of an issue.

Categories: Blog