Saturday, March 28, 2015

A super simple Introduction to version control with Git for busy Astrophysicists

Welcome, collaborator! You may be reading this, because you, I, and others are working on some kind of joint document. It could be a manuscript to be submitted to a journal, some kind of vision document, or (as is often the case) a proposal.

Sending around LaTeX files via email is so 1990s. Let's not do this. Let's also not use Dropbox. Dropbox is great for syncing files between computers and serves as a great backup solution. But it's terrible for collaborating on text documents -- it can't handle conflicts (two people editing the same file at the same time creates a second copy of the file in the dropbox; Dropbox won't merge text for you).

Let's move on and use version control.  It will make working together on the same document infinitely easier and will minimize the time we have to deal with merging different versions of the file we are working on together. There are multiple version control packages. For our project we will use git. You can learn more about git and version control in general on If you are unconvinced, read this: Why use a Version Control System?

Here is how it works:

(1) Preparations
  1. Make sure you have git installed on your computer (I presume you have either a Mac or a PC that runs a flavor of Linux). Just open a terminal, type git, hit enter, and see if the command is found. If not, you need to install git.

  2. Make and send me a password-protected public ssh key as an email attachment (not cut & paste!). The standard path and file name for this are ~/.ssh/ or ~/.ssh/ If you don't have such a key or you don't remember if it's password protected (or you dont remember your password...), type on the command line:
    ssh-keygen -t rsa
    and follow the instructions. By all means, give your key a strong password. Then send me the file ~/.ssh/ as an email attachment (do not cut & paste it into the email body) 

(2) Cloning the repository from the server
By now I have told you details about the repository and sent you the command needed to clone it from the server. Go into your preferred working directory (that's where you keep projects, papers, proposals etc.). Type:
This will clone the repo -- in other words, it will create a directory with the repository name in your working directory and will download the contents of the repo into it. If this does not happen and instead you are asked to enter a password, then your ssh key is not loaded. In this case, load your ssh key like this:
ssh-add ~/.ssh/id_rsa
Then try again. If this still does not do the trick or if you get an error message about no "ssh-agent" running, then try
This will spit out a few lines of text. Copy them and paste them back into the prompt, hit enter, then repeat
ssh-add ~/.ssh/id_rsa
This should do the trick!

(3) Pulling in changes.
Before starting to work on a section of the document, you need to make sure to get the most recent changes. You must absolutely always do this before starting to work! Go into the repository directory.
git pull -r 
(the "-r" is to make the pull a "rebase" -- don't worry about the meaning of this, just do it).  

(4) Committing and pushing changes.
Whenever you have completed a significant addition or change to the document -- this may be a completed or edited paragraph, something that takes no longer than about 30 minutes of work, you need to commit and push your changes so that others can pull them onto their computers.
git commit -m "Your commit message goes here"
git pull -r           # this is to pull in any changes by others!
git push
Please give sensible commit messages so that we others know what you changed!

(5) Looking at the history of changes and checking the status of your repo.
That's very easy! First pull in all recent changes via git pull -r. Then use git log to browse through the repository history. Try git status to see what the status of your repo is. This will tell you which files have changed and should be committed. It will also tell you which files you have not yet added (i.e. "untracked" files).

(6) Golden Rules
Working with version controlled documents is easy and painless as long as you obey two simple rules:
  1. Always git pull -r before starting to edit.
  2.  Extremely frequently:
    git add [FILE]
    git commit -m "[COMMIT MESSAGE]"
    git pull -r
    git push
If you stick to these rules, working collaboratively on the same document will be easy and conflicts will be rare.

(7) Adding additional files to the repo
That's easy!
git add [FILENAME]
git commit -m "Added file [FILENAME] to repo"
git pull -r
git push
(8) Resolving merge conflicts
Oh boy. Somebody was editing the same portion of text you were editing (did you perhaps not sufficiently frequently commit and push?!?). You now need to resolve the conflict. Let's do this by example. Let's say you were editing a file called You added, committed, then pulled and ended up with the following git merge error:
remote: Counting objects: 5, done.
remote: Total 3 (delta 0), reused 3 (delta 0) Unpacking objects: 100% (3/3), done.
23ae277..b13d53f master -> origin/master Auto-merging
CONFLICT (content): Merge conflict in
Automatic merge failed; fix conflicts and then commit the result.
Good luck! You now need to resolve the conflict. Let's look at the (hypothetical) file in question.
$ cat 
<<<<<<< HEAD 
>>>>>>> b13d53f9cf6a8fe5b58e3c9c103f1dab84026161
git has conveniently inserted information on what the conflict is. The stuff right beneath <<<<<< HEAD is what is currently in the remote repository and the stuff just below the ======= is what we have locally. Let’s say you know your local change is correct. Then you edit the file and remove the remote stuff and all conflict stuff around it.
$ cat 
Next you git add and git commit the file, then you execute git push to update the remote with your local change. This resolves the conflict and all is good!