Mark Needham

Thoughts on Software Development

Archive for the ‘git’ tag

Git: Viewing the last commit on all the tags

with 2 comments

A couple of days ago I was curious when different versions of Neo4j had been released and although the release notes page was helpful I thought I’d find more detailed information if I looked up the git tags.

Assuming that we’ve already got a clone of the repository on our machine:

$ git clone git@github.com:neo4j/neo4j.git

We can pull down the latest tags by calling git fetch –tags or git fetch -t

$ git fetch -t
remote: Counting objects: 542, done.
remote: Compressing objects: 100% (231/231), done.
remote: Total 287 (delta 247), reused 84 (delta 50)
Receiving objects: 100% (287/287), 42.85 KiB, done.
Resolving deltas: 100% (247/247), completed with 191 local objects.
From github.com:neo4j/neo4j
 * [new tag]         1.9.2      -> 1.9.2
 * [new tag]         1.9.5      -> 1.9.5
 * [new tag]         2.0.0-M06  -> 2.0.0-M06

We can get a list of all the tags in the repository with the following command:

$ git tag | head -n5
1.3
1.4
1.4.1
1.4.2
1.4.M01

Now let’s have a look which commit that tag points at:

$ git show 1.3
 
tag 1.3
Tagger: Neo4j Build Server <buildserver@neotechnology.com>
Date:   Tue Nov 20 17:03:38 2012 +0000
 
Tagging for release 1.3
 
commit ff16757dd53399eccb8f3db40eb48bab065459b0
Author: Neo Technology buildbox <buildserver@neotechnology.com>
Date:   Tue Apr 12 22:03:33 2011 +0000

That command gets us the appropriate information but ideally we want to get the commit hash and the date on a single line which we can do by passing the ‘–format‘ flag to git log:

$ git log --format="%h %ad%n" 1.3
ff16757 Tue Apr 12 22:03:33 2011 +0000
 
9651aa8 Tue Apr 12 21:58:58 2011 +0000
 
21c637d Tue Apr 12 12:39:49 2011 +0200
 
4ed65eb Tue Apr 12 12:39:28 2011 +0200

We can pipe that to head to get the most recent commit:

$ git log --format="%h %ad%n" 1.3 | head -n1
ff16757 Tue Apr 12 22:03:33 2011 +0000

I tried to pipe the output of git tag to git log using xargs but I couldn’t get it to work so I resorted to a for loop instead:

$ for tag in `git tag`; do printf "%-20s %-100s \n" $tag "`git log --format="%h %ad%n" $tag | head -n1`"; done | head -n5
1.3                  ff16757 Tue Apr 12 22:03:33 2011 +0000                                                               
1.4                  5c19dc3 Fri Jul 8 16:22:37 2011 +0200                                                                
1.4.1                55f4ab2 Tue Aug 2 15:14:11 2011 +0300                                                                
1.4.2                cb85742 Tue Sep 27 18:59:13 2011 +0100                                                               
1.4.M01              f5aacf4 Fri Apr 29 10:12:52 2011 +0200

We could then pipe that output through grep to only show non point releases:

$ for tag in `git tag`; do printf "%-20s %-100s \n" $tag "`git log --format="%h %ad%n" $tag | head -n1`"; done | grep -E "^\d\.\d "
1.3                  ff16757 Tue Apr 12 22:03:33 2011 +0000                                                               
1.4                  5c19dc3 Fri Jul 8 16:22:37 2011 +0200                                                                
1.5                  0225cb7 Thu Oct 20 03:51:06 2011 +0200                                                               
1.6                  f6f3cc1 Sun Jan 22 15:02:04 2012 +0100                                                               
1.7                  cc4ad98 Wed Apr 18 18:32:20 2012 +0200                                                               
1.8                  084acc9 Tue Sep 25 09:47:04 2012 +0100                                                               
1.9                  2efc04c Mon May 20 12:08:24 2013 +0100

I first played around with Neo4j in September 2011 and I now know that I was using version 1.4 at the time.

We’re now at 1.9.5 and the latest beta release is 2.0.0-M06 so there have been quite a few releases in between!

Written by Mark Needham

November 16th, 2013 at 9:58 pm

Posted in Version Control

Tagged with ,

Git: Commit squashing made even easier using ‘git branch –set-upstream’

without comments

A few days ago I wrote a blog post describing how I wanted to squash a series of commits into one bigger one before making a pull request and in the comments Rob Hunter showed me an even easier way to do so.

To recap, by the end of the post I had the following git config:

$ cat .git/config
[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = git@github.com:mneedham/neo4j-shell-tools.git
[branch "master"]
	remote = origin
	merge = refs/heads/master
[remote "base"]
	url = git@github.com:jexp/neo4j-shell-tools.git
	fetch = +refs/heads/*:refs/remotes/base/*
[branch "readme-pull"]
	remote = origin
	merge = refs/heads/readme-pull
[branch "readme"]
	remote = origin
	merge = refs/heads/readme

I was working against the remote ‘origin’ but the actual home of this repository is ‘base’.

I’d created a load of commits on ‘origin/readme’ and had then squashed them all into one commit on ‘origin/readme-pull’ by using the following command:

$ git rebase -i c4e94f668223d53f6c7364d19aa965d09ea7eb00

where ‘c4e94f668223d53f6c7364d19aa965d09ea7eb00’ is the hash of the last commit that was made in ‘base/master’.

Rob suggested that I should try using upstream tracking to simplify this even further. When we use upstream tracking we create a link between a local and remote repository which in this case is useful for working out where our commits start from.

I thought I’d try it out on another branch. We want to set the new branch to track ‘base/master’ since that’s the one we eventually want to have our commit applied against.

We’ll start from the ‘readme’ branch which has the list of commits that we want to squash

$ git branch
  master
* readme
  readme-pull

Now let’s create a new branch and then track it against ‘base/master’:

$ git checkout -b readme-pull-new
Switched to a new branch 'readme-pull-new'
$ git branch --set-upstream readme-pull-new base/master
Branch readme-pull-new set up to track remote branch master from base.

Squashing all our commits is now as simple as running the following command:

$ git rebase -i

And then choosing ‘squash’ against all commits except for the first one which can stay as ‘pick’. We then need to edit the commit message into shape which mostly involves deleting the commit messages from the commits we’ve squashed in this instance.

Thanks to Rob for the tip!

Written by Mark Needham

July 16th, 2013 at 8:13 am

Posted in Version Control

Tagged with

Git/GitHub: Squashing all commits before sending a pull request

with 5 comments

My colleague Michael has been doing some work to make it easier for people to import data into neo4j and his latest attempt is neo4j-shell-tools which adds some additional commands to the neo4j-shell.

I’ve spent a bit of time refactoring the readme which I’d done on a branch of my fork of the repository and consisted of 46 commits, most changing 2 or 3 lines.

I wanted to send Michael a pull request on Github but first I needed to squash all my commits down into a single one.

I initially thought there might be a way that I could do that via Github but I couldn’t see how to do that and eventually came across a post on Steve Klabnik’s blog which explained what I needed to do.

This is what my .git/config looked like initially:

[remote "origin"]
	fetch = +refs/heads/*:refs/remotes/origin/*
	url = git@github.com:mneedham/neo4j-shell-tools.git
[branch "master"]
	remote = origin
	merge = refs/heads/master
[remote "base"]
	url = git@github.com:jexp/neo4j-shell-tools.git
	fetch = +refs/heads/*:refs/remotes/base/*
[branch "readme"]
	remote = origin
	merge = refs/heads/readme

I had all my commits on the ‘readme’ branch but the easiest approach seemed to be to create another branch on which I could squash all my commit – I called that branch ‘readme-pull’:

$ git branch readme-pull
$ git checkout readme-pull
Switched to branch 'readme-pull'

I then synced myself with Michael’s repository:

$ git fetch base
remote: Counting objects: 77, done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 43 (delta 15), reused 40 (delta 12)
Unpacking objects: 100% (43/43), done.
From github.com:jexp/neo4j-shell-tools
   e81c431..c4e94f6  master     -> base/master
 
$ git rebase base/master
First, rewinding head to replay your work on top of it...

I then had to handle any conflicts when applying my changes on top of Michael’s master repository and then I was in a position to squash all my commits!

We can use rebase in interactive mode to do this and I’ve always done so by counting back how many commits I want to squash, so in this case it was 35:

$ git rebase -i HEAD~35
 
pick 141d0ae updating readme with link
pick 94f8f93 more updating
pick 03de50b readme updates
pick 4e60332 more updates
pick 3447d50 simplifying
pick d577520 tweaks
pick 2d993d4 more
pick f948582 list of commands
pick 713aae8 updating

I later realised that I could have just passed in the last commit hash from the master to the rebase command i.e.

commit c4e94f668223d53f6c7364d19aa965d09ea7eb00
Author: Michael Hunger <github@jexp.de>
Date:   Fri Jul 12 10:33:55 2013 +0200
 
    fixed test
$ git rebase -i c4e94f668223d53f6c7364d19aa965d09ea7eb00

I then set all but the first commit to ‘squash‘ and pushed to my repository:

$ git push -u origin readme-pull:readme-pull

Finally I issued my pull request and Michael merged it in!

Written by Mark Needham

July 13th, 2013 at 6:47 pm

Posted in Version Control

Tagged with

git: Having a branch/tag with the same name (error: dst refspec matches more than one.)

with 3 comments

Andres and I recently found ourselves wanting to delete a remote branch which had the same name as a tag and therefore the normal way of doing that wasn’t worked out as well as we’d hoped.

I created a dummy repository to recreate the state we’d got ourselves into:

$ echo "mark" > README
$ git commit -am "readme"
$ echo "for the branch" >> README 
$ git commit -am "for the branch"
 
$ git checkout -b same
Switched to a new branch 'same'
 
$ git push origin same
Counting objects: 5, done.
Writing objects: 100% (3/3), 263 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To ssh://git@bitbucket.org/markhneedham/branch-tag-test.git
 * [new branch]      same -> same
 
$ git checkout master
$ echo "for the tag" >> README
$ git commit -am "for the tag"
$ git tag same
$ git push origin refs/tags/same
Counting objects: 5, done.
Writing objects: 100% (3/3), 266 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To ssh://git@bitbucket.org/markhneedham/branch-tag-test.git
 * [new tag]         same -> same

We wanted to delete the remote ‘same’ branch and the following command would work if we hadn’t created a tag with the same name. Instead it throws an error:

$ git push origin :same
error: dst refspec same matches more than one.
error: failed to push some refs to 'ssh://git@bitbucket.org/markhneedham/branch-tag-test.git'

We learnt that what we needed to do was refer to the full path for the branch when trying to delete it remotely:

$ git push origin :refs/heads/same
To ssh://git@bitbucket.org/markhneedham/branch-tag-test.git
 - [deleted]         same

To delete the tag we could do the same thing:

$ git push origin :refs/tags/same
remote: warning: Deleting a non-existent ref.
To ssh://git@bitbucket.org/markhneedham/branch-tag-test.git
 - [deleted]         same

Of course the tag and branch still exist locally:

$ ls -alh .git/refs/heads/
total 16
drwxr-xr-x  4 markhneedham  wheel   136B 13 Jun 23:09 .
drwxr-xr-x  5 markhneedham  wheel   170B 13 Jun 22:39 ..
-rw-r--r--  1 markhneedham  wheel    41B 13 Jun 23:08 master
-rw-r--r--  1 markhneedham  wheel    41B 13 Jun 23:08 same
 
$ ls -alh .git/refs/tags/
total 8
drwxr-xr-x  3 markhneedham  wheel   102B 13 Jun 23:08 .
drwxr-xr-x  5 markhneedham  wheel   170B 13 Jun 22:39 ..
-rw-r--r--  1 markhneedham  wheel    41B 13 Jun 23:08 same

So we got rid of them as well:

$ git checkout master
Switched to branch 'master'
$ git branch -d same
Deleted branch same (was 08ad88c).
$ git tag -d same
Deleted tag 'same' (was 1187891)

And now they are gone:

$ ls -alh .git/refs/heads/
total 8
drwxr-xr-x  3 markhneedham  wheel   102B 13 Jun 23:16 .
drwxr-xr-x  5 markhneedham  wheel   170B 13 Jun 22:39 ..
-rw-r--r--  1 markhneedham  wheel    41B 13 Jun 23:08 master
$ ls -alh .git/refs/tags/
total 0
drwxr-xr-x  2 markhneedham  wheel    68B 13 Jun 23:16 .
drwxr-xr-x  5 markhneedham  wheel   170B 13 Jun 22:39 ..

Out of interest we’d ended up with this situation by mistake rather than by design but it was still fun to do a little bit of git digging to figure out how to solve the problem we’d created for ourselves.

Written by Mark Needham

June 13th, 2013 at 10:18 pm

Posted in Version Control

Tagged with

git: Only pushing some changes from local repository

with 4 comments

Something that we want to do reasonable frequently on my current project is to push some changes which have been committed to our local repository to master but not all of them.

For example we might end up with 3 changes we haven’t pushed:

>> ~/github/local$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 3 commits.
#
nothing to commit (working directory clean)
>> ~/github/local$ git hist
* bb7b139 Thu, 20 Oct 2011 07:37:11 +0100 | mark: one last time (HEAD, master) [Mark Needham]
* 1cef99a Thu, 20 Oct 2011 07:36:35 +0100 | mark:another new line [Mark Needham]
* 850e105 Thu, 20 Oct 2011 07:36:01 +0100 | mark: new line [Mark Needham]
* 2b25622 Thu, 20 Oct 2011 07:32:43 +0100 | mark: adding file for first time (origin/master) [Mark Needham]

And we only want to push the commit with hash 850e105 for example.

The approach which my colleague Uday showed us is to first take a temporary branch of the current state.

>> ~/github/local$ git checkout -b temp-branch
Switched to a new branch 'temp-branch'

Then immediately switch back to master and ‘get rid’ of the last two changes from there:

>> ~/github/local$ git checkout master
Switched to branch 'master'
Your branch is ahead of 'origin/master' by 3 commits.
>> ~/github/local$ git reset HEAD~2 --hard
HEAD is now at 850e105 mark: new line

We can then push just that change:

>> ~/github/local$ git push
Counting objects: 5, done.
Writing objects: 100% (3/3), 257 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
To /Users/mneedham/github/remote
   2b25622..850e105  master -> master

And merge the temporary branch back in again so we’re back where we were before:

>> ~/github/local$ git merge temp-branch
Updating 850e105..bb7b139
Fast-forward
 foo.txt |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
>> ~/github/local$ git hist
* bb7b139 Thu, 20 Oct 2011 07:37:11 +0100 | mark: one last time (HEAD, temp-branch, master) [Mark Needham]
* 1cef99a Thu, 20 Oct 2011 07:36:35 +0100 | mark:another new line [Mark Needham]
* 850e105 Thu, 20 Oct 2011 07:36:01 +0100 | mark: new line (origin/master) [Mark Needham]
* 2b25622 Thu, 20 Oct 2011 07:32:43 +0100 | mark: adding file for first time [Mark Needham]
>> ~/github/local$ git status
# On branch master
# Your branch is ahead of 'origin/master' by 2 commits.
#
nothing to commit (working directory clean)

And finally we delete the temporary branch:

>> ~/github/local$ git branch -d temp-branch
Deleted branch temp-branch (was bb7b139).

We can achieve the same thing without creating the branch and just cherry picking the commits back again after we’ve pushed our changes but this seems approach seems quicker.

Written by Mark Needham

October 20th, 2011 at 6:50 am

Posted in Version Control

Tagged with

Git: Getting the history of a deleted file

with 5 comments

We recently wanted to get the Git history of a file which we knew existed but had now been deleted so we could find out what had happened to it.

Using a simple git log didn’t work:

git log deletedFile.txt
fatal: ambiguous argument 'deletedFile.txt': unknown revision or path not in the working tree.

We eventually came across Francois Marier’s blog post which points out that you need to use the following command instead:

git log -- deletedFile.txt

I’ve tried reading through the man page but I’m still not entirely sure what the distinction between using and not using it is supposed to be.

If someone could explain it that’d be cool…

Written by Mark Needham

October 4th, 2011 at 10:33 pm

Posted in Version Control

Tagged with

Git: Deleting a remote branch on a gitolite configured repository

with one comment

We’ve had an xsbt branch on our gitolite powered repository for the last couple of weeks while we worked out how to move our build from sbt 0.7 to sbt 0.10 but having finally done that we needed to delete it.

I originally tried running the following command from one of our developer workstations:

git push origin :xsbt

But ended up with the following error:

remote: error: denying ref deletion for regs/head/xsbt

! [remote rejected] xsbt (deletion prohibited)

A bit of googling led me to this stackoverflow thread which suggested that you needed to be an administrator in order to delete a remote branch.

Once we’ve done that we can run the following command on each machine to delete the remote tracking reference to the repository:

git branch -d -r origin/xsbt

Written by Mark Needham

June 28th, 2011 at 10:09 pm

Posted in Version Control

Tagged with

Git/Mercurial: Pushing regularly

with 4 comments

I was reading a recent blog post by Gabriel Schenker where he discusses how his team is making use of Git and about half way through he says the following:

When using Git as your SCM it is normal to work for quite a while – maybe for a couple of days – in a local branch and without ever pushing the changes to the origin. Usually we only push when a feature is done or a defect is completely resolved.

We’ve been using Mercurial on the project I’m currently working on over the past few months and although it’s a similar tool we’ve been following a different approach.

We’ve got it setup the same way we would setup Subversion:

dscm.gif

We’ve been trying to push to the central repository as frequently as possible, just as we would if we were using Subversion.

I don’t know the Git workflow that well because I haven’t used it on a project yet but we’ve always found that it’s beneficial to integrate with code being written by others on the team as frequently as possible.

Not doing this can lead to the problems which Martin Fowler outlines in his post about feature branches.

We’ve tried to ensure that after every commit the build still passes although we do sometimes have broken versions in the code committed locally because we don’t run our full test suite before every local check in.

Even if a feature isn’t completed I still think it’s valuable to have what we’ve done so far checked in and it also helps remove the problem with needing to backup local repositories:

Since we are going to work locally potentially for days without pushing to the origin (our central repository) we might well loose our work if we have a hard disk crash or our office is flooded. Thus we need some backup strategy.

We just need to make sure the central repository is being backed up and then the danger of losing our work is significantly reduced.

Written by Mark Needham

June 19th, 2010 at 10:14 pm

Posted in Version Control

Tagged with ,

A reminder of the usefulness of Git

with 4 comments

Despite the fact that none of the projects that I’ve worked on have used Git or Mercurial as the team’s main repository I keep forgetting how useful those tools can be even if they’re just being used locally.

I ran into a problem when trying to work out why a Rhino Mocks expectation wasn’t working as I expected last week having refactored a bit of code to include a constructor.

I wanted to include the Rhino Mocks source code in our solution before and after the refactoring and step through the code to see what was different in the way the expectations were being setup.

My initial thought was that I could just check out the repository again in another folder and then include the Rhino Mocks source code there and step through it but unfortunately we have all the projects set up to deploy to IIS so Visual Studio wanted me to adjust all those settings in order to load the solution in the new checkout location.

I probably could have gone and turned off that setting but it seemed a bit too much effort and I realised that I could easily use Git to help me solve the problem.

I took a patch of the changes I’d made and then reverted the code before checking it into a local Git repository.

I updated the solution to include the Rhino Mocks code and then created a branch called ‘refactoringChanges’ so that I could then apply the patch that I’d created with my changes.

It was then really easy to switch back between the two branches and see the differences in the way that the Rhino Mocks was working internally.

The actual problem eventually turned out to be the way that the code calls Castle DynamicProxy but I didn’t get the chance to look further into it – we had learnt enough to know how we could get around the problem.

I’m in the process of including the source code for all the 3rd party libraries that we use in the solution on a separate Git branch that I can switch to when I want to debug through that code.

Sometimes I end up having to close down Visual Studio and re-open the solution when I switch to and from that branch but apart from that it seems to work reasonably well so far.

Written by Mark Needham

March 14th, 2010 at 12:45 am

Posted in Software Development

Tagged with ,

Pulling from github on Windows

with one comment

My colleague Dave Cameron has been telling me about his adventures playing around with Git Sharp (a C# port of the Java Git implementation jGit) so I thought I’d get a copy of the code and have a look as well.

I tend to check out all code bases from my host machine instead of virtual machine so I got the code all checked out on the Mac and accessed it via a shared folder on my VM.

The problem with doing this is I was unable to run the tests due to the following error which I received repeatedly:

System.Security.SecurityException: That assembly does not allow partially trusted callers.
at GitSharp.Tests.ObjectCheckerTests.testInvalidCommitNoTree2()

I’m not sure exactly why I got this error but it’s probably due to the fact that I didn’t open the project in normal mode since I am accessing it on a shared drive.

I decided it would probably just be quicker to checkout the code directly from the VM instead.

I installed msysgit which all worked fine and then I went to clone my repository:

C:\Playbox>git clone git@github.com:mneedham/GitSharp.git

Which resulted in the following error:

Initialized empty Git repository in C:/Playbox/GitSharp/.git/
Permission denied (publickey).
fatal: The remote end hung up unexpectedly

I had the public key setup on my github account from when I was using Git from my Mac but I hadn’t setup the private key that it requires inside my VM!

Browsing through the instructions on the github website I realised that I needed to copy my private key into the ‘.ssh’ folder (which as I understand is the default folder for ssh settings to be stored).

This folder needs to be at the root of your user folder rather than at the root of the drive as I originally thought.

Therefore it should be at ‘C:/Documents and Settings/Administrator/.ssh’ for example rather than ‘C:/.ssh’ as I mistakenly had it.

I couldn’t find a way to create a folder which started with a ‘.’ from Windows explorer so I just created a folder called ‘ssh’ and then renamed it using ‘mv ssh .ssh’ from the command line.

I then copied my private key (which I got from the Mac by running ‘less ~/.ssh/id_rsa’) into a file called ‘id_rsa’ inside the ‘.ssh’ folder and all was good:

C:\Playbox>git clone git@github.com:mneedham/GitSharp.git
Initialized empty Git repository in C:/Playbox/GitSharp/.git/
Enter passphrase for key '/c/Documents and Settings/Administrator/.ssh/id_rsa':
remote: Counting objects: 3138, done.
remote: Compressing objects: 100% (880/880), done.
remote: Total 3138 (delta 2319), reused 3039 (delta 2220)
Receiving objects: 100% (3138/3138), 3.72 MiB | 191 KiB/s, done.
Resolving deltas: 100% (2319/2319), done.
Checking out files: 100% (505/505), done.

In the process of working out what I’d done wrong I came across a nice post by Sergio Pereira which describes the whole process of getting up and running with Git in more detail.

Written by Mark Needham

August 18th, 2009 at 12:33 am

Posted in Version Control

Tagged with ,