Mark Needham

Thoughts on Software Development

Archive for August, 2012

Book Review: The Retrospective Handbook – Pat Kua

without comments

My colleague Pat Kua recently published a book he’s been working on for the first half of the year titled ‘The Retrospective Handbook‘ – a book in which Pat shares his experiences with retrospectives and gives advice to budding facilitators.

I was intrigued what the book would be like because the skill gap between Pat and me with respect to facilitating retrospectives is huge and I’ve often found that experts in a subject can have a tendency to be a bit preachy when writing about their subject!

In actual fact Pat has done a great job making the topic accessible to all skill levels and several times covers typical problems with retrospectives before describing possible solutions.

These were some of the things that I took away:

  • One of the most interesting parts of the book was a section titled ‘Be Aware of Cultural Dimensions’ where Pat covers some of the different challenges we have when people from different cultures work together.

    I found the power distance index (PDI) especially interesting:

    The extent to which the less powerful members of organisations and institutions accept and expect that power is distributed unequally

    If you come from a culture with a low PDI you’re more likely to challenge something someone else said regardless of their role but if you’re from a culture with a high PDI you probably won’t say anything.

    The US/UK tend to have low PDI whereas India has a high PDI – something I found fascinating when participating in retrospectives in India in 2010/2011. I think the facilitator needs to be aware of this otherwise they might make someone very uncomfortable by pushing them too hard to share their opinion.

  • A theme across the book is that retrospectives aren’t about the facilitator – the facilitator’s role is to help guide the team through the process and keep things moving, they shouldn’t be the focal point. In my opinion if a facilitator is doing that well then they’d be almost invisible much like a football referee when they’re having a good game!
  • The ‘First-Time Facilitation Tips’ chapter is particularly worth reading and reminded me that part of the facilitator’s role is to encourage equal participation from the group:

    A common, shared picture is only possible if all participants give their input freely and share their view of the story. This is difficult if one or two people are allowed to dominate discussions. Part of your role as a facilitator is to use whatever techniques you can to ensure a balanced conversation occurs.

    I think this is much easier for an external facilitator to do as they won’t have the burden of inter team politics/hierarchy to deal with.

    Pat goes on to suggests splitting the group up into smaller groups as one technique to get people involved, an approach I’ve found works and from my experience this works really well and gets around the problem that many people aren’t comfortable discussing things in big groups.

  • There’s nothing more boring than doing the same retrospective week after week, nor is there a quicker way to completely put people off them, so I was pleased to see that Pat dedicated a chapter to keeping retrospectives fresh.

    He suggests a variety of different techniques including bringing food or running the retrospective in a different location to normal to keep it interesting. I’ve heard of colleagues in Brazil doing their retrospectives outside which is another angle on this theme!

  • Another good tip is that when creating actions we don’t need to spend time getting someone to sign up for them right there and then – an alternative is to encourage people to walk the wall and pick ones they feel they can take care of.

I think this book compliments Esther Derby/Diana Larsen’s ‘Agile Retrospectives‘ really well.

I find their book really useful for finding exercises to use in retrospectives to keep it interesting whereas Pat’s book is more about the challenges you’re likely to face during the retrospective itself.

There’s lots of other useful tips and tricks in the book – these are just a few of the ones that stood out for me – it’s well worth a read if you’re a participant/facilitator in retrospectives on your team.

Written by Mark Needham

August 31st, 2012 at 9:18 pm

Posted in Books

Tagged with

The Curse Of Knowledge

without comments

My colleague Anand Vishwanath recently recommended the book ‘Made To Stick‘ and one thing that has really stood out for me while reading it is the idea of the ‘The Curse Of Knowledge’ which is described like so:

Once we know something, we find it hard to imagine what it was like not to know it. Our knowledge has “cursed” us. And it becomes difficult for us to share out knowledge with others, because can’t readily re-create our listeners’ state of mind.

This is certainly something I imagine that most people have experienced, perhaps for the first time at school when we realised that the best teacher of a subject isn’t necessarily the person who is best at the subject.

I’m currently working on an infrastructure team and each week every team does a mini showcase where they show the other teams some of the things they’ve been working on.

It’s a very mixed audience – some very technical people and some not as technical people – so we’ve found it quite difficult to work out how exactly we can explain what we’re doing in a way that people will be able to understand.

A lot of what we’re doing is quite abstract/not very visible and the first time we presented we assumed that some things were ‘obvious’ and didn’t need an explanation.

Having experienced a lot of blank looks we learnt that they’re only obvious if you spend the whole day doing them.

We needed to find a way to make what we were doing a bit more accessible by closing the gap between what people currently knew and our topic.

One effective way of doing this is to step up a level of abstraction and describe what the goal is rather than spending too long on the implementation details.

An example of something we presented was logstash, a tool used to collect logs from a bunch of different sources and then allows us to run regular expressions over them to make the data more easily searchable.

When explaining this in the demo a colleague explained its use from the point of view of something going wrong with one of the applications and the need to fix it quickly.

He pointed out that it’s much easier to do this if all the information we need is in one place and is easy to search, thereby explaining why logstash was useful.

By weaving our use of logstash into a more general story that people do understand we’ve been able to make this and other technical things that we’re working on more accessible to others.

We also realised that people tend to forget what we’ve talked about previously so now when we present something we spend a bit of time refreshing people’s minds so that they have a reference point to work from.

All the things we’ve tried try to make what can see abstract more concrete which is one of the six ways that the authors of ‘Made To Stick’ suggest to make our ideas more memorable.

It’s a good book, worth flicking through at the very least!

Written by Mark Needham

August 28th, 2012 at 9:22 pm

Ruby: Finding where gems are

with 2 comments

In my infrequent travels into Ruby land I always seem to forget where the gems that I’ve installed actually live on the file system but my colleague Nick recently showed me a neat way of figuring it out.

If I’m in the folder that contains all my ThoughtWorks graph code I’d just need to run the following command:

$ gem which rubygems

I then loaded up irb and wrote a simple cypher query executed using neography:

> require 'rubygems'
=> true
> require 'neography'
=> true
> neo = => 7476)
=> #<Neography::Rest:0x40d3ab8b @protocol="http://", @server="localhost", @cypher_path="/cypher", @log_file="neography.log", @authentication={}, @directory="", @log_enabled=false, @gremlin_path="/ext/GremlinPlugin/graphdb/execute_script", @parser={:parser=>CrackParser}, @max_threads=20, @port=7476>
> neo.execute_query("START n = node(1) RETURN n")
=> {"data"=>[[{"outgoing_relationships"=>"http://localhost:7476/db/data/node/1/relationships/out", "data"=>{"thoughtquitter"=>true, "name"=>"Marjorie Pries", "type"=>"person"}, "traverse"=>"http://localhost:7476/db/data/node/1/traverse/{returnType}", "all_typed_relationships"=>"http://localhost:7476/db/data/node/1/relationships/all/{-list|&|types}", "property"=>"http://localhost:7476/db/data/node/1/properties/{key}", "self"=>"http://localhost:7476/db/data/node/1", "properties"=>"http://localhost:7476/db/data/node/1/properties", "outgoing_typed_relationships"=>"http://localhost:7476/db/data/node/1/relationships/out/{-list|&|types}", "incoming_relationships"=>"http://localhost:7476/db/data/node/1/relationships/in", "extensions"=>{}, "create_relationship"=>"http://localhost:7476/db/data/node/1/relationships", "paged_traverse"=>"http://localhost:7476/db/data/node/1/paged/traverse/{returnType}{?pageSize,leaseTime}", "all_relationships"=>"http://localhost:7476/db/data/node/1/relationships/all", "incoming_typed_relationships"=>"http://localhost:7476/db/data/node/1/relationships/in/{-list|&|types}"}]], "columns"=>["n"]}

If I want to debug the execute_query function of neography then I’d need to make a change to /Users/mneedham/.rbenv/versions/jruby-1.6.7/lib/ruby/gems/1.8/gems/neography-0.0.26/lib/neography/rest.rb which is just a case of going up a few levels from where rubygems.rb lives and then finding the appropriate gem.

If I change that function to print a stupid message…

       def execute_query(query, params = {})
          puts "just testing you come up"
           options = { :body => {:query => query, :params => params}.to_json, :headers => {'Content-Type' => 'application/json'} }
           result = post(@cypher_path, options)

…when we run it through irb again we should see it:

> neo.execute_query("START n = node(1) RETURN n")
just testing you come up
=> {"data"=>[[{"outgoing_relationships"=>"http://localhost:7476/db/data/node/1/relationships/out", "data"=>{"thoughtquitter"=>true, "name"=>"Marjorie Pries", "type"=>"person"}, "traverse"=>"http://localhost:7476/db/data/node/1/traverse/{returnType}", "all_typed_relationships"=>"http://localhost:7476/db/data/node/1/relationships/all/{-list|&|types}", "property"=>"http://localhost:7476/db/data/node/1/properties/{key}", "self"=>"http://localhost:7476/db/data/node/1", "properties"=>"http://localhost:7476/db/data/node/1/properties", "outgoing_typed_relationships"=>"http://localhost:7476/db/data/node/1/relationships/out/{-list|&|types}", "incoming_relationships"=>"http://localhost:7476/db/data/node/1/relationships/in", "extensions"=>{}, "create_relationship"=>"http://localhost:7476/db/data/node/1/relationships", "paged_traverse"=>"http://localhost:7476/db/data/node/1/paged/traverse/{returnType}{?pageSize,leaseTime}", "all_relationships"=>"http://localhost:7476/db/data/node/1/relationships/all", "incoming_typed_relationships"=>"http://localhost:7476/db/data/node/1/relationships/in/{-list|&|types}"}]], "columns"=>["n"]}

And now hopefully I won’t forget where to find the gems!

Written by Mark Needham

August 25th, 2012 at 10:00 am

Posted in Ruby

Tagged with

puppetdb: Failed to submit ‘replace catalog’ command for client to PuppetDB at puppetmaster:8081: [500 Server Error]

with 2 comments

I’m still getting used to the idea of following the logs when working out what’s going wrong with distributed systems but it worked well when trying to work out why our puppet client which was throwing this error when we ran ‘puppet agent -tdv’:

err: Could not retrieve catalog from remote server: Error 400 on SERVER: Failed to submit 'replace catalog' command for client to PuppetDB at puppetmaster:8081: [500 Server Error]

We were seeing the same error in /var/log/syslog on the puppet master and a quick look at the process list didn’t show that the puppet master or puppetdb services were under a particularly heavy load.

Eventually we ended up looking at the puppetdb logs which showed that it was running out of memory:


2012-08-15 17:48:38,535 WARN  [qtp814855969-66] [server.AbstractHttpConnection] /commands
java.lang.OutOfMemoryError: Java heap space

The default /etc/default/puppetdb only has 192MB of heap space so we upped that to 1GB and problem solved! (at least for now)


# Init settings for puppetdb
# Location of your Java binary (version 6 or higher)
# Modify this if you'd like to change the memory allocation, enable JMX, etc
# These normally shouldn't need to be edited if using OS packages

Written by Mark Needham

August 16th, 2012 at 11:31 pm

Posted in Software Development

Tagged with ,

Presentations; Tell a story

without comments

A few years ago before an F# talk that I gave at the .NET user group in Sydney my colleague Erik Doernenburg gave me some advice about how I should structure the talk.


He suggested that in a lot of talks he’d seen the presenter rattle off a bunch of information about a topic but hadn’t provided any insight into their own experience with the topic.

If two people give a talk on the same topic they therefore end up being fairly similar talks even though each person may have a totally different perspective.

Erik suggested that people would find it much more interesting if I told a story about what I’d learnt about my topic (in this case F#).

I’ve been trying to follow that advice in other talks I’ve given since then and have noticed that there seem to be two main things that you need to get right for this approach to work well.

The Hook

Every story has a beginning but we need to find a way to take the audience from where they currently are to the beginning of the story and the amount of work that we need to get there can vary.

If it’s an audience that know the topic very well and are of a similar background to us then a quick explanation will do but we need to judge what our audience will be before working out how much context needs to be provided.

Most of the time I think people don’t get enough context and the talk becomes quite difficult to follow but I did give a presentation a couple of years ago where I ended up giving unnecessary context and watched people’s eyes glaze over!

In stand up comedy the key to a good joke is that you give just enough context so that the punch line of the joke makes sense. If you give a lot of context then there’s an assumption that the punch line is going to be stunning.

When writing jokes you therefore spend a lot of time refining the context until it’s just right. I think we should apply the same thinking in normal talks.

The flow

After we’ve got our hook sorted out and brought the audience to the beginning of our story the next thing to focus on is writing a coherent story which moves along at an appropriate pace.

The main thing to focus on here is to tell a story which is appropriate for the audience.

For example I recently gave two talks on a neo4j graph I’ve been working on with ThoughtWorks data – one to a ThoughtWorks audience and one to the neo4j user group.

In the first talk my story was an inward looking reflection on a bunch of data about ourselves and the tools I’d used to do that played a backseat role.

In the second talk I focused on the way that I’d iterated the model to answer questions that I had about the data over time.

In both cases I was able to identify a skeleton to hang the details of the story onto.

After we have a rough outline of an interesting story we need to fill in the details and make sure that the story actually makes sense and will be interesting to tell.

Sometimes this means switching the order that things would normally come in. I think this is ok – our goal isn’t to tell the exact chronological order in which we learnt things but to explain them in a way that’s easy to understand.

For example in my neo4j talk the order that I presented the questions I asked about the data wasn’t exactly the same as in real life. The points I wanted to make about modelling your data fitted better with a revised order so that’s what I did!

My way of preparing the story I want to tell is to sketch out some slides and then imagine presenting it to see if it makes sense – I often end up reworking it over a period of weeks until I’m happy.

It’s also useful to run through the story with someone else before hand to see if it makes any sense.

When not to tell a story

As with almost everything one idea isn’t applicable everywhere and there are other ways to present a topic other than giving your own perspective on it.

If you’re the expert on a particular topic and you’re presenting some new information about a language/tool then you don’t necessarily need to tell a story.

Having said that, a lot of good presentations I’ve watched tend to first describe the problem they’re trying to solve followed by their solution, which is effectively telling a story!

Another presentation technique is the ’10 things I learnt about…’ approach which makes sense if there doesn’t seem to be a coherent story line to weave.

Overall though I think the story telling approach is my preferred one and it pretty much mirrors what people do in normal conversations so we may as well take that familiarity with us when doing a presentation!

Written by Mark Needham

August 14th, 2012 at 10:16 pm

Posted in Communication

Tagged with

SSHing onto machines via a jumpbox

with 4 comments

We wanted to be able to ssh into some machines which were behind a firewall so we set up a jumpbox which our firewall directed any traffic on port 22 towards.

Initially if we wanted to SSH onto a machine inside the network we’d have to do a two step process:

$ ssh jumpbox
# now on the jumpbx
$ ssh internal-network-machine

That got a bit annoying after a while so Sam showed us a neat way of proxying the second ssh command through the first one by making use of netcat.

We put the following into ~/.ssh/config:

Host jumpbox jumpbox-ip
 Hostname jumpbox-ip
 User     user
 IdentityFile ~/.ssh/id_rsa
 ProxyCommand none
Host internal-network-machine
  Hostname internal-network-machine-ip
Host 10.*
 User     ubuntu
 ProxyCommand ssh jumpbox exec nc -w 9000 %h %p
 UserKnownHostsFile /dev/null
 StrictHostKeyChecking no

The ‘-w 9000’ flag defines a 2 1/2 hour wait period so that any orphaned connections will die off within that time.

%h and %p represent the host and port of the internal machine so in this case %h is ‘internal-network-machine-ip’ and the port will be 22.

We can then just do the following to ssh into the machine:

ssh internal-network-machine

Which is pretty neat!

This is explained further on benno’s blog and on the Open BSD journal.

Written by Mark Needham

August 10th, 2012 at 12:58 am

Posted in Shell Scripting

Tagged with ,

VCloud Guest Customization Script : [: postcustomization: unexpected operator

without comments

We have been doing some work to automatically provision machines using the VCloud API via fog and one of the things we wanted to do was run a custom script the first time that a node powers on.

The following explains how customization scripts work:

In vCloud Director, when setting a customization script in a virtual machine, the script:

  • Is called only on initial customization and force recustomization.
  • Is called with the precustomization command line parameter before out-of-box customization begins.
  • Is called with the postcustomization command line parameter after out-of-box customization finishes.
  • Needs to be a batch file for Windows virtual machines and a shell script for Unix virtual machines.

We wanted the script to run only when passed the ‘postcustomization’ flag because our script relied on some networking configuration which hadn’t yet been done in the ‘precustomization’ state.

We wrote something like the following script:

if [ x$1 == x"postcustomization" ]; then
  echo post customization

Unfortunately when we provisioned the node it hadn’t run any of the code within the if block and we saw the following message in /var/log/vmware-inc/customization.log:

5: [: xpostcustomization: unexpected operator

Nick pointed out that the test utility which we’re using to do the comparison on the 2nd line uses a single = in the POSIX shell even though it will work with double = in the bash shell.

We thought this was pretty strange since we are telling the script to run with the bash shell in the first line.

We eventually realised that the script was being spawned out to a POSIX shell by /root/.customization/ which is the script that gets called on power on:

 ((${SH} $POST_CUSTOMIZATION_TMP_SCRIPT_NAME "postcustomization" > /tmp/stdout.log) 2>&1 | ${TEE} -a /tmp/stderr.log)

I created a simple script to check the theory:

[ "mark" == "mark" ] && echo Mark

Which works fine when called directly:

$ ./ 

And throws the expected error when called with ‘sh’:

$ sh 
[: 3: mark: unexpected operator

We therefore needed to change our script to do the comparison with a single = like so:

if [ x$1 = x"postcustomization" ]; then
  echo post customization

Written by Mark Needham

August 6th, 2012 at 9:50 pm

Posted in Shell Scripting

Tagged with

neo4j: Creating a custom index with neo4j.rb

without comments

As I mentioned in my last post I’ve been playing around with the TFL Bus stop location and routes API and one thing I wanted to do was load all the bus stops into a neo4j database using the neo4j.rb gem.

I initially populated the database via neography but it was taking around 20 minutes each run and I figured it’d probably be much quicker to populate it directly rather than using the REST API.

Creating nodes is reasonably simple, and the code to add bus stops looks like this:

require 'neo4j' do
  stops_to_add = [ {:name => "Walworth Road", :code => 10001 }]
  stops_to_add.each do |stop|
    node = => stop[:name], :code => stop[:code], :type => "stop")
    puts "Code: #{stop[:code]}, Stop: #{stop[:name]}"

I wanted to be able to search for bus stops using cypher so I needed to create an index for each stop to allow me to do that easily.

I initially tried creating a Stop class and defining the index in there as suggested in the documentation but from what I could tell it created an index named after the string representation of the Stop object which made it difficult to use in cypher.

Eventually I came across another page which explained that I needed to create a ‘custom index’ if I wanted to be able to reference it by name.

I ended up with the following:

class StopsIndex
  extend Neo4j::Core::Index::ClassMethods
  include Neo4j::Core::Index
  self.node_indexer do
    index_names :exact => 'stops'
    trigger_on :type => "stop"
  index :code

As far as I understand this index gets triggered when you’re inside a transaction adding a node of type ‘stop’ which is what I’m doing here.

With the index defined this way it’s now possible to look up stops using cypher:

START stop = node:stops(code = "10001")

And when later on in the code I wanted to add a ‘route’ between stops I could look up the stops like so: do
  stop1 = StopsIndex.find("code: \"10001\"").first
  stop2 = StopsIndex.find("code: \"10002\"").first, stop1, stop2, { :bus_number => 1 })

The full code for this is on github.

Written by Mark Needham

August 5th, 2012 at 9:45 am

Posted in neo4j

Tagged with