Mark Needham

Thoughts on Software Development

Archive for November, 2010

The Adventures of Johnny Bunko – The Last Career Guide You’ll Ever Need: Book Review

with 3 comments

I read Dan Pink’s A Whole New Mind earlier in the year but I hadn’t heard of The Adventures of Johnny Bunko until my colleague Sumeet Moghe mentioned it in a conversation during ThoughtWorks India’s XConf, an internal conference run here.

The book is written in the Manga format so it’s incredibly quick to read and it gives 6 ideas around building a career.

I’m generally not a fan of the idea of ‘building a career’ – generally when I hear that phrase it involves having a ‘five year’ plan and other such concepts which I consider to be pointless.

I much prefer to just focus on what I’m most interested in at the moment and then have the flexibility to be able to focus on something else if that interests me more.

Luckily this book is not at all like that and these are the ideas Dan Pink suggests:

There is no plan

The most interesting part of this piece of advice is Pink’s suggestion that when we make career decisions we make them for two different types of reasons:

  • Instrumental reasons – we do something because we think it’s going to lead to something else regardless of whether we enjoy it or think it’s worthwhile.
  • Fundamental reasons – we so something because it’s inherently valuable regardless of what it may or may not lead to.

I think this pretty much makes sense – it’s very difficult to predict what’s going to happen so you might as well make sure you’re doing what you want at any given time.

Think strengths not weaknesses

Marcus Buckingham has several books on this subject but the general idea is that we should look to focus on things that we’re good at or are really motivated to become good at.

By inference this means that we want to avoid spending time on things that we’re not good at and/or not interested in becoming good at.

I recently came across a blog post written by Robbie Maciver where he suggests that we can use a retrospective to help us identify team member’s strengths and work out how best to utilise them.

I think there’s sometimes a reluctance for people to volunteer their strengths so we end up with people working on tasks that they hate that other people in the same team would enjoy.

It’s not about you

Pink then goes on to point out that even if we do work out how to play to our strengths we still need to ensure that we’re focusing on our customer/client.

Pink calls this focusing outward not inward.

In terms of working in technology consulting that would therefore be along the lines of ensuring that we’re focused on solving our clients’ problems in the best way for them rather than the most interesting way for us.

Persistence trumps talent

This idea has been covered in several books including Outliers, Talent is Overrated and The Talent Code.

The underlying idea is that people aren’t born brilliant at any skill, it only comes through practice and if we’re intrinsically motivated to do something then we’re much more likely to put in the time it requires to become really good.

To add to that I think intrinsic motivation is highest when we’re focusing on our strengths so this piece of advice is linked with the earlier one.

Make excellent mistakes

Pink talks about making excellent mistakes which he describes as ‘mistakes that come from having high aspirations, from trying to do something nobody else has done’.

I think this is somewhat linked to the idea of failing fast although it seems to be perhaps one level beyond that.

Sumeet gave a talk at XConf where he described the new approach to ThoughtWorks University and pointed out that one of the most important goals was to allow attendees to make ‘excellent mistakes’

One observation I have here is that smaller companies seems more willing to make excellent mistakes. As organisations grow the aversion to risk because of worries about loss of reputation increases which makes excellent mistakes less likely.

I find Eric Ries’ lean startup ideas particularly interesting with respect to failing fast and my former colleague Antonio Terreno recently linked to a video where the General Manager of Forward 3D explains how they do this.

Leave an imprint

Pink talks about the importance of doing work which actually has some meaning or leaves the world in a better place than it was before.

This is probably the one that I can relate to least. I’m writing software which is what I’m passionate about but I wouldn’t say the problems I work on are having much impact on the world.

This one therefore leaves me with the most to think about.

Written by Mark Needham

November 21st, 2010 at 5:02 pm

Posted in Books

Tagged with

From unconsciously incompetent to consciously incompetent

with one comment

One of the cool things about software development is that despite writing code for 5 years professionally and just under 10 altogether, there are still a phenomenal number of things that I don’t know how to do.

The learning opportunities are vast!

One of the areas which I’ve known I don’t know that much about is Unix command line tools such as awk and sed.

Since the majority of projects that I’ve worked on have involved using Windows as the development environment I’ve never had extended exposure to the types of problems we get on a project which require their use.

I’ve finally got the opportunity on my current project and my colleague Priyank and I have been looking to use those tools wherever we can.

We’ve learnt how to do some very basic commands with both the tools and although those few command have been very useful for allowing us to quickly manipulate text files we didn’t quite realise how little we knew until coming across Eric Pement’s ‘useful one-line scripts for Sed‘ article.

I knew that Sed and Awk were powerful tools but I had no idea that you could do this many things with them.

I’m now reading the sed and awk book so that I can close this gap in my knowledge and trying to use these tools wherever possible even if it does slow me down a bit initially.

In terms of the four stages of competency model I would say I’ve moved from being ‘unconsciously incompetent’ with respect to sed and awk to being ‘consciously incompetent’.

Now I just need to reach competence!

Written by Mark Needham

November 19th, 2010 at 8:20 pm

Capistrano, sed, escaping forward slashes and ‘p’ is not ‘puts’!

with 2 comments

Priyank and I have been working on automating part of our deployment process and one task we needed to do as part of this is replace some variables used in one of our shell scripts.

All the variables in the script refer to production specific locations but we needed to change a couple of them in order to run the script in our QA environment.

We’re therefore written a sed command, which we call from Capistrano, to allow us to do this.

The Capistrano script looks a little like this:

task :replace_in_shell do
	directory = "/my/directory/path"
	sed_command = "sed 's/^some_key.*$/#{directory}/' shell_script.sh > shell_script_with_qa_variables.sh"
	run sed_command
end

Unfortunately this creates the following sed command which isn’t actually valid syntactically:

sed 's/^some_key.*$//my/directory/path/' shell_script.sh > shell_script_with_qa_variables.sh

We decided to use ‘gsub’ to escape all the forward slashes in the directory path and to work out which parameters we needed to pass to ‘gsub’ we started using irb.

Executing gsub with the appropriate parameters leads us to believe that 2 backslashes will be added:

ruby-1.8.7-p299 > "/my/directory/path".gsub("/", "\\/")
 => "\\/my\\/directory\\/path"

This is because there IRB is implicitly called ‘inspect’ on the result which shows a different string than what we would actually get.

While writing this blog post I’ve also learnt (thanks to Ashwin) that ‘p’ is not the same as ‘puts’ which is what I originally thought and has been driving me crazy as I try to understand why everything I print includes an extra backslash!

The following code:

p "/mark/dir/".gsub("/", "\\/")

is the same as typing:

puts "/mark/dir/".gsub("/", "\\/").inspect

We were able to change our Capistrano script to escape forward slashes like so:

task :replace_in_shell do
	directory = "/my/directory/path"
	sed_command = "sed 's/^some_key.*$/#{directory.gsub("/", "\\/"}/' shell_script.sh > shell_script_with_qa_variables.sh"
	run sed_command
end

Written by Mark Needham

November 18th, 2010 at 6:40 pm

Posted in Ruby

Tagged with , ,

Rails: A slightly misleading error

without comments

We recently created a new project to handle the reporting part of our application and as with all our projects we decided not to checkin any configuration “.yml’ files but rather ‘.yml.example’ files which people can then customise for their own environments.

So in our config directory would look something like this when you first checkout the project:

  • config
    • database.yml.example
    • some.yml.example

And we’d need to copy those files to get ‘.yml’ versions, changing any parameters that we need to for our local environment.

The disadvantage of this approach is that you have an extra step on using a project for the first time, a step that I’ve been meaning to automate.

Several people ran into a somewhat confusing error message when running our rake file after forgetting to create these ‘.yml’ files which looked like this:

> rake db:migrate
(in /Users/mneedham/SandBox/project)
rake aborted!
uninitialized constant ActiveRecord
 
(See full trace by running task with --trace)

Running with –trace didn’t reveal our mistake but interestingly launching ‘script/console’ did!

script/console
Loading development environment (Rails 2.3.5)
/Users/mneedham/SandBox/project/config/environment.rb:4:in `initialize':Errno::ENOENT: No such file or directory - /Users/mneedham/SandBox/project/config/some.yml
/Users/mneedham/SandBox/project/vendor/rails/railties/lib/rails/backtrace_cleaner.rb:2:NameError: uninitialized constant ActiveSupport::BacktraceCleaner
/Users/mneedham/SandBox/project/vendor/rails/railties/lib/console_with_helpers.rb:5:NameError: uninitialized constant ApplicationController
ruby-1.8.7-p299 >

This is the environment.rb file in which we were loading that yml file:

config/environment.rb

....
require 'yaml'
some_file = File.join(File.dirname(__FILE__), "some.yml")
...

It’s a relatively simple mistake to make so I was surprised that rake didn’t tell us that the file didn’t exist rather than failing when trying to require ActiveRecord.

Written by Mark Needham

November 16th, 2010 at 9:17 pm

Posted in Ruby

Tagged with ,

Retrospectives: My first time facilitating

with 5 comments

Despite being part of numerous retrospectives over the past few years I don’t remember actually facilitating one until my current team’s last week.

I’ve gradually come to appreciate the skill involved in facilitating this type of meeting having originally been of the opinion that there wasn’t much to it.

I recently read Agile Retrospectives which has loads of different ideas for activities beyond just creating ‘went well’ and ‘could improve’ columns and then filling those in as a group.

In the best retrospective I’ve attended recently the facilitator had us work together in small groups while coming up with ideas to fill in on the retrospective starfish.

I really like the idea of working in small groups because I think it encourages more participation in the discussion of problems on the team.

My general observation is that a sizeable percentage of people are more comfortable taking part in discussions in smaller groups than with the whole team (~ 25 people).

We split into groups of 4 or 5 and then populated a time line of the project since the last iteration before going across the board and discussing the most prominent topics.

These were some of the main areas that I had thoughts about during and after facilitating this retro:

Time keeping

Perhaps somewhat ironically given the amount I’ve been hassling my team mates to keep meetings short this one over ran by about 25 – 30 minutes, taking around 90 minutes instead of 60.

I had a rough idea of how quickly we needed to move across the board in order to finish on time but it got thrown completely off track by one point which resulted in a much longer discussion than I had expected.

I tried to move across the rest of the items a bit more quickly to make up for that but it didn’t really work.

I’d be interested to hear what a more experienced facilitator would have done in this situation as I’m sure it’s very common.

Participating

I’ve written previously about the dangers of giving your opinion when facilitating a retrospective so I tried to make sure that I kept out of any discussions that happened and allowed others to talk.

On a couple of occasions I was asked to give my opinion on certain things so I had to step out of my facilitating role temporarily, join the discussion and then step back in.

I think this worked reasonably well but I can see how it would be difficult to keep quiet if you had really strong opinions on a topic being discussed!

Summarising the discussion

I see part of the role of the facilitator being to try and summarise what is being discussed so that it can be written up afterwards and distributed to the team.

I didn’t realise how difficult this is, especially if the discussion drifts slightly away from its original direction.

I found it quite tricky to follow what was going on at times but luckily one of my colleagues was able to help me out when I didn’t quite get it right.

Overall it was an interesting experience and I hope I’ll get another chance to try this role again soon although for now we’re trying to rotate it around the team to give everyone an opportunity to facilitate.

Written by Mark Needham

November 15th, 2010 at 7:52 pm

Posted in Agile

Tagged with

Agile: Increasing team sizes

with 3 comments

A fairly common trend on nearly every project I’ve worked on is that at some stage the client will ask for more people to be added to the team in order to ‘improve’ the velocity.

Some of the most common arguments against doing so are that it will initially slow down the team’s velocity as the new members learn the domain, code base and get to know the other members of the team.

My colleague Frank Trindade wrote a blog post about 18 months ago where he described his observations of the above happening on a team he’d been working on.

Frank also identifies the following consequence of increasing a team’s size which I think is perhaps even more important:

the decrease of communication levels brought by the addition of new nodes

The majority of teams that I’ve worked on have had 15 or less people but there have been a couple of exceptions including my current team.

We currently have somewhere around 25 people working in Pune and then perhaps another 10 in Chicago so it’s the biggest team that I’ve worked on. Having said that I do recognise that it’s quite small in size compared to some of the other projects in India.

The following are some of the consequences with respect to communication that I’ve noticed. These are applicable as we increase team size and also just generally when having larger teams.

Standup takes longer

This is somewhat inevitable since there are more people taking part. It’s therefore much easier to lose focus on what other people are saying.

We’ve managed to reduce the time to something more reasonable by not having any discussions in the stand up, instead taking them offline.

Other meetings are much more difficult to control with more people in and I think it requires quite a skilful facilitator to allow everyone to take part and still ensure that the meeting doesn’t overrun.

Technical decision consensus

Getting consensus on any technical approach is much more difficult than when there are just a few developers on the team.

Technical discussions seem to take way longer than I’m used to because we try to ensure that everyone gets to express their opinion.

There are also more people available to then disagree with that opinion which tends to mean that we go around in circles more frequently.

I’ve found that there are rarely more than 3 or 4 ways to solve any problem so a lot of the time people are expressing similar opinions in a slightly different way.

When the team gets this big I don’t think it makes sense to include everyone in all technical decisions. My current thinking is that having a group of 3 or 4 people involved in each one is more than enough.

Greater distance between team members

I think the optimal setup for a software team is to have all the team members working on a single table – this means we can have around 12 members per team.

That way everyone is within talking distance of each other and communication remains smooth.

Once we have more people than that we need another table which means that there are 2 rows of people in between people on the far side of each table.

In The Organisation and Architecture of Innovation the authors include the following diagram which I think is quite revealing:

prob_communication.tiff

Although the authors are talking about bigger distances, I think the amount of communication is less if we are separated even by this little amount of space.

You now need to take into account the number of people that you might be disrupting by shouting across the tables which was less of an issue before.

In summary

Adding people to a team may increase the velocity but it’s unlikely to lead to an improvement which is directly proportional to the number of people added.

From my experience it might make more sense to take a bit longer building the product or application rather than aiming at a strict time deadline with more people.

These are just surface observations – I’m sure others who have worked for longer in big teams would be able to point out a whole range of other consequences.

Written by Mark Needham

November 14th, 2010 at 11:51 am

Posted in Agile

Tagged with

Experiments in not using the mouse

with 5 comments

Priyank and I have been pairing a bit lately and we thought it’d be interesting to try and not use the mouse for anything that we had to do while pairing.

Editor

Priyank uses GVim (Yehuda Katz recommends MacVim if you’re using Mac OS) so we already don’t need to use the mouse at all when we’re inside the editor.

One annoying thing we found is that sometimes we wanted to copy stuff from the terminal into GVim and couldn’t think of a good way to do that without selecting the text on the terminal with a mouse and then ‘Ctrl-C’ing.

A bit of Googling led us to the xsel command which takes standard input and makes it available on the clipboard.

For example we’ve been using sed and wanted to copy the code we’d been spiking into a shell script:

# code to replace backslashes with pipes in a file
echo "sed -i 's/\\/|\g' some_file.text'" | xsel -bi

In OS X we have ‘pbcopy‘ which allows us to do the same type of thing.

Once we’ve got that onto the clipboard we need to use ‘Ctrl R’ followed by ‘+’ in order to paste into GVim.

The next step is to get away from having to use the arrow keys which are annoyingly far away from all the others on a full sized keyboard but we’re not there yet!

Browser

We’re using Chrome so Priyank has installed the Vimium extension which allows us to use Vim shortcuts inside Chrome.

So far I’ve only been using the ‘f’ command which gives you key combinations to click on any of the links on the page but it’s still much more fun than having to scroll around with the mouse!

If anyone has any other tips or tools for us to experiment with that’d be cool to hear about – it was much more fun constraining ourselves slightly and seeing how we got on!

Written by Mark Needham

November 12th, 2010 at 3:43 pm

Distributed Agile: Communicating big design decisions

with 2 comments

Although we mostly split the work on my project so that there aren’t too many dependencies between the teams in Chicago and Pune, there have still been some times when we’ve designed major parts of the code base in Pune and have needed to communicate that to our Chicago colleagues.

I’ve never seen this situation so it’s been interesting to see which approaches work in trying to do this effectively and allowing the people in the other location to have input as well.

Explanation on specific email threads

While it’s useful to communicate the approach being taken by email we’ve found that it makes more sense to have a specific email thread for that conversation rather than tagging it onto something else.

It’s surprisingly easy for emails to get lost amongst the hundreds of others that people receive each day but at least if the important messages are all under an email with a clear title then it’s much easier to deal with.

While discussing this with Saager, he suggested that an effective approach he’s used previously is to include details of design decisions taken in commit messages and then point people towards those particular commits.

Of course however skilful we may be in communicating via email it does help to talk on a conference call as well – that medium seems to work better for explaining the reasoning behind decisions, especially if there’s a disagreement in the approach.

Commit early

As much as we can explain design decisions in emails or conference calls it’s still not as useful for people as seeing the actual code which is something we didn’t totally appreciate until recently.

We’re now trying to ensure that we check in more incrementally when making big design changes.

This should help to tighten the feedback loop and ensure that we get the benefit of the design skills of people onshore as well as offshore.

Something which I initially considered as a disadvantage with checking in incrementally is that you will most likely get suggestions/criticism about what you’ve done even if you had already planned to address some of those areas anyway.

As a colleague correctly pointed out, the intention is generally good and it’s not the end of the world if someone suggests something you could have done better anyway.

In summary

These types of things seem obvious looking back at them now but I guess it isn’t so obvious to me because you never have to think about them at all when anyone interested in the code is sitting in the same room as you as it’s been for me when working onshore.

I’d be interested to hear any other approaches that have worked well for others working in a distributed fashion.

Written by Mark Needham

November 10th, 2010 at 7:58 pm

Posted in Distributed Agile

Tagged with

Active Record: Nested attributes

without comments

I recently learnt about quite a neat feature of Active Record called nested attributes which allows you to save attributes on associated records of a parent model.

It’s been quite useful for us as we have a few pages in our application where the user is able to update models like this.

We would typically end up with parameters coming into the controller like this:

class FoosController < ApplicationController
   def update
      # params = { :id => "1", :foo => { :baz => "new_baz", :bar_attributes => { :value => "something" } } }
      Foo.update_instance(params[:id], params[:foo])
      ...
   end
end

Our original implementation of ‘update_instance’ looked like this:

class Foo < ActiveRecord::Base
   has_one :bar
 
   class << self
      def update_instance(id, attributes_to_update)
         instance = Foo.find(id)
         instance.attributes = attributes_to_update
         instance
      end
   end
end

Unfortunately when we execute that code the ‘bar’ association gets completely removed because we didn’t specify the id of ‘bar’ when we were updating the attributes.

We need to change the code slightly to make sure it doesn’t do that:

class Foo < ActiveRecord::Base
   has_one :bar
 
   class << self
      def update_instance(id, attributes_to_update)
         instance = Foo.find(id)
         attributes_to_update[:bar_attributes][:id] = instance.bar.id
         instance.attributes = attributes_to_update
         instance
      end
   end
end

It now works as we’d expect.

There’s other cool stuff that you can do with nested attributes described on the documentation page if you have ‘has_many’ associations but for now we’re just using the simpler ‘has_one’.

Written by Mark Needham

November 9th, 2010 at 6:37 pm

Posted in Ruby

Tagged with ,

Distributed Agile: Communication – Reliance on one person

with one comment

Continuing with my series of observations on what it’s like working in a distributed agile team, another thing that I’ve noticed is that it’s useful to try and ensure that there is communication between as many people as possible in the two cities.

This means that we want to ensure that we don’t have an over reliance on one person to handle any communication.

We have a call once a day between developers in Pune and Chicago and the Chicago guys have been able to achieve this by rotating the person attending the call.

In Pune we’ve typically had half the developers attending the call but perhaps only one or two actually speaking.

I originally didn’t think of this as being a problem because those two guys were really good at communicating and in other teams I’ve worked on we’d typically have the tech lead and maybe one other communicating with other teams.

In this case we are communicating with people in the same team and by chance a few weeks ago both of the main communicators from Pune were away at the same time so we had to have the Chicago call without them.

Perhaps unsurprisingly others were able to fill the gap that had been left and as an added benefit the guys in Chicago had now had the opportunity to interact directly with more people from Pune.

A interesting and positive side effect I’ve noticed is that the people who had now started communicating directly with team members in Chicago seemed to become more involved in the project as a whole and are now also more likely to communicate via email than they may have been previously.

Our challenge now that both the original guys are back is to ensure that the whole team still remains involved in communication with Chicago and don’t delegate back to those two every time.

Written by Mark Needham

November 8th, 2010 at 1:56 pm

Posted in Distributed Agile

Tagged with