Mark Needham

The last two projects that I’ve worked on I’ve been on the project from right near the start, and one thing that’s been consistent in both projects is that we’ve spent time early on in the project trying to reduce technical risk.

In my most recent project this has involved getting infrastructure in place early on, and in the previous one it involved working on technical spikes for several weeks to prove that what the client was asking for was actually technically possible.

While reading Crystal Clear I noticed that this approach is the same as that advocated by Alistair Cockburn with regards to putting the highest learning tasks up front before business stories, and the idea of the walking skeleton.

Some people consider this opposed to the agile approach in that we are creating something before we absolutely need to use it. I think this is a difference in the understanding between doing something at the last responsible moment and the last possible moment.

The former is the way that I understand we should do things – get the pain out the way early on and not later on death march style

I wonder whether it should be the same on a per story basis or if there is no gain from taking this approach – i.e. usually the backend side of stories is what takes up the time so should we do this first and then drive out the rest or is it better to drive from the front?

One thing I have learnt from conversation about this idea is that we still need to show some progress to the business and therefore need to balance the need to reduce project risk with completing business stories.

Clearly there is a trade off to be made here but I think the ‘risk up front’ approach is preferable to leaving the integration until later on when it is both more difficult to do and doesn’t leave much room for error either.

Debugging 3rd party library code quickly and effectively is one of the skills which most obviously separates Senior and Junior developers from my experience.

From observation over the last couple of years there are some patterns in the approaches which the best debuggers take.

Get more information

Sometimes it’s difficult to understand exactly how to solve a problem without getting more information.

Verbose logging mode is available on the majority of libraries and provides the information showing how everything fits together which is normally enough information to work out how to solve the problem.

Look at the source code

My natural approach, when stuck using a 3rd party library for example, is to read the documentation but I have noticed that better debuggers than myself head to the source code much earlier and try to work out what is going on from there.

This approach certainly makes sense and when I have problems with project code my instinct is to look at the offending code straight away and try to work out what’s going on. I haven’t quite got this discipline when it comes to library code just yet.

The art of this approach comes in being able to read through the code and realising quickly which parts of the code are important and which can be skimmed over.

An article I read about the expert mind seems to confirm this, stating that an expert in any discipline doesn’t analyse more options than others, only better ones.

Don’t assume it works

Closely linked to the above is the assumptions we make when debugging.

The best debuggers don’t assume that the code does exactly what they expect it to, they take a more critical approach and try to work out whether or not it is actually working by changing small things and seeing what the impact is.

If it does turn out that there is a bug then we can look at the source code and work out whether there is a way around the problem or if there is only one way to solve the problem, submit a patch to fix it.

One change at a time

I think this is probably the most important yet the simplest of the approaches.

It sounds so obvious yet it’s so easy to make change after change after change and eventually solve the problem but have no idea which change or combination of changes it was that fixed it.

We need to ensure that we know why we are making each change and revert it if it doesn’t have the desired effect.

Read the error message

Reading the error messages that we get and not the error message that we think we saw is yet another obvious approach but one which I have certainly violated.

Sometimes we need to slow down a bit when debugging problems and read exactly what the message is telling us and work from there.

In Summary

Most of the approaches I’ve noticed seem very obvious but I find that it requires quite a lot of discipline to apply them. When I do approach debugging with this more logical approach problems become much easier to solve.

Some debugging books

One of the best books I have read with regards to debugging code is Debugging – The 9 Indispensable Rules for Finding Even The Most Elusuive Software and Hardware Problems – I actually had the 9 rules stuck to my machine for a couple of months earlier this year, and probably should do that again.

I also noticed that the Pragmatic Programmers have a new book coming out called ‘Debug It‘ – it will be interesting to see the similarities and differences between this book and the other one.

We have been making good use of Hamcrest matchers on my current project for making assertions, and have moved almost entirely away from the more traditional JUnit assertEquals approach.

There are several reasons why I find the Hamcrest matcher approach to be more productive – it’s more flexible, more expressive and when an assertion fails we have a much better idea about why it has failed than if we use a JUnit assertion for example.

This applies especially when we get a test failing as part of the build as compared to running a test from the IDE where the source code is close at hand and non descriptive error messages may not be such a problem.

It therefore makes sense when writing custom Hamcrest matchers to ensure that we do indeed provide a clear error message so that it is obvious how to fix the test.

The convention seems to be that we should first state the static method name of the matcher and then in brackets list the expected arguments.

To give an example from a matcher we wrote yesterday:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

import org.hamcrest.Description;
import org.hamcrest.Factory;
import org.hamcrest.TypeSafeMatcher;
 
public class ContainsAllOf<T> extends TypeSafeMatcher<T> {
    private String[] messages;
 
    public ContainsAllOf(String... messages) {
        this.messages = messages;
    }
 
    public void describeTo(Description description) {
        description.appendText("containsAllOf(");
        for (String message : messages) {
            description.appendText(",");
            description.appendValue(message);
        }
        description.appendText(")");
    }
 
    @Factory
    public static <T> ContainsAllOf containsAllOf(String... messages) {
        return new ContainsAllOf(messages);
    }
 
    public boolean matchesSafely(T t) {
        return contains(t, messages);
    }
 
    private boolean contains(T t, String[] messages) {
        boolean containsAllMessages = true;
        for (String message : messages) {
            if (!t.toString().contains(message)) {
                return false;
            }
        }
        return containsAllMessages;
    }
 
}

If we call this in our test with a value that doesn’t exist:

assertThat("mark's cool message", containsAllOf("mark", "cool", "message", "notThere"));

Running the test results in the following error:

java.lang.AssertionError: 
Expected: containsAllOf(,"mark","cool","message","notThere")
     got: "mark's cool message"

We can easily see what the problem is and how to go about fixing it, which I feel is the most important thing when it comes to test assertions.

Last week I came across what I have decided is the file system equivalent of commenting out code – not deleting directories when we are no longer using them.

The specific situation we ran into was while trying to make some Tomcat configuration changes but everything we changed was having no effect on what we were seeing on the web site.

Eventually we realised that we were actually changing the configuration in the wrong place – we actually had two Tomcat folder lying around. One for Tomcat 6 and one for Tomcat 5.5. We are running the former but were changing the configuration for the latter by mistake!

Of course the underlying problem was that we weren’t paying enough attention to what we were changing, but if there had been only one possible place to make changes then we wouldn’t have had a problem.

Lesson Learned: Treat the file system like the code – if something is not needed delete it straight away. It can always be retrieved from source control in the future if it’s actually needed.

We ran an Object Calisthenics variation of Coding Dojo on Wednesday night as part of ThoughtWorks Geek Night in Sydney.

Object Calisthenics is an idea suggest by Jeff Bay in The ThoughtWorks Anthology , and lists 9 rules to writing better Object Oriented code. For those who haven’t seen the book, the 9 rules are:

1. Use only one level of indentation per method
2. Don’t use the else keyword
3. Wrap all primitives and strings
4. Use only one dot per line
5. Don’t abbreviate
6. Keep all entities small
7. Don’t use any classes with more than two instance variables
8. Use first-class collections
9. Don’t use any getters/setters/properties

We decided to try and solve the Bowling Game Problem while applying these rules. We coded in Java as this was a language everyone in the room was comfortable with. It would have been cool to try out Ruby or another language but I’m not sure if this type of setting is the best place to learn a new language from scratch.

I hadn’t arranged a projector so we couldn’t adopt the Randori approach. Instead we split into three pairs rotating every half an hour, discussing how each pair was approaching the problem at each change.

Learning from the problem

I was surprised how difficult the problem was to solve using the Object Calisthenics rules. There were several occasions when it would have been really ease to expose some state by introducing a getter but we had to try another way to attack the problem.

We have been following the approach of wrapping all primitives and strings on my current project as ‘micro types‘ so this rule wasn’t new to me but the general feeling early on was that it was quite annoying to have to do. From my experience on my project it does help to encourage a more object oriented approach of keeping the data with the behaviour.

This approach to object orientation is very extreme but the author suggests giving it a try on some small projects as being able to code like this will result in you seeing problems in a different way. I noticed today that I was always on the lookout for ways to ensure that we didn’t expose any state so it’s had a slight influence on my approach already.

We had an interesting discussion about mid way through about whether we should implement equals and hashcode methods on objects just so that we can test their equality. My general feeling is that this is fine although it has been pointed out to me that doing this is actually adding production code just for a test and should be avoided unless we need to put the object into a HashMap or HashSet when the equals/hashcode methods are actually needed. The only alternative I can think of is to not test object equality and instead only test equality where we have primitives or to test for equality by using reflection.

From seeing the approaches others had taken I realised that the approach we took on my machine was too difficult – we would have been more successful by adopting baby steps.

Learning about the format

We initially started out trying to design a solution to the problem on a white board before getting to the coding but this didn’t work particularly well so we abandoned this and went straight to the code.

Each machine had three different pairs working on the problem over the duration of the night, with one person always staying on the machine and the others rotating. While we all had slightly different approaches to the problem it would have been interesting to see if we could have progressed further using the Randori approach with everyone having input to the same code base.

None of the pairs managed to complete the problem, and there was concern that the problem was too big to fit into the 90 minutes we spent coding. After speaking with Danilo and reading his Coding Dojo paper it seems that this is not necessarily a bad thing and the focus is supposed to be more on the learning than problem completion.

It was certainly an interesting experience and I had the opportunity to work with some people that I haven’t worked with before. We are hopefully going to make these Coding Dojos a regular feature and try out some different approaches to see which works best for us.

On this occasion I selected the problem but in the future we would look to make it a group based decision depending on what people are keen to learn.

One of the interesting situations that can arise when pair programming is that one person dominates the driving and their pair can hardly get a look in.

This is not necessarily because they are hogging the keyboard – it is often just the case that they are the stronger technically in the pair and the other person isn’t willing to ask for the keyboard.

A big part of the value in pair programming comes from having both people taking turns at driving and navigating from my experience and there are several ideas that I have come across for trying to encourage a more collaborative approach to pair programming.

Ping Pong Pairing

The idea of this approach is to encourage the switching of the keyboard between both people very explicitly.

Based on the assumption that we are adopting a TDD approach, the first person starts out by writing the test, then they switch and the second person writes the code to make the test pass. Then they write the next test and the first person writes the code to make that test pass.

With this approach both people end up with fairly equal keyboard time and will hopefully feel more involved in the development effort.

My colleague Sam Newman previously wrote about ping pong pairing, including an approach to ensure that both members get exactly the same amount of keyboard time per day.

Learning the Art of Navigating

As a general rule I think the majority of people find it much easier to drive than to navigate.

When navigating it often feels that you are not adding value and it sometimes becomes confusing as to exactly what you should be doing.

Having said that I have worked with some phenomenally good navigators during my time pair programming. While I don’t claim to know how to teach someone how to be a good navigator (I’m still working on it myself), there do seem to be some distinct skills that you can apply to carry out this role effectively.

• Keeping an eye on the big picture – are we solving the right problem, has this piece of code become too difficult to test, are we testing it in the correct way etc.
• Tracking the tasks needed to complete a story and being ready to continue with the next one when the driver has finished the current one.
• Sharing tips on how to navigate around the workspace more effectively – keyboard shortcuts, different tools etc.

If we can become equally effective at contributing as both a navigator and a driver then it doesn’t become so imperative to drive all the time.

Two of everything

This was an approach I was introduced to more recently and involves having two keyboards and two mice connected to the workstation.

The benefit of this approach is that the difficulty of prising the keyboard away is removed and we can just grab our own keyboard and start typing when we have some input to make.

Clearly this needs to be done relatively sensibly otherwise otherwise we end up with both people typing or moving the mouse at the same time. From my experience of this technique that actually rarely happens.

There is also an increased spending on equipment but I wouldn’t imagine this would be that significant.

The Book

Crystal Clear by Alistair Cockburn

The Review

This was a book which had been recommended to me by a colleague a few months ago as one of the best software development books to read, and after hearing Ian Cooper describe how his team was implementing some of the ideas at the Alt.NET conference I decided I’d give it a read.

I have been working in an Agile/XP environment at ThoughtWorks for the last two years so my context coming into the book was around understanding where the overlap with Crystal Clear was, what differences there are and how I can apply these on my projects

What did I learn?

• I thought it was very interesting that Alistair Cockburn came across the agile principles through the need for efficiency rather than the need to handle rapidly changing requirements. It is also interesting that his project background is of the fixed price, fixed scope variety. This is generally considered an area where it is difficult to achieve success with an agile approach although I have worked on projects like this which have successfully delivered as well.

  From my experience the key benefit of the agile approach touted is the ability to rapidly respond to changes in requirements but I also believe that the gains in efficiency we gain from the quick feedback cycles is certainly equally important in my book. Tim Bray has an interesting article talking about the potential for agile to thrive in this current financial climate due to the efficiency gains it can provide.

• I found the idea of software projects being a cooperative game to be a very good metaphor for understanding the trade offs that we make on projects. We need to consider the current game and future games that we will need to play. We can make decisions which benefit us in the current game (e.g. taking shortcuts on design) but we must remember that these decisions will have an impact on games that occur in the future (e.g. increased technical debt to pay off).
• The approach I have seen on previous projects has always advocated achieving business value as the number one priority for story selection. The author advocates a slightly different approach whereby we first tackle the highest risk or highest learning tasks first so that the team can gain this knowledge early while also being sure that it is technically possible to do the things they are being asked to do. This is linked to the idea of the walking skeleton – whereby we create an implementation of the system that performs a small end to end function. Clearly when using this approach it is important to still try and show that we are achieving some business value otherwise the project sponsor may become very nervous.
• The book has some interesting suggestions around retrospectives or reflection workshops. One which we are currently trying on my project is the idea of wrapping up the retrospective with two lists – ‘Keep These’ and ‘Try These’, which contain the ideas suggested in the retrospective. We can then prioritise the items on the ‘Try These’ list and go through the most important ones before the next retrospective.
• Another idea which was quite different from what I’ve come across was the idea of systems personalities. The idea here is that the system displays a different ‘personality’ to different end users. For example it might be fast and efficient or warm, friendly and informative. This seems like a good approach to allow us to drive out exactly how important different aspects of the system are and makes the trades off we inevitably have to make more related to our overall goal.
• The idea of having an expert user as well as a business expert working with the team was another interesting idea. From my experience it is very difficult to get a business expert to come and work with the team full time so I’m not sure how we would be able to get another person to come and work with our team, but several colleagues suggested to me that they do have an expert user working on their team so it is possible. I can see this being a useful role though because we often come across situations where there are two ways to do things and it’s not obvious which way the user would prefer. To have someone who actually uses the system available to answer that question would be quite valuable.
• There is an interesting discussion about the documentation that a project should look to supply – this is often something that we neglect so it was interesting to see explicit ideas on what should be provided being stated. It described the normal ideas such as the source code, architecture diagrams and so on, but one thing which I hadn’t considered as documentation was design diagrams showing the way the system has evolved. We often sketch out designs on whiteboards but rarely capture the evolution of them. It’s certainly an idea to keep in mind.
• I liked the case studies at the end of the book which detailed some experiences teams have had using the Crystal Clear approach. The most interesting part for me was looking at the individual experiences that each person on the team had. The ideas expressed weren’t new to me but it was interesting that they were so similar despite the project being described being so different to the ones I have worked on.

In Summary

The biggest idea I came away with after reading this book was the idea that we should always be looking to continuously improve. At one stage the book refers to Kaizen – a term from the Toyota Production System which means ‘continuous improvement’. As a result of reading this book I became more aware of this term and came across Kaizen Conf which took place in the US last week.

I think it would serve as a good introduction to the Agile world – it’s not quite as strict as something like Extreme Programming but it seems to get across most of the same ideas.

On my current project we’ve been having some discussions around the frequency with which we rotate pairs, the feeling being that we probably keep the same pairs for a bit too long.

We discussed using techniques such as promiscuous pairing, which takes the idea of pair rotation to an extreme, but have settled on making our rotations more or less daily.

One interesting thing I noticed from some recent pair switching was the immediate benefit we can realise from the pair rotation.

When we switch pairs mid story the story champion (person who owns the story from start to end) has to get their new pair up to speed as quickly as possible.

My colleague Liz Douglass has a very precise yet effective way of doing this. From observation:

• Describe the story or task from a high level detailing what it is we are trying to do and how it fits into the system as a whole
• Describe what has been done so far, why that approach was taken and what is left to do.
• Describe any problems encountered, previous solutions tried and new ideas to try to solve the problem.

I have found that this works really well and allows me to contribute to the task at hand more quickly than if I had to ask questions to work out what is going on.

The story champion has the benefit of having been involved in the story kick off so it is certainly very useful for them to be able to take this information and then provide it with context to their future pair.

In addition I have noticed that explaining what you are doing on a task to a new pair often leads to you noticing flaws in your logic, therefore leading you to solve the problem.

Even if this doesn’t happen, having a new pair of eyes and a new perspective on a problem can often lead to it being solved more quickly.

In order to experience the full benefits of pair programming it is important to try and reduce the chance of the navigator getting bored and losing focus.

One of the main ways that we can do this is by ensuring that we have a quick turnaround between the driver and navigator, and this can be done by ensuring that when we are driving we are doing so as quickly as possible.

There are three areas that come to mind where we can gain speed improvements when driving in a pair programming session:

IDE shortcuts

Learning the IntelliJ shortcuts was one of the first things that I was encouraged to do on the first Java project that I worked on at ThoughtWorks.

I found the most effective way for me was if my pair pointed out potential shortcuts whenever I was using the mouse to try and do something. On my current project we have installed an IntelliJ plugin called Key Promoter which pops up a message telling you the shortcuts every time you use one of the menu options. I learnt about this plugin from Neal Ford’s book, The Productive Programmer.

Additionally there are PDFs available, which list every single shortcut available for the various different platforms, on the IntelliJ documentation page.

The same applies whether we are using TextMate, Visual Studio + Resharper or any other IDE. Knowing the shortcuts allows you to execute tasks more quickly and (for me at least) also has the benefit of making you feel more confident about what you are doing.

IDE templates/Plug ins

Setting up templates for frequent operations is another useful way to save time.

In particular I find it especially wasteful to write out the code to define a JUnit test every time so I always create a template which does this for me.

IntelliJ also provides a set of built in templates for creating simple code snippets – accessible by using the Command + J keyboard shortcut on the Mac or Ctrl + J on Windows.

In addition we can install plugins which help us to be more productive. TestDox is one such plug in for IntelliJ which allows you to create the test class for any class with one keyboard shortcut. We can then switch between the test and the code using the same shortcut.

For Ruby, RubyAmp is a TextMate plugin which I have come across which adds class, method and whole project searching for Ruby code.

The Shell

Using the shell or command line effectively is another way that we can can speed improvements.

I wrote about some of the ways I have learnt for doing this and several people provided other suggestions in the comments on the post, but the key pattern here seems to be to reduce the amount of typing and avoid repetition. It’s all about reducing duplication of effort when it comes to using the shell effectively.

Often when using the shell we want to reuse some of the commands that we have entered previously.

I was aware of two ways of doing this until last week – using the up and down arrows to go through the history and searching through the history using Ctrl + R.

A third way I learnt last week was to type the following command into the shell:

history

This returns the list of items that we have previously searched for:

 533  man grep
 534  grep -r "JarTask" *.*
 535  grep -r "JarTask" .
 536  cd java/

A colleague showed me that we can repeat these commands by typing ‘![NumberOfCommand]‘. To repeat the ‘man grep’ command we would type the following:

!533

Spotting patterns of repetition is also useful. For example, to redeploy our application for manual CSS testing we had several steps which involved stopping Tomcat, redeploying our WAR, updating the database and then restarting the database.

This involved 4 separate commands until one of my colleagues put it all together into a simple ‘restart’ script. Quite a simple idea but it reduces human effort.

Overall

These are some of the ways that I have noticed where we can make pair programming go more smoothly and keep both people on the pair engaged.

The common theme is that we should make our tools do the work and save our time for thinking.

I’ve spent the last few days working with CSS, and in particular trying to make a layout which works perfectly fine in Firefox work properly in Internet Explorer 6.

I’m far from an expert when it comes to this but I’ve picked up a few lessons from our attempts to get identical layouts in both browsers.

• Internet Explorer seems to do some crazy stuff when it comes to padding and margins – we were often ending up with huge margins where we hadn’t even specified any. A useful approach pointed out by Josh was resetting the margin and padding for the whole page so that IE’s default padding was removed.

  Putting the following code in your style sheet will do this:

  * {padding:0;margin:0}

• Despite the fact that we start with IE and Firefox on an even keel, there were still differences in how they rendered some spacings that we tried to apply. I don’t think this is necessarily a good long term fix but it seems that if you start a property name with an underscore IE will respect it but Firefox will ignore it. This is known as the MSIE underscore hack, which describes it fairly accurately! We can therefore set up IE only CSS properties like this:

  _margin: 0 0 2px 0;

• One of the other problems we came across was the hidden text bug – sometimes when we refreshed the page text inside a floating div would just disappear until you selected it with the mouse when it would reappear. We found a couple of posts which explained that the problem was being caused by IE not correctly setting the element size for floating elements on the page.

  The other posts go into more detail, but the fix that worked for us involved forcing IE into layout mode by applying a small height to the element that was being hidden:

  .visualIEFloatFix {
      height: 0.01%; 
  }

I’m sure there will be further learnings on my CSS travels – I’ve been trying to find some resources which describe good patterns to use with CSS but I’ve only come across a book called Pro CSS and HTML Design Patterns so far.

Anyone know if this book is any good or if not what a better alternative is?