Mark Needham

I recently came across an interesting post about pair programming by Paritosh Ranjan where he outlines some of the problems he’s experienced with this practice.

While some of the points that he raises are certainly valid I think they’re more evidence of pair programming not being done in an effective way rather than a problem with the idea in itself.

To take one example:

Generally people don’t think a lot while pair programming as the person who wants to think about the pros and cons will be considered inefficient (as he will slow down the coding speed). So, generally people show fake confidence on the effectiveness of the proposed solution.

While it’s certainly possible to end up with this scenario, I’ve also taken part in pairing sessions where we were able to think through a problem and come up with a design much more effectively than either of us would have been able to alone.

Something that I’ve noticed that I do too frequently is seeing a practice being executed in a sub optimal way and coming to the conclusion that there’s a problem with the practice itself.

For example I wrote a post about a month ago outlining some of the problems that we’d been having with retrospectives on some of the projects that I’ve been working on and at one stage towards the beginning of the year I was wondering whether there was even a lot of value in having a retrospective.

What was pointed out in the comments of that post and subsequently on threads on our internal mailing list is that to keep people engaged we should vary the way we run the retrospective rather than running the same format every time.

I’m told Esther Derby and Diana Larsen’s ‘Agile Retrospectives‘ has a lot of ideas on how to run retrospectives more effectively but I haven’t quite got around to reading it just yet.

Another practice which I’ve been doubting is the Iteration Kick Off meeting which often seems to be a session where we read through every single story that’s coming up while the majority of the people in the room are completely disengaged. These meetings often dragged on for an hour or more.

Discussing this with a business analyst colleague last week he pointed out that he runs these meetings in a completely different way. His goal is to communicate what functionality is coming up so that any coding decisions can be made with that in mind.

That communication doesn’t necessarily have to be in a meeting, it could just be a conversation had over lunch. If it is a meeting then he’d look to keep it short and to the point.

The underlying trend behind all of these is that we saw a practice being done in a sub optimal way and came to the conclusion that there must be a problem with the practice.

I’m coming to the conclusion that it would be more effective to look at the goal the practice is trying to achieve first and see if we can change the way we’re executing the practice to achieve what we want.

If not then perhaps the practice is at fault and we need to look for another way to achieve our goal.

Ken Schwaber recently wrote a blog post where he compared the differences between the kanban, lean and scrum approaches to software development and although I haven’t had the same experiences as he has with the first two, one interesting thing he implies is that with a scrum approach we have slack time built in.

God help us. People found ways to have slack in waterfall, to rest and be creative. With Lean and Kanban, those hiding places are removed. We now have a progressive death march without pause.

The project that I’m currently working on has switched emphasis from being in pure delivery mode into a combination of delivery and handover and it’s been quite noticeable how much more slack we have in the system as a result.

I find it much more enjoyable when there’s a bit of slack such that we’re not churning out story after story.

It gives you the opportunity to explore the code base a bit more and try out any ideas that you have to see if they’re likely to improve the productivity of the team.

An example of this on our project is that we’ve had the time to introduce a calculation descriptions DSL that my colleague Dermot had written in his own time.

This has helped reduce the number of tests required in certain areas of the code base as well as making the code more intuitive and therefore easy to understand.

It can be quite difficult to create this slack time when the team is under big deadline pressure and working on anything which isn’t directly related to hitting that deadline isn’t considered valuable.

I’ve noticed that the typical slack in the projects I’ve worked on tends to be towards the end of the day if we finish a story late on.

That tends to disappear if the team is working late and people start to become too tired to notice that there might be a better way to do things.

It seems to me that it would beneficial to try and work some slack time into projects to allow for the type of innovation that allows us to come up with ideas that allow us to be more productive.

I’ve worked on two different teams this year which had quite different approaches to retrospectives.

In the first team we had a retrospective at the beginning of every iteration i.e. once every two weeks and in the second team we tried out the idea of having a rolling retrospective i.e. we put up potential retrospective items on the wall and when there were enough of those we discussed them in the standup.

The problem we had with the first approach was that we often had the same types of discussions in each retrospective and the same types of issues were raised each week so that it seemed almost pointless to even have it in the first place.

Several people ended up disliking the idea of even doing a retrospective in the first place because of this.

We came up with the idea of having a vote at the beginning of the iteration to decide whether or not we should have one with the idea being that if it didn’t seem necessary then we could skip it for one week but would definitely then have the next one.

I think this worked reasonably well although it’s slightly different to the approach taken by Rolf Knutsen which he discussed at XP2010 in his session on ‘Feedback and Retrospectives in Agile Projects’.

Rolf suggested that we should have a retrospective at a given time interval and just cut it short if we don’t have anything else to discuss.

I think I prefer Rolf’s suggestion as it’s too easy to come to the conclusion that we don’t need a retrospective especially if we think that they’re a waste of time.

I thought the rolling retrospective approach that we took on the second project would work much better as we could just address any problems as soon as they came up but it hasn’t worked quite that way from my experience.

Several issues which would normally be addressed in a retrospective seemed to go by unchecked and without being resolved.

Having a retrospective is a very useful way to put people in a reflective mindset where they look for things that aren’t being done well or that could be done differently whereas during the day we might not do that.

I hadn’t appreciated this benefit of a retrospective session and had felt that problems were often being saved up for a retrospective even if they could have been addressed earlier.

Fabio previously wrote about the idea of the future retrospective box to ensure that we don’t forget anything that happened early on in an iteration when we come to a retrospective but that still doesn’t address the problem of only addressing problems in the retrospective!

It seems like a combination of having a pre scheduled retrospective with problem solving on a day to day basis is what we need so I’d be interested to know how others have addressed this.

I’ve previously written about the danger of using velocity as a goal but on almost every project I’ve worked on at some stage we do actually end up chasing a points total.

Something I find quite interesting towards the end of an iteration is that if there is a choice of two stories to pick up then the project manager will nearly always press for one which can be completed within the remaining time in order to get the points total for that iteration higher.

This might even mean that we pick something which is lower priority because we might not finish a higher priority but larger story in time.

This would mean that the points for that card would not be recognised until the next iteration which would mean that in the current iteration the points total would be lower than expected.

Most of the time this doesn’t make a great amount of difference and if it helps take some pressure off and create an impression of ‘success’ then it seems a reasonable trade off to make.

It does still seem less than ideal to have that type of decision dictated by such a limited metric though.

Dermot pointed me to a blog post by Tim Ross titled ‘Are burndowns evil?‘ where he discusses this in more detail and although I agree with the points Tim makes, it often seems that a product owner ends up with the impression that the project is ‘failing’ or that we are ‘behind’ if the velocity ‘target’ is not being met each iteration.

It seems to take product owners who are new to a more agile approach a bit of time to get used to the idea that achieving the points total isn’t the most important thing to focus on and that it’s more useful to focus on other things which actually give them value.

I’ve worked on projects where we’ve got to the stage where the points total is a side show but it takes quite a bit of time of consistently delivering before we can get that level of trust.

On the project I’m currently working on we have several stories around writing the code that does various different calculations based on user input and then shows the results on the screen.

The original assumption on these stories was that we would be looking up the data of the business rules from a local database. The data would be copied across from a central database into that one for this project.

After some discussion about one of the stories, Christian pointed out that the only reason that we needed to have that data in the database was so that it would be configurable in production without having to make code changes and redeploy the application.

I’d been working with the assumption that these stories could only be considered done when the data was coming from the database but we’ve extracted the ‘configurability of data’ out into another story which the business can choose to prioritise if that’s more valuable than other features we’re working on.

It didn’t even occur to me that we could pull this type of thing out but it seems like a useful approach as part of our drive to deliver the core functionality of the application as quickly as possible.

If we then need to go back and put in the configurability of the data, the lookup of that data is all hidden behind an interface so we shouldn’t see too much pain by choosing to delay doing that.

Grant Joung wrote a post a while ago about velocity goals and whether they’re a good or bad idea, a topic which seems to come up from time to time on agile teams.

My colleague Danilo Sato previously wrote about the dangers of using velocity as a performance measure because it’s something that’s directly within our control and can therefore be gamed:

Value should be measured at the highest level possible, so that it doesn’t fall into one team’s (or individual’s) span of control. People tend to behave according to how they’re measured and if this metric is easy to game, it will be gamed.
…
Velocity doesn’t satisfy my criteria for a good performance measure. Quite the opposite, it’s a very easy metric to game

Danilo suggests that we should look to use metrics which are outside of our immediate control but which we can score high on if we focus on doing a good job. He cites the ‘net promoter score’ (that measures how much your custumer is willing to recommend you to a friend) as an example of this.

Dan North gave a really good presentation titled ‘Our obsession with efficiency‘ where he covers similar ground and touches on the gaming of performance measures.

From my experience having velocity as a goal doesn’t make any difference to the motivation of the team which is often cited as the reason for referring to it as a target.

In all the teams I’ve worked on people are giving their best effort anyway so they can only really have an impact on the velocity by doing one of the following:

1. Working longer hours
2. Cutting corners on quality (by less testing perhaps)
3. Finding a smarter way of working

With the 1st idea we now have an inaccurate representation of how much work we can actually complete in a given time period so our future planning is now made more difficult unless we insist that people work long hours all the time which is a recipe for disaster.

With the 2nd we will eventually suffer when there are more defects later on in our process and we have to come back and re-work those bits of the system.

The 3rd is good but then again I don’t imagine that we would need to have a velocity goal in order to start doing that – we should be working smart by default.

I recently came across an interesting paper titled ‘Goals gone wild‘ which suggests that goal setting can actually be detrimental and that we should be more careful about how we use them:

The use of goal setting can degrade employee performance, shift focus away from important but nonspecified goals, harm interpersonal relationships, corrode organizational culture, and motivate risky and unethical behaviors. We argue that, in many situations, the damaging effects of goal setting outweigh its benefits.

Setting velocity as a goal is a prime example of what the authors call a narrow goal:

With goals, people narrow their focus. This intense focus can blind peopleto important issues that appear unrelated to their goal

As Danilo points out what we really want to do is to get some features that our users want to use into production with as few defects as possible so that they actually work.

Our underlying goal might therefore be to make the lives of our users easier through their use of the website we’re building.

This is much more difficult to measure which is perhaps why the number of points completed becomes the focus when in reality that’s not what anyone cares about. It seems more to signify a lack of trust between the two parties.

In reality I haven’t noticed that people on the teams I’ve worked on pay that much attention to whether velocity is considered a target or not. People just do their job and we pretty much always have the same velocity each week regardless.

My colleague J.K. has written an interesting blog post where he describes a slightly different approach that he’s been taking to writing stories to help move the business value in a story towards the beginning of the description and avoid detailing a solution in the ‘I want’ section of the story.

To summarise, J.K.’s current approach involves moving from the traditional story format of:

As I...
I want.. 
So that...

To the following:

As I... 
I want..
By...

I quite like this idea and I’ve noticed that even without using this story format technical solutions are sometimes described as the business requirement and we need to look beyond the ‘I want’ section of the story card to find the real value and locate assumptions which have led to the story being written in that way.

To give a recent example, a colleague and I picked up the following story:

As the business
I want a HTTP module to be included on the old site
So that I can redirect a small percentage of traffic to the new site

We assumed that other options for redirecting traffic must have already been analysed and written off in order for this to be the suggested solution so we initially started looking at how to implement it.

After a bit of investigation it became clear that this was going to be quite an invasive solution to the problem and would involve re-testing of the whole old site (since we would be making a change there) before it could be put into production. That would take 2 weeks.

Speaking with Toni and Ashok about the problem it became clear that it should be possible to control whether traffic was going to the old or new site by changing the configuration in our load balancer, Netscaler.

Discussing this further we found out that this had been tried previously and hadn’t quite worked out as expected which was why it hadn’t been considered as an option.

We spent some time talking through using Netscaler with the network team and agreed to try it out on a performance environment and see whether it would balance traffic in the way that we wanted which it did.

We still need to make sure that it works as expected in production but it was an interesting example of how solutions can be excluded based on prior experience even though they might still be useful to us.

I’ll certainly be more aware of noticing when a story details a solution and try and look for the actual requirement after this experience.

I’ve spent most of my time working on agile software development teams over the last few years so for the most part each pair is only working on one story, keeping the work in progress low and allowing them to focus on that piece of work until it’s completed.

My pair and I ended up in a therefore somewhat unusual situation last week where we were attempting to work on three things at the same time and weren’t doing a particularly great job on any of them.

It wasn’t immediately obvious to me that we were doing this since the two extra tasks that we were working on were related to deployment issues on different environments.

However we eventually started making mistakes and in rushing to rectify those made even more mistakes since we were still trying to concentrate on three different things.

It became much more obvious at this point that we needed to just pick one of the items and focus on that until we were done which also served as a reminder that it’s good to use the story board as an indicator of what everyone is working on.

In this case two of the tasks we were working on weren’t on the story board otherwise it would have been more obvious to the rest of the team that we shouldn’t have been working on two of them.

I’ve never really noticed the problems of context switching before so it was interesting to get such a stark example to remind me of its dangers.

Something which an agile approach on a project typically gives us is the ability to change requirements rapidly based on the different types of feedback we typically get over the course of the project.

One way that we can lose this advantage is by getting caught up by the number of story points being completed and using this as the measure of success.

The flexibility to change has an impact on the number of story points that may be completed in a given iteration – if we start doing some work on a story and then get feedback from the business while it is still in progress it’s possible that we will end up with more work to do than we had previously.

Once a story is estimated we don’t typically go back and change that estimate unless there was a change in the assumptions being made before we play it. This means that any changes we make based on feedback would not result in re-estimation of the story card.

We now have more work to do without our measurement mechanism recognising that.

It gets even worse if we start working on a feature and then get feedback from the business that the feature is no longer useful to them and we should just stop working on it.

At this point a fairly common response is to ‘lock down’ the requirements so that we’ll only work on the currently defined requirements and no new ones or changes will be permitted.

While this might make some logical sense it misses the point that we’re supposed to try and deliver something that is actually useful rather than something which just meets the initial requirements.

In reality one of two things, or perhaps a combination of both, ends up happening:

• There will still be changes and all we’ve done is create the illusion that there won’t be any.
• The business will now push to get exactly the features that were specified originally even if the development team ends up spending a disproportionate amount of time creating those features for the value they actually provide.

Story points are really useful for allowing us to get an idea of the relative size of stories and can be helpful for getting a rough understanding of the amount of work we can complete in a given time period but we need to be careful how we use them.

Our goal at the end of the day is to deliver working software.

Story points are just a mechanism for giving us some feedback on how well we are doing that. They’re not the goal in itself.

I recently finished reading Lean Thinking and one of the things that was repeatedly emphasised is the need to look at the process as a whole rather than trying to optimise each part individually.

If we phrased this in a similar way to the Agile Manifesto it would probably read ‘Big Picture over Local Optimisations‘.

The examples in Lean Thinking tend to be more manufacturing focused but I think this idea can certainly be applied in thinking about software projects too.

Individual vs Team Focus

For me on a software development team what we care about is the quality of the team as a whole not the individuals on it.

It’s all good being very strong technically but you have to be able to work effectively with other people to solve problems otherwise your value to a development team is much reduced.

I wrote a bit about this previously in my lean look at pair programming but I think it goes beyond just ensuring that we are sharing knowledge around the team by having people collaborating closely with each other.

When I first joined ThoughtWorks my former colleague Fred George was always talking about the value of being poly skilled – having the ability to carry out more than one role – and I think this is lost when we refer to people as being their role i.e. ‘you ARE a developer’ rather than ‘you CAN PLAY the developer role’.

If we have people on a team who are polyskilled or generalising specialist (although I think generalising specialist refers more to poly skilled across a role rather than being able to carry out multiple roles) then we can get away with building teams which are smaller in number and which are more resilient and able to respond when team members are ill or absent.

Horizonal vs Vertical teams

Creating teams of people by the layer or horizontal which they work on rather than the piece of functionality or vertical which they are working on seems to be one of the most obvious ways of locally optimising instead of looking at the big picture i.e. the delivery of a piece of functionality to the business.

I guess the theory is that if we put everyone working on similar things on the same layer in the same team then it will result in greater productivity than having people who are working on different areas of the system but have the same underlying goals.

The problem is that we create a lot of handover points – opportunities for context and information to be lost – and the ineffectiveness of this approach as a communication mechanism means that it takes much longer to integrate the components from each horizontal than it would have done with a team with people from each layer in it.

In addition any techniques used to try and increase the productivity of an individual team will be ineffective since they will only satisfy a local optima and will have a knock on effect to the rest of the system.

For example, let’s say we have three teams A, B and C and team B aren’t as productive as we were expecting. Applying pressure to team B may have an impact on their immediate productivity but it’s likely to affect the other two teams since team B will probably be less communicative towards those teams since they are trying to optimise their own performance.

Having a vertical team which is responsible for the delivery of a feature or features end to end works much better from my experience.

Colocated vs Distributed teams

One of my favourite agile ideas is having a co-located team, with the business people, analysts and developers all working in the same physical location.

Sometimes though the decision is taken to distribute teams across multiple locations – be it because that’s more convenient or cheaper to do.

The big picture i.e. project delivery is likely to become more difficult to achieve due to the increased cost of communication that has been created.

For me there is nothing better than face to face communication – even with phone calls/instant messaging/emailing it is still an extremely valuable form of communication. With any of the other three it’s much more difficult to read what someone is really thinking and you can come across as being much more aggressive than you intended to in any of those mediums.

My colleagues in India/China will certainly have some tools/techniques to bridge these gaps that I’m not aware of but I still remain convinced that if we have the choice then a co-located team is the best choice.

In Summary

These are just some of the ideas that came to mind when trying to apply one of the principles of lean to software development teams. Although many of these may seem obvious sometimes it takes another angle of looking at the problem to make that visible.

If you can think of any others feel free to mention them in the comments.