Mark Needham

Despite the fact that none of the projects that I've worked on have used Git or Mercurial as the team's main repository I keep forgetting how useful those tools can be even if they're just being used locally.

I ran into a problem when trying to work out why a Rhino Mocks expectation wasn't working as I expected last week having refactored a bit of code to include a constructor.

I wanted to include the Rhino Mocks source code in our solution before and after the refactoring and step through the code to see what was different in the way the expectations were being setup.

My initial thought was that I could just check out the repository again in another folder and then include the Rhino Mocks source code there and step through it but unfortunately we have all the projects set up to deploy to IIS so Visual Studio wanted me to adjust all those settings in order to load the solution in the new checkout location.

I probably could have gone and turned off that setting but it seemed a bit too much effort and I realised that I could easily use Git to help me solve the problem.

I took a patch of the changes I'd made and then reverted the code before checking it into a local Git repository.

I updated the solution to include the Rhino Mocks code and then created a branch called 'refactoringChanges' so that I could then apply the patch that I'd created with my changes.

It was then really easy to switch back between the two branches and see the differences in the way that the Rhino Mocks was working internally.

The actual problem eventually turned out to be the way that the code calls Castle DynamicProxy but I didn't get the chance to look further into it – we had learnt enough to know how we could get around the problem.

I'm in the process of including the source code for all the 3rd party libraries that we use in the solution on a separate Git branch that I can switch to when I want to debug through that code.

Sometimes I end up having to close down Visual Studio and re-open the solution when I switch to and from that branch but apart from that it seems to work reasonably well so far.

Alan Atlas has a recent blog post where he discusses agile, lean and scrum and suggests that you can't truly achieve agility unless your company is fully committed to it which differs slightly from my experiences.

Alan makes a valid point that we're not really following an approach just because we use all the practices:

Many people make the mistake of viewing Scrum and Agile and Lean as sets of practices. “If I do kanban, I’m Lean.” “If I do sprints, I’m scrummy.”

It's quite easy to put our work up on a story wall, run a daily standup and unit test our code and not really get the full benefit that those practices are supposed to give us.

As I've mentioned before though I think it is part of the learning process that we start out focusing on practices and not principles.

However, at some stage we need to start thinking for ourselves about the value those are giving and making adjustments to them if we're not seeing much value i.e. we need to look beyond the practice and to the principle or outcome that we are trying to achieve by using that practice.

The part of the post which is different to my experiences is this bit:

So I smile to myself (I think it’s to myself) when I hear management say things like “We’ll never be totally Agile in this company. There will always be waterfall projects.” I smile because what that really means is that they’ll never be Agile at all.

I haven't heard conversations like this before but I think it is still possible to deliver software in a team working in an 'agile' way even if the rest of the organisation which that team operates in follows a different approach.

It won't be as smooth sailing as if the whole organisation buys into the lean/agile approach and there will still be some reporting and bureaucracy that you need to deal with but it's not a lost cause.

My colleague Lindy Stephens and a couple of others covered some of the issues around project governance in the enterprise in a ThoughtWorks QTB in Sydney last year. The slides from that presentation are available on slideshare.

If your company is not fully committed to agility then you won’t achieve it, and your results will be a self-fulfilling prophecy of unrealized potential.

Even if an organisation decides that it wants to be agile I think it still takes time to get used to the approach and it always seems to take a few successful deliveries to see that some of the worries that heavy weight processes and paperwork try to protect you against are not necessarily valid.

I've spent a lot of time being indignant that people didn't buy into the agile approach but the more I spend time in different organisations the more it becomes clear that even for the people with the best intentions in wanting to learn this approach it will still take time to get there.

I'm coming to the conclusion that it's a good thing that people have some level of skepticism because it forces you to really understand why an agile approach is more effective and then try and persuade other people of that.

It's also helpful to note that it makes sense to vary our approach depending on the context that we're operating in. For example if the team is distributed across different cities then we might have more written documentation than with a co-located team.

It's very rare that an organisation or group of people will just 'get it' straight away – in many ways it's quite an iterative/incremental journey.

I recently came across 37 Signals 'Getting Real' book where they go through their approach to building web applications and there have certainly been some good reminders and ideas on the best way to do this.

These are some of my favourite parts:

• Ship it!
  

  If there are minor bugs, ship it as soon you have the core scenarios nailed and ship the bug fixes to web gradually after that. The faster you get the user feedback the better.

  Often on projects I've worked on we've taken the approach that bugs get worked on before new stories which makes sense in a way because it means that we are fixing problems quickly and keeping the quality of the application high.

  In reality what often happens is that low priority bugs just end up not getting looked at but I like the fact that we can choose to make that an explicit approach rather than just allowing it to happen to us.

  Prioritize your bugs (and even ignore some of them)

  Just because you discover a bug in your product, doesn’t mean it’s time to panic. All software has bugs – it’s just a fact of life.

  I find it interesting that there might be more value in getting something out the door and then getting feedback on it rather than spending extra time perfecting it up front.

• Fix Time and Budget, Flex Scope
  

  You have to figure out what’s really important. What’s going to make it into this initial release? This forces a constraint on you which will push you to make tough decisions instead of hemming and hawing.

  From my experience a lot of times we end up implementing features just because that's what was agreed in the initial release plan and there is often a reluctance to change that even if a feature isn't really that useful anymore.

  It becomes even more problematic if we get to the stage where it's not possible to deliver all the features promised in the remaining time so it certainly makes sense to me that in that situation we would look to focus on getting the absolutely essential things in first.

• Choose any enemy
  

  Sometimes the best way to know what your app should be is to know what it shouldn’t be. Figure out your app’s enemy and you’ll shine a light on where you need to go.

  This seems to be a much better idea than just copying the ideas of your competitor which might seem the obvious thing to do if you're working in the same area.

  The problem with that approach of course is that when you do copy you have no actual vision of what you're doing with your application anyway so you'll always be playing catch up.

• Don't overcomplicate the application

  There are a few parts of the book where the authors talk about keeping the application simple and then letting the users play with it:

  The harder we tighten things down, the less room there is for a creative, emergent solution. Whether it’s locking down requirements before they are well understood or prematurely optimizing code, or inventing complex navigation and workflow scenarios before letting end users play with the system, the result is the same: an overly complicated, stupid system instead of a clean, elegant system that harnesses emergence.

  Keep it small. Keep it simple. Let it happen.

  Andrew Hunt, The Pragmatic Programmers

  The users can then decide for us where we need to fill in more details:

  Details reveal themselves as you use what you’re building. You’ll see what needs more attention. You’ll feel what’s missing. You’ll know which potholes to pave over because you’ll keep hitting them. That’s when you need to pay attention, not sooner.

  In particular they suggest that focusing on very specific details about the page layout/colour/wording can be left until later because it will only serve to hinder forward progress if we concentrate on it too early.

• Scaling an application
  

  You don’t have a scaling problem yet

  “Will my app scale when millions of people start using it?”

  Ya know what? Wait until that actually happens.

  On several projects that I've worked on there often seems to be a desire to focus on performance and scaling an application very early on which seems wasteful when we could be focusing on actually building something that has so many users that we need to scale it later on. I think this advice is spot on.

• Write less software

  A common theme throughout the book is that of writing less software to achieve our goals:

  The best designers and the best programmers…are the ones that can determine what just doesn’t matter.

  That’s where the real gains are made.

  Most of the time you spend is wasted on things that just don’t matter. If you can cut out the work and thinking that just don’t matter, you’ll achieve productivity you’ve never imagined.

  Innovation is not about saying yes to everything. It’s about saying NO to all but the most crucial features.

  Throw away customer feature requests – if they're really important then they'll come back anyway

  Don’t worry about tracking and saving each request that comes in. Let your customers be your memory. If it’s really worth remembering, they’ll remind you until you can’t forget.

  The authors ideas around preferences were particularly interesting to me:

  Preferences are also evil because they create more software.

  More options require more code. And there’s all the extra testing and designing you need to do too.

  I hadn't appreciated until recently quite how much complexity we can add to an application by allowing users to play around with the display of information on a screen.

  It seems like a nice feature to have but it would be interesting to see statistics that could tell us what percentage of users actually that type of thing when it's not the core idea of the application.

  I also quite liked the following and I think it's something that we need to do more often on teams:

  Encourage programmers to make counteroffers.You want to hear: “The way you suggested will take 12 hours. But there’s a way I can do it that will only take one hour. It won’t do x but it will do y.” Let the software push back. Tell programmers to fight for what they think is the best way.

• Decisions are temporary so make the call and move on
  

  So don’t do the “paralysis through analysis” thing. That only slows progress and saps morale.

  Instead, value the importance of moving on and moving forward. Get in the rhythm of making decisions. Make a quick, simple call and then go back and change that decision if it doesn’t work out.

  I think a big part of this is getting the mentality that it's fine to make changes after we've 'finished' something. Any other approach doesn't work from my experience.

• Reduce meetings
  

  Meetings usually arise when a concept isn’t clear enough. Instead of resorting to a meeting, try to simplify the concept so you can discuss it quickly via email or im or Campfire.

  I find it interesting that they prefer communicating by email because I've often found that it's not the best communication mechanism since it's really easy to misinterpret what people mean.

  Having said that if we can make concepts clearer and the need for a meeting is an indicator that we need to do that then perhaps we can still meet in person and just make the meeting much shorter.

• Design the interface before you start programming
  

  Too many apps start with a program-first mentality. That’s a bad idea. Programming is the heaviest component of building an app, meaning it’s the most expensive and hardest to change. Instead, start by designing first.

  I've certainly fallen into this trap a lot but I've been trying to follow the outside in approach more strictly recently and so far I'm finding that it reduces the feedback cycle quite substantially which is only a good thing.

• Design for regular, blank, and error states
  

  For each screen, you need to consider three possible states:

  Regular
  The screen people see when everything’s working fine and your app is flush with data.

  Blank
  The screen people see when using the app for the first time, before data is entered.

  Error
  The screen people see when something goes wrong.

  I'd never even though of this at all and I'm certainly guilty of only ever considering applications when all the data is filled in so this is certainly something else to consider.

• Tear down the walls between support and development
  

  In the restaurant business, there’s a world of difference between those working in the kitchen and those out front who deal with customers. It’s important for both sides to understand and empathize with the other. That’s why cooking schools and restaurants will often have chefs work out front as waiters so the kitchen staff can interact with customers and see what it’s actually like on the front lines.

  My colleague Chris Read and some others seem to be trying to close this gap with the devops movement which also has a track at QCon London this week.

  The idea of working in support to see what an application is like from that perspective is something that more experienced colleagues often recommend although I've not done it as yet.

Overall I found this book a really interesting and quick read and although many of the ideas suggested seem like common sense it's strange that we often don't do all of them.

The 37 Signals guys also have a new book coming out in the UK tomorrow titled 'Rework' which sounds like it could be quite a good read as well.

I had an interesting conversation with my colleague David Santoro last week where I described the way that I often pick out the riskiest parts of a story or task and do those first and David pointed out that this approach didn't seem to fit in with the idea of outside in development.

The idea with outside in development as I understand it is that we would look to drive any new functionality from the UI i.e. the outside and work our way inwards through the various layers and probably eventually end up with persistence i.e. the inside.

In the particular example that were basing our discussion on I described a story that I was working on with my pair where we needed to apply some constraints to certain items in our data set and then display them differently on the UI as a result of that.

We went through the existing domain model to see if there was anything in there that we could make use of, and having realised that there wasn't anything we mapped out the tasks we would need to do to implement this functionality.

The most difficult/tricky task was to handle the data migration as we realised that we would need to add an extra column to one of the tables. We also needed to do the associated Hibernate mapping and the code from the UI and controller level downwards to make use of this.

We did the data migration and associated work first and one we were happy that was working we went and coded from the UI downwards until we reached the persistence layer.

In this situation it seemed like it worked out reasonably well and I couldn't see that we would have ended up with a different solution if we had started off driving through the UI.

In fact we did end up spending most of our time doing the data migration so to me it seemed somewhat justified doing that first since we did run into a couple of problems.

Outside in development in general seems a good thing to me so I'm curious as to whether I'm justifying a sub optimal approach to myself or whether there are some situations where we can vary the approach a bit?

Reading through some of Simon Harris' blog entries I came across his thoughts on a presentation Eric Evans did at QCon titled 'Strategic Design – Responsibility Traps' which seems to cover a lot of the ground from the second half of Domain Driven Design and more.

In the presentation Evans make some really insightful comments and points out a lot of mistakes that I've made on projects. It certainly serves as a reminder to go back and read part 4 of the book again and really understand the material from that section.

These were the most interesting observations for me:

• In this talk he makes some similar points to those that he made in his 'What I've learned about DDD since the book' presentation that I had the chance to see at QCon in London last year. One of these is that there is no such thing as a "right model" – there are only models and some will help us describe our system better than others.
• Evans suggests that we need to spend most of our time modeling in the core domain since this is the code that gives us a competitive advantage and allows us to differentiate ourselves from competitors.

  I often wonder where the best places to focus efforts on code bases is and I've typically been of the opinion that the pain points are the best place to work on since we can see an immediate reward from doing this. As I've mentioned before I quite like Fabio's technical debt quadrant as a mechanism for measuring where we should focus our efforts with this approach.

  It still seems different to what Evans suggests although I'm inclined to believe that the areas of most pain could well be the areas we need to be subtle to change and there could therefore be some correlation between those areas and the core domain.

  I've not seen a distinction between the core domain and other parts of the domain model on any projects I've worked on so it'd be interesting to hear other opinions on this.

• A related point to this which I haven't completely grasped is that there are some areas of the code which we shouldn't bother trying to improve and instead should just work around and not worry too much about creating intricate models.

  I like this advice in a way although it seems a little dangerous to me and perhaps seems to conflict with Uncle Bob's idea of following the boy scout rule and improving the code slightly every time we touch it. As I understand it, Evans advice would be to only follow this advice when we're working in the core domain.

• Evans points out that we should try to avoid the universal domain model, something which Dan North also points out in his article on SOA. As I see it we can either decide to explicitly mark out multiple different models in our code explicitly otherwise we'll just end up with one mediocre model being bent to fit the needs of every part of the system.

  I guess it seems intuitive to try and reduce the amount of code required in a system by just doing the modeling once but different teams in different contexts have different meanings and uses for the model that it doesn't make sense as an approach.

• My favourite quote from the presentation is the following:
  

  Be truly responsible, don't just satisfy the emotional need to be responsible.

  I really like refactoring code so this is a good reminder to me to take a step back when I find myself doing that and consider whether what I'm doing is actually useful.

  Evans suggests that it's pointless being the team janitor and cleaning up the code after everyone else has rushed to get features delivered. The suggestion seems to be to get the strongest developers on the team working on the most important domain code rather than creating the infrastructure/platform for the rest of the team to work from.

At the moment I feel like watching this presentation has made me think more about the value of what I'm doing when working on a code base. I don't feel so inclined to randomly refactor code and I'm more keen to work out which bits of the system I'm working on would benefit from this kind of attention.

As I mentioned earlier I now need to finish off reading section 4 of the big blue book!

Cosmin Stejerean linked to a really interesting article on wired.com which tells the story of how Duke Nukem failed over 12 years to ship their latest game, eventually giving up.

Phil has written a post about the article from the angle of his experience working with these types of companies and working out how to get something into production but as I read this article it seemed to have some relation to reworking code and why/how we approach this.

It can be reworking of code through either rewriting or refactoring, but the general idea is that it's not directly contributing to getting something released.

One particular bit of the article stood out to me as being particularly interesting – it describes how they decided to change from the Quake II game engine to the Unreal one:

One evening just after E3, while the team sat together, a programmer threw out a bombshell: Maybe they should switch to Unreal? “The room got quiet for a moment,” Broussard recalled. Switching engines again seemed insane — it would cost another massive wad of money and require them to scrap much of the work they’d done.

But Broussard decided to make the change. Only weeks after he showed off Duke Nukem Forever, he stunned the gaming industry by announcing the shift to the Unreal engine. “It was effectively a reboot of the project in many respects,”

What they effectively did here is to rip out a core bit of the architecture and totally change it which would be quite a difficult decision to make if you knew you had to deliver by a certain date.

We've made that decision on projects that I've worked on but you have need to come up with a very compelling argument to do so i.e. typically that productivity will be much improved by making the change.

Whenever we talked about refactoring code during our technical book club Dave always pointed out that refactoring for the sake of doing so is pointless which to an extent explains why it makes most sense to refactor either around the code that we're currently working on or in the areas that are causing us most pain.

For me the goal of refactoring code is to make it easier to work with or easier to change but it's useful to remember that we're refactoring to help us reach another goal.

An idea which I quite like (suggested by Danilo) but haven't tried yet is running technical retrospectives more frequently so that we can work out which areas of the code we need to work out for our current release and then make use of Stuart Caborn's bowling card idea to keep track of how much effort we've spent on these problems.

It seems like any refactorings we decide to do need to be linked to a common vision which we're trying to achieve and that seems to be where Duke Nukem went wrong. The vision of what was actually required for the game to be successful was lost and as many features as possible were added in.

The Duke Nukem approach seems quite similar to going through the code and making refactorings just to make the code 'better' even though we might not see any return from doing so.

In Debug It Paul Butcher suggests that we need to approach bugs in software with pragmatic zero tolerance by realising that while our aim is to have no bugs we need to keep sight of our ultimate goal while doing so.

I think we can apply the same rule when reworking code. We should look to write good code which is well factored and easy to change but realise that we'll never be able to write perfect code and we shouldn't beat ourselves up about it.

I'm reading through Paul Butcher's 'Debug It' book and one of his suggestions when trying to diagnose a problem in our code is to only change one thing at a time.

In a way this might seem fairly obvious but I've certainly fallen into the trap of making multiple changes at the same time in the misled belief that it'll lead to the problem being solved more quickly.

When making changes to code Butcher has the following piece of advice which I quite like:

Once you see a change in behavior, undo whatever apparently caused it, and verify that the behavior retur ns to what it was before-hand. This is a very power ful indication that you’re looking at cause and effect rather than serendipity.

I noticed this while debugging my F# word count application. As I mentioned, I thought that the problem was that I was storing the text from all the files in memory so instead of doing that I made that part of the application lazy so that the text would only be loaded when required.

When I first did this the program still didn't work but it failed later on than it had previously.

I thought that had shown where the problem was so I put the code back to how it was previously to check.

To my surprise it still failed in the same place which meant that the change in how it executed had been coincidental rather than related to any code change I'd made.

I think this idea is more widely applicable though as I've noticed that it works quite well when refactoring as well. If we can be really certain about which changes work and which don't to a very fine grained level then we have more chance of successfully refactoring our code while ensuring that it still functions correctly.

The goal seems to be the same in both of these situations – take small steps and then get feedback quickly on how successful that small step was.

Another paper that I read on my Sydney to London flight was one titled 'You and Your Research' by Richard Hamming.

It's a transcript of a talk that Richard Hamming gave to Bellcore employees at the Morris Research and Engineering Centre in 1986.

The talk is aimed at computer science researchers and Hamming describes ways for them to do the best research that they can. I think several of the ideas in the talk relate to software development as well.

These were some of the bits I found interesting:

• Hamming makes an observation about working conditions which I thought was quite interesting:

  So ideal working conditions are very strange. The ones you want aren't always the best ones for you.

  From what I've noticed this is true in software development too.

  As a contrived example, the 'ideal' software project would have unlimited time and money but that's never the case so we have to find ways to deal with the fact that we have to work within that framework. One of the benefits of this is that we try only to develop the features which are the most important rather than absolutely everything which might be the case if we didn't have that constraint.

  I'm inclined to believe that when we're working in a difficult environment we have more freedom to try things out because what's currently being tried is probably not working anyway. This seems to be the type of situation where innovation can happen.

• Hamming seems to touch on the idea of 10,000 hours of practice to achieve expertise that J. Anders Ericcsson has researched and Malcolm Gladwell popularised:
  

  On this matter of drive Edison says, "Genius is 99% perspiration and 1% inspiration." He may have been exaggerating, but the idea is that solid work, steadily applied, gets you surprisingly far. The steady application of effort with a little bit more work, intelligently applied is what does it. That's the trouble; drive, misapplied, doesn't get you anywhere…the misapplication of effort is a very serious matter. Just hard work is not enough – it must be applied sensibly.

  This seems to cover the same type of ground that Geoff Colvin covers with the idea of deliberate practice where we should always look to practice something which is a bit beyond our current ability. That way we're always improving.

• I really liked the following part of the article:
  

  But most great scientists are well aware of why their theories are true and they are also well aware of some slight misfits which don't quite fit and they don't forget it. Darwin writes in his autobiography that he found it necessary to write down every piece of evidence which appeared to contradict his beliefs because otherwise they would disappear from his mind.

  I often forget that there are other ways of solving problems than the ways I currently use and it's always good to read about people being successful with different approaches. That way we can keep questioning what we're doing rather than just blindly doing so.

• He also talks about the idea of bouncing ideas of other people because they will get you to think about the idea in different ways:
  

  What you want to do is get that critical mass in action; "Yes, that reminds me of so and so," or, "Have you thought about that or this?" When you talk to other people, you want to get rid of those sound absorbers who are nice people but merely say, "Oh yes," and to find those who will stimulate you right back.

  I think this is where something like a technical book club can be invaluable. On every single paper or chapter we read in the ThoughtWorks Sydney book club others had different/better ideas than I did and it was really useful for showing me perspectives that I hadn't even thought of.

  In 'Pragmatic Learning and Thinking' Andy Hunt suggests that whenever we read something we should try and explain the idea to a peer to see how well we understand the material and to get other ideas that we hadn't thought of.

  I'd really recommend this approach.

• There's also some interesting advice about giving presentations:
  

  The technical person wants to give a highly limited technical talk…the audience wants a broad general talk and wants much more background than the speaker is willing to give. As a result, many talks are ineffective.

  …

  You should paint a general picture to say why it's important, and then slowly give a sketch of what was done…the tendency is to give a highly restricted, safe talk; this is usually ineffective. Furthermore, many talks are filled with far too much information. So I say this idea of selling is obvious.

  This is very similar to the advice Dan North gives. He suggests that three ideas/new things in a talk is all that people will be able to remember and anything more than this is a bit overwhelming.

There's certainly other interesting ideas in this paper but those were some of the bits that stood out for me. Worth reading.

I've come across a couple of posts recently talking about the gender specificness of the term 'Software Craftsman' and Victoria suggests that the term 'Codesmith' would be a more appropriate name to use.

I'm not that bothered what the name is but I was reading the transcript of Fred Brooks' acceptance speech for winning the ACM Allen Newell Award in 1994 titled 'The Computer Scientist as Toolsmith' which has some interesting ideas about what our role should be.

These were some of the parts that I found interesting in the talk:

• I quite liked the following quote and it seems to cover the same type of ground that we try to cover with the agile approach to software development with respect to writing software that actually provides value to our users:

  If we perceive our role aright, we then see more clearly the proper criterion for success: a toolmaker succeeds as, and only as, the users of his tool succeed with his aid.

  The article goes on to say the following:

  …we tend to forget our users and their real problems, climbing into our ivory towers to dissect tractable abstractions of these problems , abstractions that may have left behind the essence of the real problem.

  I also came across an interesting blog post by Rob Bowley where he discusses some of the limitations he's noticed in the agile approach with respect to addressing the needs of our customers which seems to cover similar ground.

• He also makes some interesting points around interdisciplinary collaboration:
  

  There are real costs associated with any professional collaboration, and interdisciplinary collaborations have some unique costs. I find that our teams spend about a quarter of our professional effort on routine work that supports our collaborators

  I'm not sure whether that last statistic stands true for collaboration between software development teams and the business but a fairly common objection from the business is that they don't have the time to interact with the software guys and that we should just get on build what they want.

  Rob's post seems to suggest that we're not collaborating in a particularly effective way and Brooks goes on to suggest that we need to do some preparation before interacting with these guys:

  Our Ph.D. students often take introductory courses in the using disciplines, and they always take reading courses from our collaborators to prepare them for their dissertation work. One need not become an expert in the partner's field, of course, but one does need to learn the basic principles, the vocabulary, and the partner's research objectives.

  I wonder if this is where we sometimes go wrong – we're focused on the software solution rather than stepping back and working out our customers' real problems and helping them solve those.

  This part of the article also reminded me of comments made by Eric Evans in his QCon talk about not wasting domain experts time.

• In one part of the article Brooks talks about artificial intelligence, suggesting:
  

  …intelligence amplifying systems can, at any given level of available systems technology, beat AI systems. That is, a machine and a mind can beat a mind-imitating machine working by itself

  This seems quite similar to an idea that I read in Taaichi Ohno's Workplace Management whereby we look to automate processes but not just for the sake of automation. We should automate so that the human can do their job more effectively.

  On the projects I've worked on we often make use of automation to provide us with code metrics but a human would then analyse those and work out whether we need to make any changes to the way that we do things based on those metrics.

  Google seem to be going with an even more automated approach with respect to understanding which tests are useful as Marcus Striebeck described in a talk at XP Day and since it seems to be working well for them, perhaps we haven't yet worked out where the usefulness of a machine ends and a human is required.

Oredev have put some of the videos from the conference on Vimeo and one of my favourites is 'Our obsession with efficiency' by my colleague Dan North.

The slides for the talk are available on SlideShare.

In this talk Dan leads from the following statement about efficiency:

So here's the thing, I don't believe in efficiency. It's our obsession with efficiency that has got us into the current technology mess, and which has led almost directly to heavy waterfall processes. Efficiency is how you let the big vendors sell their bloated technologies to the poor CIOs.

What did I learn?

• Dan spends quite a bit of time explaining how what we should really care about is effectiveness and not efficiency. Efficiency is defined as:
  

  the accomplishment of or ability to accomplish a job with a minimum expenditure of time and effort

  which makes sense in a way but what tends to happen is that achieving that becomes our goal at the expense of everything else. For example adhering to the DRY principle is considered a good thing and not repeating code is efficient. However, if we take that to an extreme then it results in code that is difficult to understand and difficult to change which defeats the purpose of that efficiency.

  In lean terms we want to look to favour the big picture over local optimisations whereby we start to measure our effectiveness rather than efficiency. i.e. we focus on the results rather than the effort.

• Later on he points out that effectiveness is often inefficient which makes a lot of sense to me.

  Pair programming is not an efficient way to get the most code produced – we can do that much more efficiently if we have people working individually. However it allows us to create a much greater shared understanding of the code, reduce the defects and increase the cohesion of that code which is a more important goal overall. The same can be said for something like set based concurrent engineering where we work on two solutions simultaneously and then throw one away. It's inefficient but we can delay potentially making the wrong decision so it makes sense to do it.

• Dan also suggests that 'you get what you measure' and lists a series of examples where aiming for a certain target is actually not very effective and doesn't give us what we want anyway.

  For example if our goal is to have all the tests passing by the end of the month then we may be tempted to comment out failing tests to achieve this. One which I notice quite frequently is aiming for a story point total which tends to result in reduced communication between business and IT and we end up delivering features that may not have that much value. It might seem to be locally efficient but in the grand scheme of things it's not efficient at all.

• There's also some discussion around the desire for people to be 'busy' all the time which is quite common from my experience. Dan points out that if we can get to a stage where we have time when we're not busy then we have more time to think about what we're doing and perhaps we can even spend this time innovating. I think the key here is to ensure that we're still doing something which is contributing to the overall system goal rather than not doing nothing at all!

There's loads more – it's a very good talk – but these are some of the bits that stand out for me.