Our obsession with efficiency - Dan North
Oredev have put some of the videos from the conference on Vimeo and one of my favourites is 'Our obsession with efficiency' by my colleague Dan North.
The slides for the talk are available on SlideShare.
In this talk Dan leads from the following statement about efficiency:
So here’s the thing, I don’t believe in efficiency. It’s our obsession with efficiency that has got us into the current technology mess, and which has led almost directly to heavy waterfall processes. Efficiency is how you let the big vendors sell their bloated technologies to the poor CIOs.
What did I learn?
-
Dan spends quite a bit of time explaining how what we should really care about is effectiveness and not efficiency. Efficiency is defined as:
the accomplishment of or ability to accomplish a job with a minimum expenditure of time and effort
which makes sense in a way but what tends to happen is that achieving that becomes our goal at the expense of everything else. For example adhering to the DRY principle is considered a good thing and not repeating code is efficient. However, if we take that to an extreme then it results in code that is difficult to understand and difficult to change which defeats the purpose of that efficiency. In lean terms we want to look to favour the big picture over local optimisations whereby we start to measure our effectiveness rather than efficiency. i.e. we focus on the results rather than the effort.
-
Later on he points out that effectiveness is often inefficient which makes a lot of sense to me. Pair programming is not an efficient way to get the most code produced - we can do that much more efficiently if we have people working individually. However it allows us to create a much greater shared understanding of the code, reduce the defects and increase the cohesion of that code which is a more important goal overall. The same can be said for something like set based concurrent engineering where we work on two solutions simultaneously and then throw one away. It’s inefficient but we can delay potentially making the wrong decision so it makes sense to do it.
-
Dan also suggests that 'you get what you measure' and lists a series of examples where aiming for a certain target is actually not very effective and doesn’t give us what we want anyway. For example if our goal is to have all the tests passing by the end of the month then we may be tempted to comment out failing tests to achieve this. One which I notice quite frequently is aiming for a story point total which tends to result in reduced communication between business and IT and we end up delivering features that may not have that much value. It might seem to be locally efficient but in the grand scheme of things it’s not efficient at all.
-
There’s also some discussion around the desire for people to be 'busy' all the time which is quite common from my experience. Dan points out that if we can get to a stage where we have time when we’re not busy then we have more time to think about what we’re doing and perhaps we can even spend this time innovating. I think the key here is to ensure that we’re still doing something which is contributing to the overall system goal rather than not doing nothing at all!
There’s loads more - it’s a very good talk - but these are some of the bits that stand out for me.
About the author
I'm currently working on short form content at ClickHouse. I publish short 5 minute videos showing how to solve data problems on YouTube @LearnDataWithMark. I previously worked on graph analytics at Neo4j, where I also co-authored the O'Reilly Graph Algorithms Book with Amy Hodler.