Mark Needham

Thoughts on Software Development

TDD: Big leaps and small steps

with 6 comments

About a month ago or so Gary Bernhardt wrote a post showing how to get started with TDD and while the post is quite interesting, several comments on the post pointed out that he had jumped from iteratively solving the problem straight to the solution with his final step.

Something which I’ve noticed while solving algorithmic problems in couple of different functional programming languages is that the test driven approach doesn’t work so well for these types of problems.

Dan North points out something similar in an OreDev presentation where he talks about writing a BDD framework in Clojure.

To paraphrase:

If you can’t explain to me where this approach breaks down then you don’t know it well enough. You’re trying to sell a silver bullet.

The classic failure mode for iterative development is the big algorithm case. That’s about dancing with the code and massaging it until all the test cases pass.

Uncle Bob also points this out while referring to the way we develop code around the UI:

There is a lot of coding that goes into a Velocity template. But to use TDD for those templates would be absurd. The problem is that I’m not at all sure what I want a page to look like. I need the freedom to fiddle around with the formatting and the structure until everything is just the way I want it. Trying to do that fiddling with TDD is futile. Once I have the page the way I like it, then I’ll write some tests that make sure the templates work as written.

I think the common theme is that TDD works pretty well when we have a rough idea of where we intend to go with the code but we just don’t know the exact path yet. We can take small steps and incrementally work out exactly how we’re going to get there.

When we don’t really know how to solve the problem – which more often than not seems to be the case with algorithmic type problems – then at some stage we will take a big leap from being nowhere near a working solution to the working solution.

In those cases I think it still makes sense to have some automated tests both to act as regression to ensure we don’t break the code and to tell us when we’ve written the algorithm correctly.

An example of a problem where TDD doesn’t work that well is solving the traveling salesman problem.

In this case the solution to the problem is the implementation of an algorithm and it’s pretty difficult to get there unless you actually know the algorithm.

During that dojo Julio actually spent some time working on the problem a different way – by implementing the algorithm directly – and he managed to get much further than we did.

It seems to me that perhaps this explains why although TDD is a useful design technique it’s not the only one that we should look to use.

When we have worked out where we are driving a design then TDD can be quite a useful tool for working incrementally towards that but it’s no substitute for taking the time to think about what exactly we’re trying to solve.

Written by Mark Needham

December 10th, 2009 at 10:14 pm

Posted in Testing

Tagged with

  • Pingback: Tweets that mention TDD: Big leaps and small steps at Mark Needham -- Topsy.com

  • http://malvasiabianca.org/ David Carlton

    One thing I’ve noticed when trying to learn Haskell is that I’m having a hard time doing TDD properly – I keep on needing to make larger leaps of inspiration than I’m used to or comfortable with. I’m trying to figure out how much of this is the “TDD is hard for deriving algorithms, and functional programming is more about the algorithms” problem, and how much is just needing to learn a different collection of refactorings. (I’m positive that the latter is at least part of the mix.)

    I’m trying to be more conscious about finding small refactorings to do; one first attempt is here: http://malvasiabianca.org/archives/2009/12/small-steps-in-haskell/ (With a bit more explanation in the comments.) I’m pretty curious how this will turn out as I go farther along with the language.

  • http://blog.extracheese.org Gary Bernhardt

    I’m sure you saw it already, but the next post on my blog, “The Limits of TDD”, addressed this issue, as well as other complaints that came up in the comments. At the end, I conclude:

    “””
    Many complaints about TDD are complaints that it doesn’t solve some problem. These are not problems with TDD – it’s not supposed to solve every problem!

    Dynamic languages don’t make coffee, continuous integration doesn’t shine shoes, and TDD doesn’t make code scale. It’s simply the basis of a solid, disciplined process for building software – a beginning, not an end.
    “””

    In cases where I don’t know where I’m going (like templates), I certainly don’t TDD. If it’s code, I’ll come back and TDD it once I understand it (making the original a spike).

    In cases where I know exactly where I’m going (like known algorithms), I’ll still TDD a naive solution, as I did in the post you link to. Then I’ll use the resulting test suite as a safety net to refactor to the standard solution (e.g., iterative rather than recursive Fibonacci). In these cases, I still get the test coverage benefits of TDD, but the design benefits aren’t relevant.

    Thanks, Mark. I liked this post. :)

  • http://www.markhneedham.com Mark Needham

    Hey Gary,
    Yeh I saw that post as well, was trying to work out how I could work it into this one but I didn’t quite figure out how to!

    That’s a pretty good summary though and I like the idea that sometimes we can’t TDD it to start with but maybe later we can come back and do that. Hadn’t quite connected that bit – neat.

    Cheers, Mark

  • http://www.clear-lines.com/blog Mathias

    Mark,
    Can you elaborate on the traveling salesman case? Specifically, in the exercise, are you trying to find the optimal solution, or to find a sub-optimal but computationally efficient solution? And what made it hard?
    I am curious about this, because most of my code is math-oriented, and I am a long-time fan of TDD, which in general I found to be a good match, but in some cases, on algorithms, it just doesn’t flow at all, so I am very interested in figuring out patterns or symptoms that indicate a rough road ahead!

  • http://www.markhneedham.com Mark Needham

    Hey Mathias,

    I shall try!

    From what I recall we were actually struggling a bit just to come up with a solution at all.

    What made it hard was that we ended up having to write a functional test that effectively solved the whole problem for one set of inputs and we found it quite difficult to find some small steps to incrementally work up to that.

    We managed to create a domain model to represent the problem but to actually even get a simple route calculated I felt that we were at the stage where we needed to go and read up on the algorithm and work out how you calculate the distances between two different nodes.

    Cheers, Mark