Mark Needham

Thoughts on Software Development

Testing: What is a defect?

with 2 comments

One of the key ideas that I have learnt from my readings of The Toyota Way and Taaichi Ohno's Workplace Management is that we should strive not to pass defects through the system to the next process, which you should consider to be your customer.

As a developer the next process for each story is the testing phase where the testers will (amongst other things) run through the acceptance criteria and then do some exploratory testing for scenarios which weren't explicitly part of the acceptance criteria.

The question is how far should we go down this route and what exactly is a defect using this terminology – if a tester finds a bug which was listed in the acceptance criteria then I think it's reasonable enough to suggest that the developer has moved a defect onto the next stage.

But what about if that bug only appears on one particular browser and that's one that the developer didn't test against but the tester did. Clearly automating tests against different browsers can help solve this problem but there are still some types of tests (particularly ones requiring visual verification) where it's much more grey.

We want developers to write code with as few defects as possible but at the end of the day testers are much better at using software in ways that is likely to expose defects that developers wouldn't even think about and I think this is definitely a good thing.

My current thinking around this area is that a defect is something which was covered by the acceptance criteria or something which has been previously exposed by exploratory testing and reappears.

Anything else is a normal part of the process.

Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • HackerNews
  • StumbleUpon
  • Twitter

Written by Mark Needham

December 18th, 2008 at 10:34 pm

Posted in Testing

Tagged with

2 Responses to 'Testing: What is a defect?'

Subscribe to comments with RSS or TrackBack to 'Testing: What is a defect?'.

  1. [...] Testing: What is a defect? at Mark Needham [...]

  2. It's worth looking at Lean Product Development and the Toyota Product Development System to understand how to think about this better. What we're really thinking about is efficiency of detection. Some "defects" are really things we are learning about how the technology works. If we are having to re-learn, that's a defect.

    One way to think about this is that testing to a specification is waste (although perhaps necessary waste) while testing to understand boundaries is quite valuable.

    Jason Yip

    21 Dec 08 at 9:22 am

Leave a Reply