Mark Needham

While working on some code with Toni we realised that we'd managed to create two functions that were almost exactly the same except they made different service calls and returned collections of a different type.

The similar functions were like this:

private IEnumerable<Foo> GetFoos(Guid id)
{
    IEnumerable<Foo> foos = new List<Foo>();
    try
    {
        foos = fooService.GetFoosFor(id);
    }
    catch (Exception e)
    {
        // do some logging of the exception
    }
    return foos;
}

private IEnumerable<Bar> GetBars(Guid id)
{
    IEnumerable<Bar> bars = new List<Bar>();
    try
    {
        bars = barService.GetBarsFor(id);
    }
    catch (Exception e)
    {
        // do some logging of the exception
    }
    return bars;
}

We're defining the empty lists so that if the service throws an exception we can make use of an empty list further on in the code. A failure of the service in this context doesn't mean that the application should stop functioning.

My thinking here was that we should be able to pull out the service calls into a function but the annoying thing is that they return different types of collections so I initially thought that we'd be unable to remove the duplication.

Thinking about the problem later on I realised we could just define the return value of the service call in the function to use generics.

We therefore end up with this solution:

private IEnumerable<Bar> GetBars(Guid id)
{
	return GetValues(() => barService.GetBarsFor(id));
}

private IEnumerable<Foo> GetFoos(Guid id)
{
	return GetValues(() => fooService.GetFoosFor(id));
}

private IEnumerable<T> GetValues<T>(Func<IEnumerable<T>> getValues)
{
    IEnumerable<T> values = new List<T>();
    try
    {
        values = getValues();
    }
    catch (Exception e)
    {
        // do some logging of the exception
    }
    return values;
}

I think the code is still quite readable and it's relatively obvious what it's supposed to be doing.

I came across an interesting post that Roy Osherove wrote a few months ago where he talks about 'Willed vs Forced Designs' and some common arguments that people give for not using TypeMock on their projects.

I'm not really a fan of the TypeMock approach to dealing with dependencies in tests because it seems to avoid the fact that the code is probably bad in the first place if we have to resort to using some of the approaches it encourages.

Having said that Roy makes the following point which I think is quite accurate:

You let an automated tool (rhino mocks, Moq etc..) tell you when your design is OK or not. That point alone should go against anything ALT.NET has ever stood for, doesn’t it? If you need a tool to tell you what is good or bad design, then you are doing it wrong.

While it is true that it's useful to be able to know for ourselves whether our code is drifting into territory where it's become way too complicated, I think it is useful to have the tests as a reminder that this is becoming the case.

It's quite easy when you have a delivery deadline and are under pressure to stop being as observant about the quality of what you're coding and to rush to complete our particular task.

In these situations it can be useful to be restricted by our framework to the extent that the pain we'll feel in trying to test our code will act as an indicator that we're doing something wrong.

What I found interesting when reading Roy's post is that the arguments sound sounds quite similar to the discussion a couple of years ago with respect to whether using Mockito instead of jMock was bad because it hides design problems that you have with dependencies. Steve Freeman wrote the following comment on Dan's post:

But, it also became clear that he wrote Mockito to address some weak design and coding habits in his project and that the existing mocking frameworks were doing exactly their job of forcing them into the open. How a team should respond to that feedback is an interesting question.

In the meantime, I’ve found that I /can/ teach using the existing frameworks if I concentrate on what they were intended for: focusing on the relationships between collaborating objects. I’ve seen quite a few students light up when they get the point. In fact, the syntactic noise in jMock really helps to bring this out, whereas it’s easy for it to get lost with easymock and mockito.

In this case I definitely prefer the style of mocking that we get with Mockito over jMock even though I've worked on code bases where we've created objects with way too many dependencies and haven't felt the pain as much because the framework is so easy to use.

I can't think of a compelling argument for why this is different to the TypeMock vs other mocking frameworks argument. It seems to be a similar argument around dependencies in our code.

The other thing I'm intrigued about is whether the choice of framework should be in some way linked to the level of skill of the people who are going to use it.

If someone is a Dreyfus Model novice with respect to object oriented design then it would make much more sense to use a tool which makes it really obvious that they're doing something wrong. In that case using a perhaps more limited tool would just be a quick feedback mechanism.

Once we have a bit more skill then it would seem more appropriate to use the more powerful tool which we have the ability to abuse but hopefully now have the experience to know when we can and cannot get away with doing so.

In the end the argument seems quite similar to ones I've often heard about programming in Ruby and whether or not we should give programmers powerful language features because they're liable to hang themselves.

In conclusion I'm thinking that perhaps TypeMock in experienced hands isn't such a bad thing and could actually be useful in some select situations but would probably be quite a dangerous tool for someone new to the whole unit testing game.

I've been browsing through Chris Smith's Programming F# book and in the chapter on pattern matching he describes the 'function' key word which I haven't used before.

It's used in pattern matching expressions when we want to match against one of the parameters passed into the function which contains the pattern match.

For example if we have this somewhat contrived example:

let isEven value = match value with 
                    | x when (x % 2) = 0 -> true
                    | _ -> false

That could be rewritten using the function keyword to the following:

let isEven  = function 
               | x when (x % 2) = 0 -> true
               | _ -> false

It's a relatively straight forward way to simplify code like this although one thing I noticed while looking back through some old code I've written is that if we use this syntax then we need to ensure that the parameter we want to pattern match against is passed as the last parameter to a function.

For example this function which is used to parse the arguments passed to a script was originally written like this:

let GetArgs initialArgs  =
    let rec find args matches =
        match args with
        | hd::_ when hd = "--" -> List.to_array (matches)
        | hd::tl -> find tl (hd::matches) 
        | [] -> Array.empty
    find (List.rev (Array.to_list initialArgs) ) []

If we want to use 'function' then we'd need to put 'args' implicitly as the second argument passed to the recursive 'find' function:

let GetArgs initialArgs  =
    let rec find matches =
        function
        | hd::_ when hd = "--" -> List.to_array (matches)
        | hd::tl -> find (hd::matches) tl
        | [] -> Array.empty
    find [] (List.rev (Array.to_list initialArgs) )

I'm not sure that the resulting code is necessarily more intention revealing if the function has more than one argument passed to it. The second version of this function could be very confusing if you didn't know what the 'function' keyword actually did.

One of the common discussions that I've had with several colleagues when we're making use of some of the higher order functions that can be applied on collections is whether to use the LINQ style syntax or to chain the different methods together.

I tend to prefer the latter approach although when asked the question after my talk at Developer Developer Developer I didn't really have a good answer other than to suggest that it seemed to just be a personal preference thing.

Damian Marshall suggested that he preferred the method chaining approach because it more clearly describes the idea of passing a collection through a pipeline where we can apply different operations to that collection.

I quite like that explanation and I think my preference for it would have probably been influenced by the fact that when coding in F# we can use the forward piping operator to achieve code which reads like this.

For example if we had a list and wanted to get all the even numbers, double them and then add them up we might do this:

[1..10] |>
List.filter (fun x -> x % 2 = 0) |>
List.map (fun x -> x * 2) |>
List.fold (fun acc x -> acc + x) 0

If I was in C# I'd probably do this:

Enumerable.Range(1, 10)
.Where(x => x % 2 == 0)
.Select(x => x * 2)
.Sum(x => x);

I found it quite difficult to work out what the equivalent LINQ syntax would be because I don't use it but I think something like this would be what you'd need to write to do the same thing:

from x in Enumerable.Range(1, 10)
where x%2 == 0
select x * 2).Sum(x => x);

I'm not sure if there's a way to do the sum within the LINQ statement or whether you need to do it using the method as I have here.

Even just writing this example I found that the way I had to write the LINQ code seemed quite counter intuitive for me with the way that I typically try to solve problems like this.

At least now thanks to Damian I now understand why that is!

One of the things which Nat Pryce and Steve Freeman suggest in their book Growing Object Oriented Software guided by tests is the idea of wrapping any third party libraries that we use in our own code.

We came across a situation where we did this and then later on I made the mistake of not following this advice.

To start with my colleague David had created a DSL which kept all the calls to Selenium nicely wrapped inside one class.

The problem we were experiencing was that we hadn't evolved the DSL with the webpage evolution to the point where we weren't taking into account that some fields on the page weren't visible until ones before them had been filled in.

We needed to change the DSL slightly and it seemed like an interesting opportunity to try and convert them to use Webdriver as it seems more suited for heavy filling in of forms which is our use case.

My first thought was that it should just be possible to change those Selenium calls to call the equivalent Webdriver methods instead but having done that we realised that the way the two tools interact with the page is slightly different so the direct replacement approach wasn't really working.

We decided to adopt a different approach whereby we would just try and change individual tests to use Webdriver instead and leave all the other tests as they are using Selenium.

I thought about creating another version of the DSL to encapsulate the Webdriver interaction with the page but decided against the idea as it didn't seem like it would add much value and the only way I thought of at the time was to create a clone of the original DSL.

We managed to get one of our tests working more effectively using Webdriver having sorted out the problems with the different interactions between fields but unfortunately the current C# API doesn't seem that stable and seems to fail somewhat randomly for reasons we haven't been able to work out yet.

As a result we now want to convert those tests I'd rewritten to take advantage of the new way they're written but to use Selenium instead!

Sadly the approach I took has made this really difficult and it's now a very frustrating journey to get the tests back into shape.

It's quite frustrating to make this type of mistake especially when I read about a solution so recently.

In hindsight I think a better approach may have been to pull our an interface to represent our DSL – currently it's a series of method calls on a few classes – and then create Webdriver and Selenium specific versions of that.

A few of things stand out for me from this experience:

• I talked myself out of wrapping Webdriver because I saw the main reason for doing so being to shield us in case we chose to change the library and I didn't anticipate having to do that. As it is I was wrong but I didn't totally appreciate how we can benefit from defining an API that defines how we want to interact with the web page rather than how the library wants us to.
• We need to evolve our DSLs with the application and not be afraid to change them if the application changes.
• It probably wasn't a good idea to try and fix the DSL and change the underlying library at the same time. Small steps!

One of the more interesting higher order functions that I've come across while playing with F# is the partition function which is similar to the filter function except it returns the values which meet the predicate passed in as well as the ones which don't.

I came across an interesting problem recently where we needed to do exactly this and had ended up taking a more imperative for each style approach to solve the problem because this function doesn't exist in C# as far as I know.

In F# the function makes use of a tuple to do this so if we want to create the function in C# then we need to define a tuple object first.

public class Tuple<TFirst, TSecond>
{
	private readonly TFirst first;
	private readonly TSecond second;
 
	public Tuple(TFirst first, TSecond second)
	{
		this.first = first;
		this.second = second;
	}
 
	public TFirst First
	{
		get { return first; }
	}
 
	public TSecond Second
	{
		get { return second; }
	}
}

public static class IEnumerableExtensions
{
	public static Tuple<IEnumerable<T>, IEnumerable<T>> Partition<T>(this IEnumerable<T> enumerableOf, Func<T, bool> predicate)
	{
		var positives = enumerableOf.Where(predicate);
		var negatives = enumerableOf.Where(e => !predicate(e));
		return new Tuple<IEnumerable<T>, IEnumerable<T>>(positives, negatives);
 
	}
}

I'm not sure of the best way to write this function – at the moment we end up creating two iterators to cover the two different filters that we're running over the collection which seems a bit strange.

In F# 'partition' is on List so the whole collection would be evaluated whereas in this case we're still only evaluating each item as it's needed so maybe there isn't a way to do it without using two iterators.

If we wanted to use this function to get the evens and odds from a collection we could write the following code:

var evensAndOdds = Enumerable.Range(1, 10).Partition(x => x % 2 == 0);
 
var evens = evensAndOdds.First;
var odds = evensAndOdds.Second;

The other thing that's nice about F# is that we can assign the result of the expression to two separate values in one go and I don't know of a way to do that in C#.

let evens, odds = [1..10] |> List.partition (fun x -> x % 2 = 0)

We don't need to have the intermediate variable 'evensAndOdds' which doesn't really add much to the code.

I'd be interested in knowing if there's a better way to do this than what I'm trying out.

I did a presentation titled 'Mixing functional and object oriented approaches to programming in C#' at the Developer Developer Developer conference in Reading.

The slides from the talk are below:

I've not done many technical talks so far. My only previous attempt was a talk on F# one at the Sydney Alt.NET user group last year so I'm still learning how to do this effectively.

It was quite interesting reading through some of the feedback from twitter and several people pointed out that the content was too basic which was something I was concerned about while writing the talk:

OK. 30 mins into #ddd8 talk of functional vs OO approaches and I've been told one small thing I didn't know.

#ddd8 finished @markhneedham talk… Good speaker,but but basic For me, the last 20% of the content should have been 80% of the talk

"If you want to learn more…" Er…yeah. I certainly want to learn more than I just did. #ddd8

Mark did a good job but difficulty lvl would help as it was quite basic. Missed caching bit looking at twitter. Doh!

Mark Needham's talk on functional and OO at #DDD8 was OK, but the first 2/3 was too much of a simple intro.

The feedback I got in person was that the talk worked alright and there were a couple more positive comments on twitter too:

Great day at #ddd8 – really enjoyed Mark Needham's functional programming session & testing ASP.Net with Ruby

Listening to @markneedham at #ddd8. He has a fantastic abilty to break things down and explain them

I think there were probably too many slides in the talk – I found myself pausing unnaturally so that I could move to the next slide which was also pointed out on twitter:

In my first session at #ddd8 on functional programming – loadsa slides – someone get the guy a remote!

I think I'll probably go with less slides the next time – some of them weren't adding much value anyway so I don't think it would hurt too much to drop them.

Overall though it was a good experience for me getting the chance to put this talk together and I think I learnt a little more about better ways to understand the way we can use functional programming in C# by having the opportunity to prepare this.

My colleague David Santoro has started up a technical book club at the client we're working at in Wales and the book choice for the first session was Chapter 7 – Achieving Object Oriented Design – of Growing Object Oriented Software, guided by tests written by Steve Freeman and Nat Pryce.

In this chapter they cover various approaches for driving towards object oriented code including techniques to find new objects and a detailed description of TDD and how we can approach this in a way that allows us to drive out new behaviour effectively.

These are some of my thoughts and our discussion of the chapter:

• I quite liked the quote that the authors open the chapter with:
  

  In matters of style, swim with the current; in matters of principle, stand like a rock.
  —Thomas Jefferson

  I think we often end up spending a lot of time arguing about style when a lot of the time keeping to core principles is more important if we're to keep our state in a maintainable state. I suppose it's a bit of an abstract quote but it seems a useful one to keep in mind.

• One suggestion that the authors make is to organise our code into layers – an implementation layer where have our different objects (which Martin Fowler refers to as the push button API) and then a declarative layer on top of that i.e. a domain specific language.

  On the projects I've worked on I don't recall us doing this except for in test code where we would often end up writing DSLish code to describe how the application was supposed to work. I suppose this is just a logical next step.

• Somewhat related to this the authors point out that following a TDD approach ensures that we have to specify the 'what' of what we want to achieve before we specify the 'how'. In this case the test describes what we want to achieve and the production code is the implementation of that.

  They also emphasise the importance of ensuring our tests are understandable by limiting their scope. From my experience we can often get away with using very minimal test data in most of our unit tests and if we need to write a lot of setup code for each test then we've probably done something wrong somewhere.

  Anwar pointed out that if we're finding it difficult to test what we want to from one level then it can make sense to try and test that from further up the stack. While this is certainly true I think we need to be careful to ensure that we're not just doing that because we don't want to work out how to make our code easier to test further down.

• I first realised how useful value objects could be after seeing a talk by Dan Bergh Johnsson at QCon in London last year and the authors describe several ways that we can try and get more of these types of objects in our code.
  • Breaking out – if code's becoming too complicated we might split the responsibility of a large object into several smaller ones. I find that the key here is to focus on making incremental improvements to this type of code. It can often be a big job to get the design of these types of objects as we want them and since we might not get the time to do that, just making the smallest improvement we can think of repeatedly can be quite useful. Having said that I still find it quite difficult to do.
  • Budding off – to mark a domain concept in the code we might create an object that wraps a field or has no fields at all. I find this approach is the hardest to introduce as people often see it as adding pointless code when we could just use a primitive instead. This is the area of micro types and I think there's probably a measure of judgement required here to determine in which situations we get benefit from taking this approach.
  • Bundling up – if we see a group of values being used together then we might bundle them together into a type. I find that it's easier to do this when it's groups of primitives that you're bringing together. I don't find it as easy to spot when groups of objects are being used together and a concept is probably missing.
• While I've been doing TDD for a few years now I don't think I've yet totally grasped the idea of pulling interfaces into existence to the extent that the authors describe while writing tests:
  

  We think of this as “on-demand” design: we “pull” interfaces and their imple- mentations into existence from the needs of the client, rather than “pushing” out the features that we think a class should provide.

  Liz Keogh refers to this as Pixie Driven Development and I think following this approach would help us to end up with meaningful interfaces containing groups of functionality rather than seemingly random groups of methods which seem somewhat related.

  It's certainly an area I need to spend more time practicing on.

I wrote about my first thoughts using Automapper last week and although I realised that it makes use of the static gateway pattern we ran into a problem where two consecutive calls to a method using AutoMapper always returned the same value for one of the mappings.

The code was roughly like this:

public Bar CreateNewBar(Bar originalBar, string someNewValue)
{
	Mapper.CreateMap<Baz, Baz>()
      .ForMember(x => x.Id, opts => opts.Ignore())
      .ForMember(x => x.SomeProperty, opts => opts.MapFrom(source => someNewValue));
}

In our test everything worked fine because we were only calling the method once but when testing it out we were making multiple calls to the method and always receiving the same value for 'someNewValue'.

I hadn't quite pieced together that each mapping was probably only created once for each source/destination pair. I thought it would be recreated each time but browsing through the code we can see that's what's going on:

Configuration.cs

public IMappingExpression CreateMap(Type sourceType, Type destinationType)
{
   var typeMap = CreateTypeMap(sourceType, destinationType);
 
   return new MappingExpression(typeMap, _serviceCtor);
}

public TypeMap CreateTypeMap(Type source, Type destination)
{
   TypeMap typeMap = FindExplicitlyDefinedTypeMap(source, destination);
 
   if (typeMap == null)
   {
   ...
   }
   return typeMap;
}

private TypeMap FindExplicitlyDefinedTypeMap(Type sourceType, Type destinationType)
{
   return _typeMaps.FirstOrDefault(x => x.DestinationType == destinationType && x.SourceType == sourceType);
}

We put a test which called the method 'CreateNewBar' method twice to make sure that was actually the problem and then made use of the 'Reset' method on 'Mapper' to avoid the problem:

public Bar CreateNewBar(Bar originalBar, string someNewValue)
{
    Mapper.Reset();
	Mapper.CreateMap<Baz, Baz>()
      .ForMember(x => x.Id, opts => opts.Ignore())
      .ForMember(x => x.SomeProperty, opts => opts.MapFrom(source => someNewValue));
}

I've read several times about the dangers of the big rewrite when it comes to production code but I've recently been wondering whether or not we should apply the same rules when it comes to test code or not.

I worked with Raphael Speyer for a few weeks last year and on the code base we were working on he often spent some time rewriting tests originally written using rMock to use mockito which was the framework we were driving towards.

One of the benefits that he was able to get from doing this was that he had to understand the test in order to change it which enabled him to increase his understanding of how the code was supposed to work and identify anything that didn't seem quite right.

I quite liked this idea at the time and while spending some time recently working with some tests which required quite a lot of setup and were testing several different things in the same test.

Unfortunately a few of them were failing and it was quite difficult to work out why that was.

My initial approach was to try and work my way through the tests inlining all the test code to start with and then extracting out irrelevant details to make the tests easier to understand.

Despite those attempts I was unable to work out why the test was failing so I worked out what the main things the test was trying to verify and then wrote tests from scratch for each of those cases.

I was able to write tests covering everything the original test did in several smaller tests in less time than I had spent trying to debug the original one and with a fair degree of confidence that I'm testing exactly the same thing.

As I see it the big danger of rewriting is that we're always playing catch up with the current system which is still being worked on in production and we never quite catch up.

I'm not so sure this logic applies in this case because we're only rewriting small bits of code which means that we can replace the original test very quickly.

My main driver when working with tests is to ensure that they're easy to understand and make it easy to reason about the code so if I have to rewrite some tests to make that happen then I think it's a fair trade off.

My initial approach would nearly always be to refactor the tests that are already there. Rewriting is something I'd look to do if I was really struggling to refactor effectively.