Mark Needham

Ken Schwaber recently wrote a blog post where he compared the differences between the kanban, lean and scrum approaches to software development and although I haven’t had the same experiences as he has with the first two, one interesting thing he implies is that with a scrum approach we have slack time built in.

God help us. People found ways to have slack in waterfall, to rest and be creative. With Lean and Kanban, those hiding places are removed. We now have a progressive death march without pause.

The project that I’m currently working on has switched emphasis from being in pure delivery mode into a combination of delivery and handover and it’s been quite noticeable how much more slack we have in the system as a result.

I find it much more enjoyable when there’s a bit of slack such that we’re not churning out story after story.

It gives you the opportunity to explore the code base a bit more and try out any ideas that you have to see if they’re likely to improve the productivity of the team.

An example of this on our project is that we’ve had the time to introduce a calculation descriptions DSL that my colleague Dermot had written in his own time.

This has helped reduce the number of tests required in certain areas of the code base as well as making the code more intuitive and therefore easy to understand.

It can be quite difficult to create this slack time when the team is under big deadline pressure and working on anything which isn’t directly related to hitting that deadline isn’t considered valuable.

I’ve noticed that the typical slack in the projects I’ve worked on tends to be towards the end of the day if we finish a story late on.

That tends to disappear if the team is working late and people start to become too tired to notice that there might be a better way to do things.

It seems to me that it would beneficial to try and work some slack time into projects to allow for the type of innovation that allows us to come up with ideas that allow us to be more productive.

My colleague Dermot Kilroy attended the DDD 2010 Excange in London last week and one of the ideas that he’s been sharing with us from that is that of thinking how the user would solve a given problem without a technological solution i.e. how was something done before computers existed.

This encourages us to take a bigger picture view and can actually lead to a much simpler solution than we’d otherwise come up with.

An example that came up recently on the project I’m working on was related to how we should handle the situation where two agents tried to access the same users record at the same time. The agents are most likely in the same building.

The original technically focused solution was that we needed to lock the record so that if one agent had already opened the record then if another tried to access it they would be unable to.

Of course it’s technically possible to do this but it’s a bit tricky.

If there were no computers then there would probably be a physical folder which would contain all the information about the user.

It’s not actually possible for two agents to both access that at the same time – if one wants to access it and it’s already in use then they need not speak to their colleague and work out when they’ll be finished with it.

If we consider it from this point of view then it becomes clear that putting in the effort to lock the record is unnecessary and that it would make more sense to just pop up a message on the screen informing the agent of the situation and allowing them to decide what to do.

This is the solution that we went with. It took way less time to implement and it’s worked out fine so far.

–

I’ve retrospectively looked at this situation with my colleagues and realised that it seems to fit this pattern quite well but the way that it was reached originally was by looking at the requirement and finding the (initially hidden) reason for it.

It would be interesting to see whether we can rely on this degree of user integrity on a public facing application, I suspect that we can to a much greater degree than we might think.

Dermot and I spent a bit of time today refactoring some code where the logic had ended up in the wrong place.

The code originally looked a bit like this:

public class LookupService
{
	public LookUp Find(UserData userData)
	{
		var param1 = GetParam1From(userData);
		var param2 = GetParam2From(userData);
		var param3 = GetParam3From(userData);
 
		var lookupKey = new LookUpKey(param1, param2, param3);
		return lookupRepository.Find(lookupKey);
	}
}

public class LookUpKey
{
	private readonly string param1;
	private readonly string param2;
	private readonly string param3;
 
	public LookUpKey(string param1, string param2, string param3)
	{
		this.param1 = param1;
		this.param2 = param2;
		this.param3 = param3;
	}
}

We wanted to push the logic used to create the params in ‘LookUpKey’ into the ‘LookUpKey’ since it seemed that was a more natural place for that behaviour to live.

Unfortunately there were also a few other places that were use LookUpKey’s constructor so we couldn’t just go and change that to take in a ‘UserData’ otherwise we’d break other places in the code base.

My initial thought was that we could overload the constructor and temporarily have two constructors until we got rid of the original.

Dermot pointed out a better approach which was to add a static factory method to ‘LookUpKey’ and initially push the logic into that method.

public class LookUpKey
{
	...
 
	public static LookUpKey CreateFrom(UserData userData)
	{
		var param1 = GetParam1From(userData);
		var param2 = GetParam2From(userData);
		var param3 = GetParam3From(userData);
 
		return new LookUpKey(param1, param2, param3);
	}
}

The service is now much simplified:

public class LookupService
{
	public LookUp Find(UserData userData)
	{
		var lookupKey = LookUpKey.CreateFrom(userData);
		return lookupRepository.Find(lookupKey);
	}
}

We can also move tests that were originally indirectly testing the key creation by checking what was passed to the repository to be directly against the ‘LookUpKey’.

Eventually we want to make LookUpKey’s constructor private but for now we’ve been able to make our change without breaking the code base elsewhere.

We’ve been trying to cache some objects using NHibernate’s second level cache which always proves to be a trickier task than I remember it being the previous time!

We’re storing some reference data in the database and then using LINQ to NHibernate to query for the specific row that we want based on some user entered criteria.

We can cache that query by calling ‘SetCacheable’ on the ‘QueryOptions’ property of our query:

1
2
3
4
5
6
7
8
9

public class OurRepository
{
	public ReferenceDataObject Find(string criteria1, string criteria2)
	{
		var query = session.Linq<ReferenceDataObject>();
		query.QueryOptions.SetCachable(true).SetCacheMode(CacheMode.Normal);
		// and so on
	}
}

That helps to ensure that if we do the exact same query again it won’t go to the database again. Instead it will just retrieve the id of the appropriate object from the 2nd level cache and then go and retrieve that.

In order to ensure that the ‘ReferenceDataObject’ does not get retrieved from the database each time we need to ensure that it is cached as well.

We can do that by adding the following to its mapping file:

public class ReferenceDataObjectMappings : ClassMap<ReferenceDataObject>
{
	...
 
	Cache.ReadOnly();
 
	// or
 
	Cache.ReadWrite();
 
	// depending on whether we want to update the value in the cache based on it changing in the database or not
 
	..
 
}

The next step is to ensure that we have the query cache and second level cache turned on in the place where we configure our session factory:

Fluently
.Configure()
.Database(MsSqlConfiguration.MsSql2000.ConnectionString("connection string")
	.Cache(c => c.UseQueryCache().ProviderClass(typeof(NHibernate.Caches.SysCache.SysCacheProvider).AssemblyQualifiedName)))
.Mappings(m => m.FluentMappings.AddFromAssemblyOf<ICurrentSessionFactory>())
.BuildSessionFactory();

The ‘SysCacheProvider’ is one we can use for web applications although there are all sorts of others available too.

Having got all this setup we wrote an integration test which added an item to the reference data table and then tried to retrieve it several times with the theory being that we would only see one call to the database.

[Test]
public void ShouldCacheTheReferenceData()
{
	var myReferenceObject = new ReferenceDataObject(...);
 
	using(var session = SessionFactory.OpenSession())
	{
		session.Save(myReferenceDataObject);
		session.Flush();
	}
 
	// try and retrieve the object a few times here
}

Unfortunately we were seeing the database being hit every single time which confused us greatly until we came across the following explanation on Gabriel Schenker’s blog:

A common error (It happened to me as well!) is to forget to commit or omit a transaction when adding or changing an entity/aggregate to the database. If we now access the entity/aggregate from another session then the 2nd level cache will not be prepared to provide us the cached instances and NHibernate makes an (unexpected round trip to the database).

We need to change our test to commit the object in a transaction if we want the object to be propagated down to the 2nd level cache:

[Test]
public void ShouldCacheTheReferenceData()
{
	var myReferenceObject = new ReferenceDataObject(...);
 
	using(var session = SessionFactory.OpenSession())
	{
		using(var tx = session.BeginTransaction())
		{
    			session.Save(myReferenceObject);
    			tx.Commit();        // important otherwise caching does NOT work!
		}
	}
 
	// try and retrieve the object a few times here
}

We’ve been fiddling around with Fluent NHibernate a bit over the last couple of days and one of the things that we wanted to do was output the NHibernate mapping files being generated so we could see if they were as expected.

I couldn’t figure out how to do it but thanks to the help of James Gregory, Andrew Bullock and Matthew Erbs on twitter this is the code that you need in order to do that:

1
2
3
4
5

Fluently
.Configure()
.Database(MsSqlConfiguration.MsSql2000.ConnectionString("connection string"))
.Mappings(m => m.FluentMappings.ExportTo("c:\\directory-to-output-files-to"))
.BuildSessionFactory();

The 4th line is the important one here and we can choose which directory the mappings files get outputted to and then check that everything is getting setup as we’d expect.

About a year ago I wrote a post about a book club we ran in Sydney covering ‘The readability of tests’ from Steve Freeman and Nat Pryce’s book in which they suggest that their preferred way of writing tests is to drive them from the assertion up:

Write Tests Backwards

Although we stick to a canonical format for test code, we don’t necessarily write tests from top to bottom. What we often do is: write the test name, which helps us decide what we want to achieve; write the call to the target code, which is the entry point for the feature; write the expectations and assertions, so we know what effects the feature should have; and, write the setup and teardown to define the context for the test. Of course, there may be some blurring of these steps to help the compiler, but this sequence reflects how we tend to think through a new unit test. Then we run it and watch it fail.

At the time I wasn’t necessarily convinced that this was the best way to drive but we came across an interesting example today where that approach might have been beneficial.

The test in question was an integration test and we were following the approach of saving the test object directly through the NHibernate session and then loading it again through a repository.

We started the test from the setup of the data and decided to get the mappings and table setup in order to successfully persist the test object first. We didn’t write the assertion or repository call in the test initially.

Having got that all working correctly we got back to our test and wrote the rest of it only to realise as we drove out the repository code that we actually needed to create a new object which would be a composition of several objects including our original test object.

We wanted to retrieve a ‘Foo’ by providing a key and a date – we would retrieve different values depending on the values we provided for those parameters.

This is roughly what the new object looked like:

public class FooRecord
{
   public Foo Foo { get; set; }
   public FooKey FooKey { get; set; }
   public DateTime OnDate { get; set; } 
}

‘FooRecord’ would need to be saved to the session although we would still retrieve ‘Foo’ from the repository having queried the database for the appropriate one.

public class FooRepository
{
   public Foo Find(Date onDate, FooKey fooKey)
   {
      // code to query NHibernate which retrieves FooRecords
      // and then filters those to find the one we want
   }
}

We wouldn’t necessarily have discovered this more quickly if we’d driven from the assertion because we’d still have had to start driving the implementation with an incomplete test to avoid any re-work.

I think it would have been more likely that we’d have seen the problem though.

A few weeks ago we had some C# code around calcuations which had got a bit too imperative in nature.

The code looked roughly like this:

public class ACalculator
{
	public double CalculateFrom(UserData userData)
	{
		if(userData.Factor1 == Factor1.Option1)
		{
			return 1.0;
		}
 
		if(userData.Factor2 == Factor2.Option3)
		{
			return 2.0;
		}
 
		if(userData.Factor3 == Factor3.Option2)
		{
			return 3.0
		}
		return 0.0;
	}
}

I think there should be a more object oriented way to write this code whereby we push some of the logic onto the ‘UserData’ object but it struck me that it reads a little bit like pattern matching code you might see in F#.

I decided to drive the code to use a dictionary which would store functions representing each of the conditions in the if statements:

public class ACalculator
{
	private Dictionary<Func<UserData, bool>, double> calculations;
 
	public ACalculator()
	{
    		calculations = new Dictionary<Func<UserData,bool>,double>
                       {
                           {u => u.Factor1 == Factor1.Option1, 1.0},
                           {u => u.Factor2 == Factor2.Option3, 2.0},                                       
                           {u => u.Factor3 == Factor3.Option2, 3.0}                                 
                       };	
	}	
 
	public double CalculateFrom(UserData userData)
	{
    		var calculation = calculations.Keys.FirstOrDefault(calc => calc(userData));
    		if(calculation != null)
    		{
        		return calculations[calculation];
    		}
 
    		return 0.0;
	}
}

It’s less readable than it was before and it’s not obvious that the adding of the functions to the dictionary needs to be in that order in order for it to work.

I’ve simplified the real example a bit to show the idea but I don’t think it works as the best abstraction in this situation either way although it was an interesting experiment.

On several of the projects that I’ve worked on over the last couple of years we’ve seen the following situation evolve:

• The team starts coding the application.
• At some stage there is a breakthrough in understanding and a chance to really improve the code.
• However the deadline is tight and we wouldn’t see a return within the time left if we refactored the code now
• The team keeps on going with the old approach
• The project ends up going on for longer than the original deadline
• It’s now much more work to move towards the new solution

In the situations I describe the refactorings could have been done incrementally but doing that would take longer than continuing with the original approach and also leave the code in an inconsistent state.

I think the reason this situation evolves consistently in this manner is because although we talk about writing maintainable code, delivery is often considered more important. Pushing out a delivery date in order to refactor code so that it will be easier to work with in the future isn’t something that I’ve seen happen.

Pushing a delivery date out is a cost that we can see straight away.

On the other hand it’s quite difficult to estimate how much of a gain you’ll get by refactoring to a more maintainable/easier to test solution and that gain will not be immediate.

We therefore end up in the situation where we tend to make major refactorings only if we’re going to see a benefit from doing that refactoring before the project ends.

In one sense that seems reasonable because we’re trying to ensure that we’re adding as much value as possible while the client is paying for our time.

On the other hand we’re making life harder for future maintainers of the code base which may in fact be us!

I’d be keen to hear how others handle these types of situations because it feels like this trade off is quite a common one and the way we’ve dealt with it doesn’t seem optimal.

I’ve worked on two different teams this year which had quite different approaches to retrospectives.

In the first team we had a retrospective at the beginning of every iteration i.e. once every two weeks and in the second team we tried out the idea of having a rolling retrospective i.e. we put up potential retrospective items on the wall and when there were enough of those we discussed them in the standup.

The problem we had with the first approach was that we often had the same types of discussions in each retrospective and the same types of issues were raised each week so that it seemed almost pointless to even have it in the first place.

Several people ended up disliking the idea of even doing a retrospective in the first place because of this.

We came up with the idea of having a vote at the beginning of the iteration to decide whether or not we should have one with the idea being that if it didn’t seem necessary then we could skip it for one week but would definitely then have the next one.

I think this worked reasonably well although it’s slightly different to the approach taken by Rolf Knutsen which he discussed at XP2010 in his session on ‘Feedback and Retrospectives in Agile Projects’.

Rolf suggested that we should have a retrospective at a given time interval and just cut it short if we don’t have anything else to discuss.

I think I prefer Rolf’s suggestion as it’s too easy to come to the conclusion that we don’t need a retrospective especially if we think that they’re a waste of time.

I thought the rolling retrospective approach that we took on the second project would work much better as we could just address any problems as soon as they came up but it hasn’t worked quite that way from my experience.

Several issues which would normally be addressed in a retrospective seemed to go by unchecked and without being resolved.

Having a retrospective is a very useful way to put people in a reflective mindset where they look for things that aren’t being done well or that could be done differently whereas during the day we might not do that.

I hadn’t appreciated this benefit of a retrospective session and had felt that problems were often being saved up for a retrospective even if they could have been addressed earlier.

Fabio previously wrote about the idea of the future retrospective box to ensure that we don’t forget anything that happened early on in an iteration when we come to a retrospective but that still doesn’t address the problem of only addressing problems in the retrospective!

It seems like a combination of having a pre scheduled retrospective with problem solving on a day to day basis is what we need so I’d be interested to know how others have addressed this.

I had the chance to attend the XP2010 conference in Trondheim, Norway for a couple of days last week as I was presenting a lightening talk based on a blog post I wrote last year titled ‘Tough projects and the Kubler Ross Grief Cycle‘.

It was interesting to see the way another conference was organised as the only other conference I’ve attended was QCon which is a much more technical conference.

It seemed that a lot of people I spoke to were coming to the same types of conclusions around different software development issues.

Notably that it only makes sense to refactor code mercilessly if we’re actually working around a specific piece of code and that having a target velocity is fairly pointless but seems to be necessary when working with stakeholders who aren’t used to an agile approach.

David Anderson did the day’s keynote which was titled ‘Catalyzing Lean: Building a Limited WIP Society in Your Organization‘.

There were a couple of standout points from this talk for me:

• He talked about using a Work in Progress limit with respect to the number of stories that we can have in each swim lane of the story wall and how it can be used to provoke conversation in the team.

  For example if we notice that there are no items currently being tested then we need to look back to the place just before that i.e. in development to see where the bottleneck in our process is.

  We can then have a conversation about what we can do to fix that. David pointed out that it still takes leadership to make the conversation happen – just using a WIP limit won’t fix all our problems.

• He also showed the idea of using opportunity cost of delay diagrams showing what happens if deliver something late so that we can see which features really are a priority.

  This is linked with the idea of each story having a class of service.

  If the client loses a lot of money if a story isn’t completed by a certain date then this would become a very high priority story for us to complete and it would have a high class of service.

  This seems like a much more useful way of working out the priority of stories as we now understand exactly why something is high priority rather than it just being because someone said it is.

Another interesting session that I attended was one run as a series of Pecha Kucha’s where several high profile agilists described their ‘Agile Suitcase‘ – a list of items that they would take around with them whenever they have to coach the agile approach to software development.

There were lots of cool ideas but the most interesting to me was Joshua Kerievsky’s where he mentioned that running an agile readiness assessment as something that he likes to do with new clients.

I’ve not come across the idea before but it strikes me at least as an approach that would help to get some data on what level agile coaches should start at when working in this type of role.

I don’t know much more about what it entails but found the following on twitter from Kerievsky:

An Agile Readiness Assessment is a time to demonstrate your agility, assess their resistance, explore their future and gain mutual trust.

Another interesting idea from a ‘Learning and Improvement’ lightning talk about ‘Agile teams and rescue dogs’ was that of selecting the team member with the least experience as the team leader.

In the rescue dogs organisation the goal of the leader would be to ensure that the right leader was leading the team depending on the situation.

The leader would vary depending on the situation and the expertise required which seems like it should fit quite well in agile teams.

I’ve not worked in a team where this approach was taken but it seems like something interesting to try out.