Mark Needham

I did a presentation of some of the stuff that I've learnt from playing around with F# over the last six months or so at the most recent Alt.NET Sydney meeting.

I've included the slides below but there was also some interesting discussion as well.

• One of the questions asked was around how you would deal with code on a real project with regards to structuring it and ensuring that it was maintainable. I'm not actually sure what the answer is to this question as I haven't written any code in F# that's in production but there are certainly applications written n F# that are in production – the main one that I know a bit about is one which Amanda Laucher worked on which she spoke about at the Alt.NET conference in Seattle.
• There was some discussion about dynamic v static languages – Phil spoke of not caring about what type something is rather caring about what it does. I pretty much agree with this and I think when using languages which have quite strong type inference such as F# (and more-so Haskell from what I hear) then I think we do move more towards that situation.
• Erik raised the point that functional languages aren't the solution for everything and I certainly feel it's niche is probably around operations with heavy data parsing/mining involved. I'm not sure I'd fancy doing an ASP.NET MVC application only in F# although I've seen some WPF code written using F# (unfortunately can't remember where) which looked reasonable so I'm not sure we should write it off just yet.

I've put the code that I walked through in the presentation on bitbucket.

One of the most frustrating things for me lately about interacting with C# libraries from F# has been setting up objects through the use of properties.

While I am against the use of setters to construct objects in the first place, that's the way that a lot of libraries work so it's a bit of a necessary evil!

In C# we would typically make use of the object initializer syntax to do this, but in F# I've been writing code like this to do the same thing:

type MessageBuilder(?message:string, ?user:string) =              
	let buildMessage message user =
	   let twitterStatusBuilder = new TwitterStatus()
	      twitterStatusBuilder.Text <- message
	      twitterStatusBuilder.User <-
	      	let userBuilder = new TwitterUser()              
	         userBuilder.ScreenName <- user
	         userBuilder
	      twitterStatusBuilder
 
	member self.Build() = 
		buildMessage (if message.IsSome then message.Value else "") (if user.IsSome then user.Value else "")

This is more verbose than strictly necessary but I wanted to try and ensure all mutations to objects were being done within the function creating it rather than creating an object and then mutating it which feels strange to me in F# land.

I recently realised that it's actually possible to call properties in the same way that we can create objects using named parameters.

We therefore end up with the following code:

type MessageBuilder(?message:string, ?user:string) =              
    let buildMessage message user = new TwitterStatus(Text = message, User = new TwitterUser(ScreenName = user))
 
    member self.Build() = 
        buildMessage (if message.IsSome then message.Value else "") (if user.IsSome then user.Value else "")

Which is much more concise and does the same thing.

I've been spending a bit of time reading through the Fake source code to try and understand how it works and one of the things which I quite like about it is the way the authors have made use of different F# operators to make expressions easier to read by reducing the number of brackets that need to be written and reordering the functions/values depending on the particular context.

One which I hadn't seen before is the application operator which is the opposite of the forward operator which I have previously written about.

The application operator (<|) applies a value to a function, the function being on the left and the value on the right.

It is used in FileHelper.fs as part of the DeleteFile function:

let DeleteFile x =   
  let file = new FileInfo(x)    
  if file.Exists then 
    log <| sprintf "Deleting %s" file.FullName
    file.Delete()
  else
    log <| sprintf "%s does not exist." file.FullName

The log function is of type 'string -> unit' and the sprintf call helps create that string. Without the application operator we would have to put in extra parentheses:

log (sprintf "Deleting %s" file.FullName)

The code also makes use of the forward operator which I think panders more to the object oriented style of programming whereby you have have some data/object and then apply a method/function to that. I find that code written in this way reads more intuitively to me at the moment.

One example of this is the SetDirReadOnly function in FileHelper.fs

  let rec SetDirReadOnly readOnly (dir:DirectoryInfo) =
    dir.GetDirectories() |> Seq.iter (fun dir ->
      SetDirReadOnly readOnly dir
      setDirectoryReadOnly readOnly dir)
    dir.GetFiles() |> Seq.iter (fun file -> file.IsReadOnly <- readOnly)

In this case if we didn't have the forward operator then in theory we should be able to just put the 'dir.GetFiles()' can be passed as the second argument to 'Seq.iter':

  let rec SetDirReadOnly readOnly (dir:DirectoryInfo) =
    dir.GetDirectories() |> Seq.iter (fun dir ->
      SetDirReadOnly readOnly dir
      setDirectoryReadOnly readOnly dir)
    Seq.iter (fun file -> file.IsReadOnly <- readOnly)  dir.GetFiles()

In fact what we get is a compilation error:

Successive arguments should be separated by spaces or tupled, and arguments involving function or method applications should be parenthesized.

In this case we need to paranthesise the 'dir.GetFiles()' method call:

  let rec SetDirReadOnly readOnly (dir:DirectoryInfo) =
    dir.GetDirectories() |> Seq.iter (fun dir ->
      SetDirReadOnly readOnly dir
      setDirectoryReadOnly readOnly dir)
    Seq.iter (fun file -> file.IsReadOnly <- readOnly)  (dir.GetFiles())

Which leads to another compilation error:

Lookup on object of indeterminate type based on information prior to this program point. A type annotation may be needed prior to this program point to constrain the type of the object. This may allow the lookup to be resolved

In this case what we're being told is that the compiler is unable to work out the type of 'file' in the function being passed to 'Seq.iter'. We can fix this by specifically stating its type:

  let rec SetDirReadOnly readOnly (dir:DirectoryInfo) =
    dir.GetDirectories() |> Seq.iter (fun dir ->
      SetDirReadOnly readOnly dir
      setDirectoryReadOnly readOnly dir)
    Seq.iter (fun (file:FileInfo) -> file.IsReadOnly <- readOnly)  (dir.GetFiles())

It works but it seems to miss the point of getting the F# compiler to infer which types you're talking about – the forward operator simplifies the code a lot. I also think the code is more readable having 'files' at the beginning as it seems more obvious that the function is being applied to the sequence of files when written this way.

These operators are pretty cool and I've found it quite useful to look at the full list of the F# operators available on the Microsoft Research website as there may well be even more built in functions that can help simplify our code further.

This week's dojo involved coding a familiar problem – the bowling game – in a different language, Groovy.

The code we wrote is available on bitbucket.

The Format

Cam, Dean and I took turns pairing with each other with the code projected onto a TV. As there were only a few of us the discussion on where we were taking the code tended to included everyone rather than just the two at the keyboard.

What We Learnt

• I've sometimes wondered about the wisdom of running dojos in newer languages but this one worked quite well because Cam has been learning Groovy and he was able to point us in the right direction when we started writing Java-esque Groovy code. The particular syntax that I didn't know about was that you can define and put items into a list in a much simpler way than in Java:

  I was starting to write code like this:

  def frames = new List<Frame>()
  frames.Add(frame)

  Which Cam simplified down to:

  def frames = []
  frames << frame

  I didn't feel that I missed the static typing you get in Java although IntelliJ wasn't quite as useful when it came to suggesting which methods you could call on a particular object.

• I'm even more convinced that using various languages functional equivalent of the 'for each' loop, in this case 'eachWith' and 'eachWithIndex' is not the way to go and we could see our code becoming very complicated when trying to work out how to score strikes thanks to our use of it!
• I think we actually got further this time in terms of the implementation although we did slow down when it came to scoring strikes to try and work out exactly how we wanted to do it.

  Prior to this we had been following the idea of just getting the tests to pass and driving the design of the code that way but we at this stage it seemed foolish to keep doing that as the code would increased dramatically in complexity by doing so.

  The two approaches we were thinking of involved using the state pattern to determine what the current frame outcome was and then work out the cumulative score based on that by looking forward to future frames or an approach that would make use of functional collection parameters (not sure exactly which ones!) to calculate the score in a more function rather than OO way.

For next time

• We'll probably keep going with some more Groovy as it's actually more interesting than I thought it would be. I'm also keen to do a coding dojo where we never make use of the if statement.

I previously wrote about an approach we took to safely remove some duplication and I recently followed a similar mantra to replace an object initializer call which had around 40 properties being setup with a builder to try and make the code a bit easier to understand.

We did have tests checking the values being setup by the object initializer so I was already able to refactor with some degree of safety – it would probably have been possible to just create the builder and build the object from that and then delete the old code and replace it with the new but I've caused myself too many problems from doing that before that I decided to try a more incremental approach.

The idea was to have the builder and the object initializer both creating the object at the same time and as I built a property from the builder I would set it in the object initializer until eventually all of the properties were being set directly from the builder's object and then I could just return that instead – this approach feels quite similar to what Kent Beck describes as having parallel implementations in his recent presentation on Responsive Design.

The code I started with was something like this:

public Foo CreateMeAFoo() 
{
	return new Foo
	{
		Bar = "someValue",
		OtherBar = "someOtherValue",
		UltraBar = "aValue",
		...
		AnotherArgumentWayDownHere = 1
		...
		AndSoOn = "yes"
 
	}
}

I worked together with one of the business analysts on our team who pointed out that there were actually some clear groupings in what I had just been considering 'data' and we were able to put those explicit domain concepts into the code as part of the builder.

My first step however was to remove the object initializer to avoid making any silly mistakes – an example of one I have made when using object initializers is to set a property using 'Int16.Parse' and then passing in a string with a value of "53700″ which causes the method to throw an exception and the stack trace just directs you to the first line of the object initializer, making it quite difficult to work out which line has failed.

Having worked the code into a sequence of setters I gradually added methods to the builder to create the policy:

public Foo CreateMeAFoo() 
{
	var fooBuilder = new FooBuilder().AddBars("someValue", "someOtherValue", "aValue);
	var newFoo = fooBuilder.Build();
 
	var foo = new Foo();
	foo.Bar = newFoo.Bar;
	foo.OtherBar = newFoo.OtherBar;
	foo.UltraBar = newFoo.UltraBar;
	...
	foo.AnotherArgumentWayDownHere = 1
 
	return foo;
}

public class FooBuilder  
{
	private string bar;
	private string otherBar;
	private string ultraBar;
 
	public FooBuilder AddBars(string bar, string otherBar, string ultraBar)
	{
		this.bar = bar;
		this.otherBar = otherBar;
		this.ultraBar = ultraBar;
		return this;
	}
 
	public Foo Build()
	{
		var foo = new Foo();
		foo.Bar = bar;
		foo.OtherBar = otherBar;
		foo.UltraBar = ultraBar;
		return foo;
	}
}

I created some duplication by doing this – I am now creating the 'Foo' twice – but I didn't check any of the code into the main branch until I had totally replaced the original Foo creation with the builder.

I did about two of three properties at a time and then ran the tests which I thought might be too small a step but actually saved me on a couple of occasions so it probably actually saved me time.

Eventually when I had all the tests passing I got rid of the original Foo and replaced it with the one from the builder:

public Foo CreateMeAFoo() 
{
	var fooBuilder = new FooBuilder().AddBars("someValue", "someOtherValue", "aValue);
	return fooBuilder.Build();
}

This code is still in a state of transition – it is still possible to create an object with half the fields not set by passing in nulls to the builder for example – and I'm trying to work out what the best step is to fix that.

I generally prefer to have everything setup in the constructor and then you know that the object is in a good state as soon as you create it, but in this case moving everything straight into the constructor will probably make the object even more difficult to deal with.

My current thinking is to maybe check the pre conditions for creating the object inside the builder and then refactor out value objects so that there are not so many properties overall but I'm open to other ideas.

I attended the most recent ThoughtWorks Quarterly Technology briefing on Tuesday which was titled 'Agile Adoption – How to stuff it up' and presented by my colleagues Andy Marks and Martin Fowler.

There seems to be quite a few books out at the moment about how to introduce a more agile approach into your organisation – I've been reading Lean-Agile Software Development and Becoming Agile and there is also a book called Scaling Lean and Agile Development – so I was intrigued to see whether the messages from this talk would be similar to those in these books.

What did I learn?

• I often find when listening to Martin Fowler speak that although what he says is quite similar each time when speaking about agile there always seems to be something different that stands out for me each time.

  This time what stood out was his mention of the Dreyfus model with regards to people's level of skill when it comes to agile – when you first start out as a novice it's quite hard to keep the principles in mind so you spend a lot of time focusing on the practices and getting better at these but if you want to keep improving then at some stage you need to move up to a level where the principles do become more predominant.

• Andy made an interesting point that IT and in particular software development is pretty much made for people who want to learn new things and he also pointed out the myth that 'learning finishes at school'. I never really considered this before but for me it definitely applies – the process of learning new ideas appeals far more to me than the results and outcomes of projects so it was interesting to hear this coming from someone else.
• Transparency with regards to bad news was something else which was pointed out as being fairly important and it's certainly an area where we often run into trouble – often organisations aren't used to bad news being delivered to them early and they get the impression that if it's going badly now then it's going to keep on going badly, rather than seeing that it's quite good to get bad news early because then you have time to fix it.
• Martin described the 'pilot project anti pattern' which he has come across where organisations make use of agile on a project which noone really cares about and use it as a training ground. It was suggested that this is not an effective way of introducing an an agile approach as it doesn't matter to anyone so there's no incentive to work out whether the new approach is really beneficial or not.
• I liked the question that Andy posed about success and failure. He first of all asked anyone in the room who had seen an email from their CEO talking about a really successful project and congratulating the team to put their hands up. Pretty much the whole room did.

  He then asked who had received an email from their CEO talking about a failure and the lessons learned from that and only one person's hand went up!

  Andy is definitely right when he suggests that "if you're not failing you're not learning anything" and this is something which I've also come across from Andy Hunt in Pragmatic Learning and Thinking and more recently I'm trying to get into the 'improvement ravine' with regards to learning F# as I'm still writing object oriented F# which I think is missing the point a bit. Thinking in a more functional way is the key for me there.

• A question was raised about how agile can fit in with fixed price projects at the end where it was pointed out that if the price and the time are fixed then the scope has to be variable – it can be infinitely flexible. It's actually often the case that a lot of value can be delivered with reduced scope even though it doesn't seem that way when you're told that all three of them are fixed!

We've been using Fiddler to debug the requests and responses sent via web services to a service layer our application interacts with and it works pretty well when you run the application using Cassini but by default won't work when you run the website through IIS.

The key to this as one of my colleagues (who gives credit to Erik) showed me today is to ensure that IIS is running under the same user that Fiddler is running under which in our case is the 'Administrator' account.

The default user for IIS is 'Network Service' but as far as I'm aware you can't actually launch an application such as Fiddler from that account.

We therefore changed IIS to run under the 'Administrator' account on local development machines since this is where we typically use Fiddler.

To do this we need to:

• Create a new application pool by going to the 'Application Pool' menu in IIS Manager and setting the 'Identity' of the new pool to be the Administrator account.
• Change the application pool for our application to match that new application pool.
• Go to 'Computer Management > Local Users and Groups > IIS_WPG' and add the Administrator as a user of that group.

Fiddler should now capture requests/responses!

I'm not sure running IIS as the Administrator account is such a great idea although it's only on the local development environment so maybe it's a reasonable trade off for the benefits we get from being able to debug web service communication.

We've recently run into some places in our tests where the expectation and actual values passed into NUnit's 'Assert.AreEqual' are the wrong way round, therefore meaning that the error messages we get when tests fail are somewhat confusing!

Assert.AreEqual(theActualValue, "the expectation");

We can change the arguments around using Resharper by using the key combination 'Ctrl-Alt-Shift-ArrowKey' but you can only do this one line at a time which was a bit annoying as there were about 20 to change.

I got a bit bored of doing this after a while so I thought I'd look into whether it would be possible to do this with a 'Find & Replace'.

After a bit of trial and error this is what I've ended up with:

• Select all the areas of code that you want to change and press 'Ctrl-H'
• In the find box type:

  \({.*}, {\".*\"}

• And in the replace box type:

  (\2, \1

The '{}' define a matching group of which we define two in this case and then switch them around. Visual Studio's regex seems a bit different than the one I'm used to – the reference list for the syntax is available on MSDN.

It's not too complicated and I'm sure there are edge cases where it wouldn't work but for the little case I had it did the job reasonably well.

I recently came across the idea of continuations while reading Real World Functional Programming and Wes Dyer has a blog post where he explains continuations in more detail and also talks about the idea of using a continuation passing style in languages which don't support Call/CC (Call with Current continuation).

As I understand it we can achieve a continuation passing style of programming by passing in the bit of code that we went executed next (i.e. the continuation) as an argument to a function.

Wes has a series of examples in C# so I thought I'd see what they look like in F#:

The first example is a function 'Identity' which originally returns the value it is given before the calling function prints it to the screen.

let identity value = value
printfn "%s" (identity "mark")

In CPS the function 'identity' would take in the printfn statement as a continuation:

let identity value k = k value
identity "mark" (printfn "%s")

The type of identity has change from "'a -> 'a" to "'a -> ('a -> 'b) -> 'b" which in our example is "string -> (string -> unit) -> unit).

The next example is to convert a 'Max' function to CPS style:

let max m n = if m > n then m else n
printfn "%d" (max 2 3)

That would become:

let max m n k = if m > n then k m else k n
max 2 3 (printfn "%d")

In both of these cases the "printfn.." part of the statement implicitly defines a function which takes in a value of 'string' in the first case and 'int' in the second.

We could just have easily written the code like this:

let max m n k = if m > n then k m else k n
max 2 3 (fun number -> printfn "%d" number)

I found the next example a bit tricky as it involves passing the continuation on to another function.

We originally have the following:

let g n = n + 1    
let f n = g(n + 1) + 1    
printfn "%d" (f(1) + 1)

This becomes:

let g n k = k(n+1)
let f n k = g (n + 1) (fun x -> k(x + 1))
f 1 (fun n -> printfn "%d" (n + 1))

The thing I found strange when trying to think about how to do this is that anything which would happen after the initial function call needs to be passed into the continuation instead.

You need to always think about what happens next instead of just thinking about the result that needs to be calculated. At the moment I think I'm more used to a sequential flow of operations but I'm sure that will change.

The same applied for the last example, factorial:

let rec factorial n = 
	if n = 0 
	then 1 
	else n * factorial (n-1)

Apply the same idea of putting any additional calculations outside the function call into the continuation we end up with:

let rec factorial n k =
	if n = 0 
	then k(1)
	else factorial (n-1) (fun x -> k(n*x))

The first factorial I define here is not tail recursive as we still need to multiply the result of all the recursive calls by n at the end, meaning that the compiler can't optimise this code to just keep the latest call to the function on the stack.

The usual way I've seen to get around this is to thread an accumulator to an inner function inside factorial which would keep the running total of the factorial calculation but passing continuations is another way of doing this as we are passing on all the data needed to factorial every time we call it.

Looking at the C# via IL code version of these two functions we get the following:

public static int factorial(int n)
{
    if (n == 0)
    {
        return 1;
    }
    return (n * factorial(n - 1));
}

public static a factorial2<a>(int n, FastFunc<int, a> k)
{
    while (n != 0)
    {
        k = new factorial2@103<a>(n, k);
        n--;
    }
    return k.Invoke(1);
}

The second one has been converted into a while loop which I hadn't realised was what happened with tail recursive calls until I came across Jomo Fisher's post about tail recursion.

I'm still getting the hang of this and would definitely agree with Wes' comment:

CPS turns a program inside-out. In the process, the programmer may feel that his brain has been turned inside-out as well.

When I first read about continuations they sounded a bit similar to callbacks and Wes defines a callback as being an 'explicity-passed continuation' which is effectively what we are doing here. The difference for me is that we are passing the continuation around whereas when I've used a callback it's typically executed straight away.

William Caputo has also been playing around with continuations in C# and has some interesting thoughts as well.

I pick up Michael Feathers' Working Effectively with Legacy Code book from time to time and one of my favourite parts of the book is the chapter where he talks about 'Seams'.

To quote the book:

A seam is a place where you can alter behaviour in your program without editing in that place

Seams in the book are generally discussed in terms of how we can get tests around legacy code which was written without easy testability in mind but I've noticed that the ideas behind seams seem to be more widely applicable than this.

The reason for using seams elsewhere is fairly similar as far as I can tell – we want to alter the way that code works in a specific context but we don't want to change it in that place since it needs to remain the way it is when used in other contexts.

Removing dependencies

One place where we recently used what I consider to be a seam was to remove calls to external sites from our application when we were running our tests against the application in our normal build.

We don't have any control over these dependencies as they are completely external so we made calls to those urls go to localhost instead by adding an entry in our hosts file (C:\WINDOWS\system32\drivers\etc) like so:

127.0.0.1		external.site.com

On our UAT and production servers further down the pipeline we don't have that type of setting in the hosts file so those run against the real dependencies.

I think the ideal place to apply this seam would be in a configuration file where you would be able to configure dependencies but in this case the dependency is actually added in outside of our team's control so we decided to adjust the behaviour where we could albeit a bit further away from where the behaviour is than we would have liked.

Impersonators

Another area where we might make use of seams in our systems is in the creation of what my colleague Julio Maia refers to as 'impersonators'.

Impersonators are little pieces of code that we write to impersonate 3rd party systems we need to integrate with – ideally we would have an impersonator for every integration point and we can make use of these impersonators at certain stages of our build rather than always having to call the real end point.

One example of an impersonator that we've used recently is a proxy which captures requests and responses being made to a service layer from our application and then just replays these back every other time the same requests are made.

The ideal place for this seams to be triggered (i.e. the seam's enabling point) is in a configuration which is independent of our production code and therefore allows us to choose when we want to make use of the impersonator and when we need to use the real service.

In our case we have the endpoint defined in our 'web.config' and a step in our build process generates versions of that file for each of the environments that we might run our application in.

We can then change the 'service.Endpoint' entry to point to the proxy for our development environment but have it point to the real service layer for other environments.