Mark Needham

I've written previously about function overloading in F# and my struggles working out how to do it and last week I came across the concept of inline functions and statically resolved parameters as a potential way to solve that problem.

I came across a problem where I thought I would be able to make use of this while playing around with some code parsing Xml today.

I had a 'descendants' function which I wanted to be applicable against 'XDocument' and 'XElement' so I originally just defined the functions separately forgetting that the compiler wouldn't allow me to do so as we would have a duplicate definition of the function:

let descendants name (xDocument:XDocument) = xDocument.Descendants name
let descendants name (xElement:XElement) = xElement.Descendants name

I wanted to make use of the inline function to define a function which would allow any type which supported the 'Descendants' member:

let inline descendants name (xml:^x) =  
    (^x : (member Descendants : XName -> seq<XElement>) (xml))

I couldn't work out how I could pass the 'name' input parameter to 'Descendants' so I was getting the following error:

expected 2 expressions, got 1

I posted the problem to StackOverflow and 'Brian' pointed out the syntax that would allow me to do what I wanted:

let inline descendants name (xml:^x) =  
	(^x : (member Descendants : XName -> seq<XElement>) (xml,name))

Tomas Petricek pointed out that in this case we could just write a function which took in 'XContainer' since both the other two types derive from that anyway:

let descendants name (xml:XContainer) = xml.Descendants name

In this situation that certainly makes more sense but it's good to know how to write the version using member constraints for any future problems I come across.

I've been playing around with some F# code this evening and one of the bits of code needs to make a HTTP call and return the result.

I wrote this code and then tried to make use of the 'Async.RunSynchronously' function to execute the call.

The code I had looked roughly like this:

namespace Twitter
 
module RetrieveLinks
    open System.Net
    open System.IO
    open System.Web
    open Microsoft.FSharp.Control
 
    let AsyncHttp (url:string) = async {
        let request =  HttpWebRequest.Create(url)
        let! response = request.AsyncGetResponse()
        let stream = response.GetResponseStream()
        use reader = new StreamReader(stream  )
        return! reader.AsyncReadToEnd() }
 
    let getData =
        let request = "http://some.url"
        AsyncHttp <| request
 
	Async.RunSynchronously getData

The problem was I was getting the following error on the last line:

Error	3	Unexpected identifier in implementation file

I've seen that error before and it often means that you haven't imported a reference correctly and hence the compiler doesn't know what you're trying to refer to.

In this case I was fairly sure all my references were correct and I was still getting the same error when I used the full namespace to 'Async.RunSynchronously' which seemed to suggest I'd done something else wrong.

After comparing this file with another one which was quite similar but didn't throw this error I realised that I'd left of the '=' after the module definition. Putting that in solved the problem.

namespace Twitter
 
module RetrieveLinks = 
	// and so on

As I understand it if we don't use the '=' then we've created a top level module declaration and if we do use the '=' then we've created a local module declaration.

From MSDN:

You do not have to indent declarations in a top-level module. You do have to indent all declarations in local modules. In a local module declaration, only the declarations that are indented under that module declaration are part of the module.

Given this understanding another way to solve my problem would be to remove the indentation of the functions inside the module like so:

module RetrieveLinks
open System.Net
open System.IO
open System.Web
open Microsoft.FSharp.Control
 
// and so on until...
 
Async.RunSynchronously getData

That compiles as expected.

From reading the MSDN page it would suggest that in my first example I'd created a top level module declaration but indenting the code inside that module somehow meant that the 'Async.RunSynchronously' function wasn't recognised.

I don't quite understand why that is so if anyone can enlighten me that would be cool!

One thing which I've often wondered when playing around with F# is that when writing the following function the type of the function is inferred to be 'int -> int -> int' rather than allowing any values which can be added together:

let add x y = x + y
> val add : int -> int -> int

It turns out if you use the 'inline' keyword then the compiler does exactly what we want:

> let inline add x y = x + y
val inline add : 
	^a -> ^b -> ^c
	when ( ^a or  ^b) : (static member ( + ) :  ^a *  ^b ->  ^c)

Without the inline modifier type inference forces the function to take a specific type, in this case int. With it the function has a statically resolved type parameter which means that "the type parameter is replaced with an actual type at compile time rather than run time".

In this case it's useful to us because it allows us to implicitly define a member constraint on the two input parameters to 'add'. From the MSDN page:

Statically resolved type parameters are primarily useful in conjunction with member constraints, which are constraints that allow you to specify that a type argument must have a particular member or members in order to be used. There is no way to create this kind of constraint by using a regular generic type parameter.

The neat thing about the second definition is that we can add values of any types which support the '+' operator:

add "mark" "needham";;
> val it : string = "markneedham"

> add 1.0 2.0;;
val it : float = 3.0

From a quick look at the IL code in Reflector it looks like the 'add' function defined here makes use of the 'AdditionDynamic' function internally to allow it to be this flexible.

One thing which I found quite interesting while reading about inline functions is that it sounds like it's quite similar to duck typing in that we're saying a function can be passed any value which supports a particular method.

Michael Giagnocavo has a post where he covers the idea of statically type resolved parameters in more detail and describes what he refers to as 'statically typed duck typing'.

I've been browsing through Chris Smith's Programming F# book and in the chapter on pattern matching he describes the 'function' key word which I haven't used before.

It's used in pattern matching expressions when we want to match against one of the parameters passed into the function which contains the pattern match.

For example if we have this somewhat contrived example:

let isEven value = match value with 
                    | x when (x % 2) = 0 -> true
                    | _ -> false

That could be rewritten using the function keyword to the following:

let isEven  = function 
               | x when (x % 2) = 0 -> true
               | _ -> false

It's a relatively straight forward way to simplify code like this although one thing I noticed while looking back through some old code I've written is that if we use this syntax then we need to ensure that the parameter we want to pattern match against is passed as the last parameter to a function.

For example this function which is used to parse the arguments passed to a script was originally written like this:

let GetArgs initialArgs  =
    let rec find args matches =
        match args with
        | hd::_ when hd = "--" -> List.to_array (matches)
        | hd::tl -> find tl (hd::matches) 
        | [] -> Array.empty
    find (List.rev (Array.to_list initialArgs) ) []

If we want to use 'function' then we'd need to put 'args' implicitly as the second argument passed to the recursive 'find' function:

let GetArgs initialArgs  =
    let rec find matches =
        function
        | hd::_ when hd = "--" -> List.to_array (matches)
        | hd::tl -> find (hd::matches) tl
        | [] -> Array.empty
    find [] (List.rev (Array.to_list initialArgs) )

I'm not sure that the resulting code is necessarily more intention revealing if the function has more than one argument passed to it. The second version of this function could be very confusing if you didn't know what the 'function' keyword actually did.

Since I started playing around with F# one of the things I've been trying to do is not use the 'for' keyword because I was trying to avoid writing code in an imperative way and for loops are a big part of this for me.

Having read Jon Harrop's solution to the word count problem where he made use of both sequence and for expressions I thought it'd be intersting to see what some of the code I've written would look like using that approach.

An example of a function that I wrote which could be rewritten the other way is the following:

let delimeters (value:string) = Regex.Matches(value, "\[([^]]*)\]") |> Seq.cast |> 
                                Seq.map (fun (x:Match) -> x.Groups) |>
                                Seq.map (fun x -> x |> Seq.cast<Group> |> Seq.nth 1) |>
                                Seq.map (fun x -> x.Value)

This could be written like this if we used a sequence expression instead of chaining map operations:

let delimeters (value:string) = seq { for m in Regex.Matches(value, "\[([^]]*)\]") do yield m.Groups.Item(0).Value }

One interesting thing I found about writing it like this was that I noticed that 'GroupCollection' had the 'Item' property on it which would let me get the match much more easily.

I completely missed that when I was writing the first solution so I'm not sure if that was just due to my lack of knowledge of that part of the API or whether the second approach actually encouraged me to explore more and therefore end up with a simpler solution.

Another example I found was this expression for getting the matches for a regular expression:

let regex pattern input = Regex.Matches(input, pattern) |> Seq.cast |> Seq.map (fun (x:Match) -> x.Value)

That can be simplified to the following with a sequence expression:

let regex pattern input = seq { for m in Regex.Matches(input, pattern) do yield m.Value }

One neat thing about using sequence expressions is that we don't need to make use of 'Seq.cast' to convert a value to a typed sequence – we can just use it as it is.

The following function can be rewritten to just use a for expression:

let writeTo (path:string) (values:seq<string * int>) = 
    use writer = new StreamWriter(path)
    values |> Seq.map (fun (value, count) -> value + " " + count.ToString()) |> Seq.iter (fun x -> writer.WriteLine(x))

Like so:

let writeTo (path:string) (values:seq<string * int>) = 
    use writer = new StreamWriter(path)
    for (value,count) in values do writer.WriteLine(value + " " + count.ToString())

We eventually iterate through the sequence anyway so I think it's more intention revealing to just do the iteration and mapping in one step.

This is a function from when I was writing the little Feedburner application:

let calculateWeeklyAverages =
    Seq.reverseSequence >>
    Seq.windowed days >>
    Seq.map (fun (entries:array<Entry>) -> 
                (entries.[0]).Date , entries |> Array.map (fun e -> e.Circulation |> toDecimal) |> Array.average ) >>
    Seq.reverseSequence

If we use a sequence expression it'd look like this:

let calculateWeeklyAverages entries =
    seq { for (e:array<Entry>) in (entries |> Seq.reverseSequence |> Seq.windowed days) do 
              yield ((e.[0]).Date, entries |> Array.map (fun e -> e.Circulation |> toDecimal) |> Array.average) } 
    |> Seq.reverseSequence

The resulting code is shorter but it seems to me like the focus when you read the code has moved to the line which yields the tuple whereas in the first version I find that I read the function as a whole.

I've not really used sequence expressions that much so it's been interesting going through the code and seeing where they might be useful.

I found several places where I'd used lists because I find those easier to pattern match against but I wonder whether it would make sense to use sequences there as well.

I was looking through some of the F# code I've written recently and I realised that I was very much writing C# in F# with respect to the number of if statements I've been using.

I thought it would be interesting to see what the code would look like if I was able to refactor some of that code to make use of pattern matching instead which would be a more idiomatic way of solving the problem in F#.

The first example of if statements is in my post about my F# solution to Roy Osherove's TDD Kata.

I originally wrote a parse function which was able to parse a string and give it's decimal value or 0 if it couldn't be parsed.

let parse value = 
    let (itParsed, value) = Decimal.TryParse value
    if (itParsed) then value else 0.0m

If we use a pattern match expression we'd end up with the following:

let parse value = match Decimal.TryParse value with | (true, value) -> value | (false, _) -> 0.0m

The neat thing about this approach is that we don't need to store the result of the 'Decimal.TryParse' function as we did in my original version.

We could in theory also write it like this…

let parse value = match Decimal.TryParse value with | (true, value) -> value | (_, _) -> 0.0m

…but while that is slightly less code I quite like the other version because it's a bit more intention revealing that 0 will be returned if we fail to parse the string. I think there's less thinking needed to understand the code.

Another example is this bit of code:

let add value = if ("".Equals(value) or "\n".Equals(value)) then 0.0m
                else match digits value |> Array.filter (fun x -> x < 1000m) with 
                     | ContainsNegatives(negatives) -> raise (ArgumentException (buildExceptionMessage negatives))
                     | NoNegatives(digits)          -> digits |> Array.sum

If we convert that to use pattern matching we would get this:

let add value = match value with
                | "" -> 0.0m
                | "\n" -> 0.0m
                | value ->  match digits value |> Array.filter (fun x -> x < 1000m) with 
                            | ContainsNegatives(negatives) -> raise (ArgumentException (buildExceptionMessage negatives))
                            | NoNegatives(digits)          -> digits |> Array.sum

That's maybe slightly easier to read mainly because of the splitting up of the two inputs which lead to a 0 result.

The final example of using if statements is the following bit of code:

let (|CustomDelimeter|NoCustomDelimeter|) (value:string) = 
	...
 
     if (value.Length > 2 && "//".Equals(value.Substring(0, 2))) then
         if ("[".Equals(value.Substring(2,1))) then CustomDelimeter(delimeters value)
         else CustomDelimeter([| value.Substring(2, value.IndexOf("\n") - 2) |])
     else NoCustomDelimeter(",")

The only way I could see how to make this use active patterns is on the boolean statements like so:

let (|CustomDelimeter|NoCustomDelimeter|) (value:string) = 
	...
 
    match (value.Length > 2 && "//".Equals(value.Substring(0, 2))) with
    | true -> match ("[".Equals(value.Substring(2,1))) with 
              | true  ->  CustomDelimeter(delimeters value)
              | false ->  CustomDelimeter([| value.Substring(2, value.IndexOf("\n") - 2) |])
    | false -> NoCustomDelimeter(",")

I quite like that it lines up the return values of the active pattern which seems to make it a bit more readable.

I think overall maybe the pattern matching versions are slightly more readable but maybe not by much. What do you think?

As I've mentioned in a few of my recent posts I've been having another go at Roy Osherove's TDD Kata but this time in F#.

One thing I've been struggling with when coding in F# is working out how many intermediate variables we actually need. They can be useful for expressing intent better but they're clutter in a way.

I've included my solution at the end and in the active pattern which determines whether or not we have a custom delimeter defined in our input string I can't decide whether or not to create a value to represent the expressions that determine that.

3
4
5
6
7
8
9
10

    let (|CustomDelimeter|NoCustomDelimeter|) (value:string) = 
        ...
        let hasACustomDelimeter = value.Length > 2 && "//".Equals(value.Substring(0, 2))
 
        if (hasACustomDelimeter) then
            if ("[".Equals(value.Substring(2,1))) then CustomDelimeter(delimeters value)
            else CustomDelimeter([| value.Substring(2, value.IndexOf("\n") - 2) |])
        else NoCustomDelimeter(",")

In a way it's quite obvious that the expression on line 3 is what we're using to determine if the input string has a custom delimeter because we state that on the next line.

    let (|CustomDelimeter|NoCustomDelimeter|) (value:string) = 
        ...
        if (value.Length > 2 && "//".Equals(value.Substring(0, 2))) then
            if ("[".Equals(value.Substring(2,1))) then CustomDelimeter(delimeters value)
            else CustomDelimeter([| value.Substring(2, value.IndexOf("\n") - 2) |])
        else NoCustomDelimeter(",")

I can't decide which I prefer so any thoughts on that would be welcome.

I ran into a bit of trouble trying to make the following requirement work because my original parse function was hiding the fact that the code was failing on this step:

Delimiters can be of any length with the following format:  “//[delimiter]\n” for example: “//***\n1***2***3” should return 6

The parse function was originally defined to returns a zero value if it failed to parse the string which meant that the function which decomposed the string into a sequences of numbers could fail and we wouldn't see an exception, just a failing test.

    let parse value = 
        let (itParsed, value) = Decimal.TryParse value
        if (itParsed) then value else 0.0m

Having the function defined like this simplified the code a bit because I didn't need to deal with ignoring some characters at the beginning of the string when a custom delimeter was being specified.

One of the instructions for the exercise is to focus on writing tests for the valid inputs and not for invalid inputs which I initially struggled with. Usually if I was test driving code I would have written tests against invalid inputs to help me drive out the design.

Once I started focusing on just making the test past instead of finding a generic solution for the whole problem this became much easier and I didn't need to test with the invalid inputs.

I wrote tests for the code in C# using NUnit so that I could run the tests from Resharper. I still haven't found a good way to run automated tests from inside Visual Studio when they're written in F# otherwise I'd have probably just done that.

All the tests I wrote were against the 'add' function but the way the code is written at the moment it would be possible to write tests against the other functions directly if I wanted to.

If I was working in C# perhaps some of those functions would be classes and I would write tests directly against those but I haven't done that here and I'm not sure whether it is necessary. 'digits' is the only function where that would seem to add value.

This is the code I've got at the moment:

module FSharpCalculator
    open System
    open System.Text.RegularExpressions
 
    let split (delimeter:array<string>) (value:string) = value.Split (delimeter, StringSplitOptions.None)
    let toDecimal value = Decimal.Parse value
 
    let (|CustomDelimeter|NoCustomDelimeter|) (value:string) = 
        let delimeters (value:string) = Regex.Matches(value, "\[([^]]*)\]") |> Seq.cast |> 
                                        Seq.map (fun (x:Match) -> x.Groups) |>
                                        Seq.map (fun x -> x |> Seq.cast<Group> |> Seq.nth 1) |>
                                        Seq.map (fun x -> x.Value) |>
                                        Seq.to_array
 
        if (value.Length > 2 && "//".Equals(value.Substring(0, 2))) then
            if ("[".Equals(value.Substring(2,1))) then CustomDelimeter(delimeters value)
            else CustomDelimeter([| value.Substring(2, value.IndexOf("\n") - 2) |])
        else NoCustomDelimeter(",")    
 
    let digits value = match value with 
                       | CustomDelimeter(delimeters)  -> value.Substring(value.IndexOf("\n")) |> split delimeters  |> Array.map toDecimal 
                       | NoCustomDelimeter(delimeter) -> value.Replace("\n", delimeter) |> split [|delimeter |] |> Array.map toDecimal
 
    let buildExceptionMessage negatives = 
        sprintf "No negative numbers allowed. You provided %s" (String.Join(",", negatives |> Array.map (fun x -> x.ToString())))
 
    let (|ContainsNegatives|NoNegatives|) digits =
        if (digits |> Array.exists (fun x -> x < 0.0m)) 
        then ContainsNegatives(digits |> Array.filter (fun x -> x < 0.0m))
        else NoNegatives(digits)
 
    let add value = if ("".Equals(value) or "\n".Equals(value)) then 0.0m
                    else match digits value |> Array.filter (fun x -> x < 1000m) with 
                         | ContainsNegatives(negatives) -> raise (ArgumentException (buildExceptionMessage negatives))
                         | NoNegatives(digits)          -> digits |> Array.sum

I've been playing around with more F# code and after realising that I'd peppered the code with if statements I thought it would be interesting to try and refactor it to make use of active patterns.

The code is part of my F# solution to Roy Osherove's TDD Kata and is used to parse the input string and find which delimeters are being used.

This is the original code:

    let hasCustomDelimeter (value:string) = value.Length > 2 && "//".Equals(value.Substring(0, 2))
    let hasMultipleDelimeters (value:string) = hasCustomDelimeter value && "[".Equals(value.Substring(2,1))
 
    let delimeter value = if (hasCustomDelimeter value) then value.Substring(2, value.IndexOf("\n") - 2) else ","
    let delimeters (value:string) = Regex.Matches(value, "\[(.)\]") |> 
                                    Seq.cast |> 
                                    Seq.map (fun (x:Match) -> x.Groups) |>
                                    Seq.map (fun x -> x |> Seq.cast<Group> |> Seq.nth 1) |>
                                    Seq.map (fun x -> x.Value) |>
                                    Seq.to_array   
 
    let digits value = 
        let delimeter = delimeter value
        if (hasMultipleDelimeters value) then value.Substring(value.IndexOf("\n")) |> split (delimeters value) |> Array.map parse  
        else if (hasCustomDelimeter value) then value.Substring(value.IndexOf("\n")) |> split [| delimeter |] |> Array.map parse
        else value.Replace("\n", delimeter) |> split [|delimeter |] |> Array.map parse

The active pattern that we want to use is known as a 'multi case active pattern' which Chris Smith defines as "partitioning the entirety of the input space into different things". In this case given an input string we want to determine what type of delimeters are contained within that string.

This is the code after I'd created the active pattern:

    let (|MultipleCustomDelimeters|SingleCustomDelimeter|NoCustomDelimeter|) (value:string) =  
        if (value.Length > 2 && "//".Equals(value.Substring(0, 2))) then
            if ("[".Equals(value.Substring(2,1))) then MultipleCustomDelimeters(delimeters value)
            else SingleCustomDelimeter(delimeter value)
        else NoCustomDelimeter(delimeter value)    
 
    let digits value =  
        match value with 
        | SingleCustomDelimeter(delimeter)     -> value.Substring(value.IndexOf("\n")) |> split [| delimeter |] |> Array.map parse
        | MultipleCustomDelimeters(delimeters) -> value.Substring(value.IndexOf("\n")) |> split delimeters  |> Array.map parse 
        | NoCustomDelimeter(delimeter)         -> value.Replace("\n", delimeter) |> split [|delimeter |] |> Array.map parse

One thing I didn't realise is that you can set different types for the constructor values of each of the active patterns. In this case I want to match single or multiple delimeters and then return those delimeters. We can define one active pattern which matches a single delimeter and just returns that and then another one which returns an array of delimeters which is quite neat.

The way it works is quite similar to discriminated unions.

I found that that having all the code around parsing the input in the same function made it easier for me to understand that code and I quite like that it's possible to match the pattern and also get the delimeter/delimeters in one expression.

Although there are more lines of code I think this code is more expressive and it wouldn't be too hard to add in another active pattern if there's another delimeter type that needs to be handled.

I can't decide whether the 'value.SubString' and 'value.Replace' code should also be contained within the active pattern or not. At the moment I'm thinking perhaps not because it's not related to the actual delimeters.

In my continued efforts at Roy Osherove's TDD Kata I've been trying to work out how to split a string based on a delimeter which contains more than one character.

My original thinking was that it should be possible to do so like this:

"1***2".Split("***".ToCharArray());;

I didn't realise that splitting the string like that splits on each of the stars individually which means that we end up getting 2 empty values in the result:

val it : string [] = [|"1"; ""; ""; "2"|]

If we want to split on '***' then we have to pass it in as a value in a string array:

"1***2".Split([| "***" |], StringSplitOptions.None);;

That way we only get the 1 and 2 which is what we want:

val it : string [] = [|"1"; "2"|]

I'd expected there to be an overload which takes in a string and then just splits on that but since there isn't this isn't a bad alternative.

Sam Allen has a very interesting article which covers all sorts of way to split different types of strings.

A while ago I wrote about F#'s forward and application operators where I'd looked at how these could be used to simplify code and while trying out Roy Osherove's TDD Kata I realised that perhaps the choice of which of these to use or whether to use them at all depends on what intent we're expressing.

The specific bit of code I was writing was for raising an exception if negative values were provided and I originally thought I'd use the forward operator to express this code:

    let digits = [| 1;2;3;-3 |] 
    let buildExceptionMessage negatives = sprintf "No negative numbers allowed. You provided %s" 
                                                  (String.Join(",", negatives |> Array.map (fun x -> x.ToString())))
 
    raise (ArgumentException (digits |> Array.filter (fun x -> x < 0) |> buildExceptionMessage))

I think in this case the forward operator doesn't actually express the intent of the code better because it puts the focus on the digits rather than on the building of the exception message.

I changed that a bit to emphasise the importance of the 'buildExceptionMessage' function:

raise (ArgumentException (buildExceptionMessage (digits |> Array.filter (fun x -> x < 0))))

I thought it might be possible to get rid of the brackets around the filtering of the digits and instead apply that expression to 'buildExceptionMessage' using the application operator:

raise (ArgumentException (buildExceptionMessage <| digits |> Array.filter (fun x -> x < 0)))

That actually results in the following error message:

Type mismatch. Expecting a  string -> string but given a  'a array -> 'a array. The type 'string' does not match the type ''a array'

The problem is that it applies digits to 'buildExceptionMessage' first and then tries to apply the result of that to Array.filter instead of applying the filter to the digits and then passing the result of that calculation to 'buildExceptionMessage'.

One way to get around this is to remove the forward operator and move digits to be the second argument passed to Array.filter instead:

raise (ArgumentException (buildExceptionMessage <| Array.filter (fun x -> x < 0) digits))

This is the version that I've got at the moment and I think it expresses the intent of the code the best.

I'd be interested in hearing more thoughts on the best way to use or not use these operators in idiomatic F# code.