In reality there's no reason to create a macro to do what I want to do but I wanted to keep the example simple so I could try and understand exactly how macros work.

I want to create a macro which takes in one argument and then prints hello and the person's name.

In the book Halloway suggests that we should start with the expression that we want to end up with, so this is what I want:

(println "Hello" person)

My first attempt to do that was:

(defmacro say-hello [person]
  println "Hello" person)

I made the mistake of forgetting to include the brackets around the 'println' expression so it doesn't actually pass '"Hello"' and 'person' to 'println'. Instead each symbol is evaluated individually.

When we evaluate this in the REPL we therefore don't quite get what we want:

user=> (say-hello "mark")          
"mark"

Expanding the macro results in:

user=> (macroexpand-1 '(say-hello "Mark"))
"Mark"

Which is the equivalent of doing this:

user=> (eval (do println "hello" "Mark")) 
"Mark"

As I wrote previously this is because 'do' evaluates each argument in order and then returns the last one which in this case is "Mark".

I fixed that mistake and got the following:

(defmacro say-hello [person]
  (println "Hello" person))

Which returns the right result…

user=> (say-hello "Mark")
Hello Mark
nil

…but actually evaluated the expression rather than expanding it because I didn't escape it correctly:

user=> (macroexpand-1 '(say-hello "Mark"))
Hello Mark
nil

After these failures I decided to try and change one of the examples from the book instead of my trial and error approach.

One approach used is to build a list of Clojure symbols inside the macro definition:

(defmacro say-hello [person]
  (list println "hello" person))

user=> (macroexpand-1 '(say-hello "Mark"))
(#<core$println__5440 clojure.core$println__5440@681ff4> "hello" "Mark")

This is pretty much what we want and although the 'println' symbol has been evaluated at macro expansion time it doesn't actually make any difference to the way the macro works.

We can fix that by escaping 'println' so that it won't be evaluated until evaluation time:

(defmacro say-hello [person]
  (list 'println "hello" person))

user=> (macroexpand-1 '(say-hello "Mark"))
(println "hello" "Mark")

I thought it should also be possible to quote(') the whole expression instead of building up the list:

(defmacro say-hello [person] 
  '(println "hello" person))

This expands correctly but when we try to use it this happens:

user=> (say-hello "Mark")
java.lang.Exception: Unable to resolve symbol: person in this context

The problem is that when we use quote there is no evaluation of any of the symbols in the expression so the symbol 'person' is only evaluated at runtime and since it hasn't been bound to any value we end up with the above error.

If we want to use the approach of non evaluation then we need to make use of the backquote(`) which stops evaluation of anything unless it's preceded by a ~.

(defmacro a [person]
  `(println "hello" ~person))

This allows us to evaluate 'person' at expand time and replace it with the appropriate value.

In hindsight the approach I took to write this macro was pretty ineffective although it's been quite interesting to see all the different ways that I've found to mess up the writing of one!

Thanks to A. J. Lopez, Patrick Logan and fogus for helping me to understand all this a bit better than I did to start with!

(defmacro say-hello [person]
  println "Hello" person)

This macro doesn't even expand like I thought it would:

user=> (macroexpand-1 '(say-hello blah))
blah

That seemed a bit strange to me but I eventually realised that I'd missed off the brackets around 'println' and the arguments following it which would have resulted in 'println' being evaluated with those arguments.

I was a bit curious as to why that happened so I tried the following expression without any brackets to see what would happen:

user=> println "hello" "mark"
#<core$println__5440 clojure.core$println__5440@681ff4>
"mark"
"random"

It seems to just evaluate each thing individually and when we put this type of expression into a function definition the function will do the same thing but also return the last thing evaluated:

(defn say-hello [] println "hello" "mark")

user=> (say-hello)
"mark"

A. J. Lopez pointed out that this is quite like progn in other LISPs and is the same as doing the following:

user=> (do println "hello" "mark")
"mark"

do is defined as follows:

(do exprs*)
Evaluates the expressions in order and returns the value of the last. If no expressions are supplied, returns nil.

The way to write a function which passes those two arguments to 'println' is of course to put brackets around the statement:

(defn say-hello [] (println "hello" "mark"))

user=> (say-hello) 
hello mark
nil

'when-let' is used when we want to bind an expression to a symbol and only execute the body provided as the second argument to the macro if that symbol evaluates to true.

As I wrote previously, a value of 'false' or 'nil' would result in the second argument not being evaluated.

A simple example of using 'when-let' would be:

(when-let [a 2] (println "The value of a is:" a))

This is the definition:

1
2
3
4
5
6
7
8
9
10
11
12
13

(defmacro when-let
  "bindings => binding-form test
 
  When test is true, evaluates body with binding-form bound to the value of test"
  [bindings & body]
  (assert-args when-let
     (vector? bindings) "a vector for its binding"
     (= 2 (count bindings)) "exactly 2 forms in binding vector")
   (let [form (bindings 0) tst (bindings 1)]
    `(let [temp# ~tst]
       (when temp#
         (let [~form temp#]
           ~@body)))))

The 'assert-args' call at the beginning of the macro is quite interesting.

Two assertions are stated:

• The first argument should be a vector
• That vector should contain exactly two forms

I've not used dynamic languages very much before but it seems like this is one way for a dynamic language to fail fast by checking that the arguments are as expected. In a static language that would be a compile time check.

Line 9 is quite interesting as we know that 'bindings' will be a vector so we can take the '0th' and '1st' elements from it and bind them to 'form' and 'tst' respectively. I didn't quite pick up on the first few times I read it.

On line 10 it makes use of 'auto-gensym' to create a unique name which begins with 'temp' and is bound to the value of 'tst' which in the simple example provided would be the value '2′. As I understand it the name would be something like 'temp__304′ or something similarly random!

'when 2′ evaluates to true which means that we execute the body provided as the second argument.

user=> (when-let [a 2] (println "The value of a is:" a))
The value of a is: 2
nil

This is a bit of a contrived example of using the construct and it seems to be properly used when we're getting a value out of a list and want to check whether or not we've reached the end of that list or not. If we have then eventually we'll have a value of 'nil' bound by the 'let' and then we'll know we're finished.

An example of where the body wouldn't be evaluated is:

user=> (when-let [a nil] (println "This won't get printed"))
nil

I don't really understand why we need to bind 'form' to 'temp' on the second last line as it doesn't seem like the value is used? I'm sure there's probably something I'm missing there so if anyone could point it out that'd be cool!

As I understand it, the '~@body' on the last line is called the 'splicing unquote' and it allows the individual values in 'body' to be put into the template started at '`(let [temp# ~tst]' individually rather than just being put in there as a list.

We can do the same thing in Clojure but it's even better because we can also define and run unit tests which I think is pretty neat.

Nurullah Akkaya has a good post which describes how to use clojure.test, a testing framework written by Stuart Sierra so I've been using that to define some tests cases for the little RSS feed parser that I'm writing.

To use clojure.test straight out the box you need the latest version of the clojure source code as Stuart Sierra points out on his website.

I ran the ant task for the project and then launched the REPL pointing to the 'alpha snapshot' jar instead of the '1.0.0′ jar and it seems to work fine.

I managed to break the 'get-title' function while playing with it before so I thought that would be a good one to try out the tests in the REPL with.

This function is supposed to strip out the name and the following colon which appears in every title and just show the title of the blog post.

I originally had this definition:

(defn get-title [title]
  (second (first (re-seq #".*:\s(.*)" title))))

I hadn't realised that this strips from the last colon in the string and therefore returns the wrong result for some inputs.

I created the following tests:

(use 'clojure.test)
(deftest test-get-title
  (is (= "Clojure - It's awesome"  (get-title "Mark Needham: Clojure - It's awesome")))
  (is (= "A Book: Book Review" (get-title "Mark Needham: A Book: Book Review"))))

We can run those with the following function:

(run-tests)

FAIL in (test-get-title) (NO_SOURCE_FILE:19)
expected: (= "A Book: Book Review" (get-title "Mark Needham: A Book: Book Review"))
  actual: (not (= "A Book: Book Review" "Book Review"))
 
Ran 1 tests containing 2 assertions.
1 failures, 0 errors.

Changing the function helps solve the problem:

(defn- get-title [title]
  (second (first (re-seq #"[a-zA-Z0-9 ]+:\s(.*)" title))))

Ran 1 tests containing 2 assertions.
0 failures, 0 errors.

We can also run the assertions directly without having to call 'run-tests':

(is (= "A Book: Book Review" (get-title "Mark Needham: A Book: Book Review")))

true

(is (= "Something Else" (get-title "Mark Needham: A Book: Book Review")))

expected: (= "Something Else" (get-title "Mark Needham: A Book: Book Review"))
  actual: (not (= "Something Else" "A Book: Book Review"))
false

Nurullah has more detail in his post about how to integrate tests into a build although I don't need to do that just yet!

It's not finished yet but I'm finding the way that we parse documents like this in Clojure quite intriguing.

The xml to parse looks roughly like this:

<rss version="2.0"> 
	<channel> 
 		...
		<item> 
			<title>Simon Brunning: Links for 2009-11-27 [del.icio.us]</title> 
			<link>http://feedproxy.google.com/~r/SmallValuesOfCool/~3/WDqeLyMA-RE/brunns</link> 
		</item>
		<item> 
			<title>Alex Hung: Extending iPhone battery life</title> 
			<link>http://alexhung.vox.com/library/post/extending-iphone-battery-life.html?_c=feed-atom-full</link> 
		</item>
		...
	</channel> 
</rss>

I've only included the parts of the document that I'm interested in getting.

Following the examples from Stuart Halloway's book one approach to do this is to make use of the 'clojure.xml.parse' and 'clojure.core.xml-seq' functions to create a sequence representing the tree structure of the feed.

I'm used to parsing XML with XPath but that doesn't make as much sense when we have a sequence of hash maps. Instead I'm using 'filter' and 'map' to try and achieve the same outcome.

I found that while I was trying to work out how to use these functions together I was often trying to solve the whole problem in one go instead of breaking it down into smaller more manageable pieces.

I also noticed that I was using 'filter' more often than I needed to instead of filtering the data to the point that everything I wanted to extract was in the remaining data set.

When I was playing with F# I got into the habit of trying to minimise the number of intermediate values I created but this seemed to be making life more difficult so I've allowed myself some intermediate values for the moment!

The goal is to poll the ThoughtWorks RSS feed and then update the planettw account with the latest blog posts. The current setup does that but doesn't include people's Twitter names in the tweet so I'm trying to sort that out.

This is the code I have so far:

(use '[clojure.xml :only (parse)])
(def feed (xml-seq (parse (java.io.File. "clojure-play/tw-blogs-rss.txt"))))
 
(def rss-entries (filter #(= :item (:tag %)) feed))
 
(defn- get-href [link]
  ((comp :href :attrs) link))
 
(defn- get-value [node]
  (first (:content node)))
 
(defn rss-link [entry]
  (get-value (first (filter #(= :link (:tag %))
                            (:content entry)))))
 
(defn rss-title [entry]
  (get-value (first (filter #(= :title (:tag %))
                            (:content entry)))))
 
(def rss-titles (map #(rss-title %) rss-entries))
(def rss-links (map #(rss-link %) rss-entries))
 
(defn- get-author [title]
  (second (first (re-seq #"([\w ]+):" title))))
 
(defn- get-title [title]
  (second (first (re-seq #"[a-zA-Z0-9 ]+:\s(.*)" title))))
 
(def authors (map #(get-author %) rss-titles))
(def titles (map #(get-title %) rss-titles))
 
(defn- get-display-name [twitter-names real-name]
  (let [twitter-name (twitter-names real-name)]
    (if twitter-name (str "@" twitter-name) real-name)))
 
(def twitter-names {"Mark Needham" "markhneedham"
                    "Alex Hung" "alexhung"
                    "Simon Brunning" "brunns"
                    "Ola Bini" "olabini"
                    "Patrick Kua" "patkua"
                    "Marc McNeill" "dancingmango"
                    "Dahlia Bock" "dlbock"
                    "Sumeet Moghe" "sumeet_moghe"
                    "Brian Guthrie" "bguthrie"
                    "Ian Robinson" "iansrobinson"
                    "Ian Cartwright" "cartwrightian"
                    "Duncan Cragg" "duncancragg"
                    "David Cameron" "davcamer"
                    "Steven List" "athought"
                    "Philip Calcado" "pcalcado"
                    "Perryn Fowler" "perrynfowler"
                    "Jason Yip" "jchyip"
                    "Christopher Read" "cread"
                    "Jim Webber" "jimwebber"
                    "John Hume" "duelin_markers"
                    })
 
(defn- create-blog-post [title link author]
  {:tweet (str title " by " (get-display-name twitter-names author) " " link)})
 
(defn create-blog-posts [titles links authors]
  (map #(create-blog-post %1 %2 %3) titles links authors))

To use that you'd need to do this:

(create-blog-posts titles rss-links authors)

Which returns a sequence of hash maps with key 'tweet' and a value of the tweet to display on Twitter:

({:tweet "Links for 2009-11-27 [del.icio.us] by @brunns http://feedproxy.google.com/~r/SmallValuesOfCool/~3/WDqeLyMA-RE/brunns"} 
{:tweet "Extending iPhone battery life by @alexhung http://alexhung.vox.com/library/post/extending-iphone-battery-life.html?_c=feed-atom-full"} 
{:tweet "Threshold Anxiety by Adrian Wible http://thoughtadrian.blogspot.com/2009/11/threshold-anxiety.html"})

The next step is to get this hooked up to the Twitter API.

There are still some things I'm unsure of when it comes to writing applications in Clojure:

• I'm not sure what to do with 'twitter-names'. It's pretty much a global data store so I can't decide whether to just refer to it directly inside other functions or if it should be passed in as a parameter.
• I used 'first' quite a few times in the code to get the first value in a sequence but it doesn't feel like the code expresses the structure of the document very well.
• What's the best way to lay out code for 'defn' expressions? I've been putting the signature on it's own line and then the implementation on other lines which seems to be the way that it's done in the Clojure source code but it sometimes seems like I could just write it all on one line.

The example that I've been trying to understand is applying 'str' to a collection of values.

I started off with the following:

(str [1 2 3])
=> "[1 2 3]"

This just returns the string representation of the vector that we passed it, but what we actually want is to get an output of "123″.

The 'apply' function allows us to do that:

(apply str [1 2 3])
=> "123"

That is semantically/conceptually the same as doing this:

(str 1 2 3)
=> "123"

I didn't quite understand how that could work though and my assumption was that somewhere in the Clojure source the above function call would be happening.

The definition of 'apply' is as follows:

(defn apply
  "Applies fn f to the argument list formed by prepending args to argseq."
  {:arglists '([f args* argseq])}
  [#^clojure.lang.IFn f & args]
    (. f (applyTo (spread args))))

The first thing which I hadn't realised is that when you have an '&' before a parameter definition then any arguments provided will be put into a list.

If we break down the example above we end up with the following:

(. str (applyTo (spread [[1 2 3]])))

The 'spread' function is defined like so:

1
2
3
4
5
6
7

(defn spread
  {:private true}
  [arglist]
  (cond
   (nil? arglist) nil
   (nil? (next arglist)) (seq (first arglist))
   :else (cons (first arglist) (spread (next arglist)))))

In this case we only have one item in 'arglist' so on line 6 the 'next arglist' expression evaluates to nil.

This means that we create a seq from the first argument of the 'arglist' which is '[1 2 3]'.

Working our way back up to the 'apply' function what we end up with is this:

(. str (applyTo (seq [1 2 3]))) 
=> "123"

This calls through to an 'applyTo' method defined on the 'clojure.lang.IFn' interface.

I'm not sure which of the implementations 'str' maps to but it seems like the 'str' function would eventually be called from the Java code with each of the values in the sequence passed in as a separate argument which is pretty neat!

If there were gaps at the beginning or end of the sequence then that was fine but gaps in the middle of the sequence were not.

null, 1, 2, 3 => no gaps
1, 2, 3, null => no gaps
1, null, 2, 3 => gaps

The Java version looked a bit like this:

public boolean hasGaps(List<BigInteger> values) {
    Iterator<BigInteger> fromHead = values.iterator();
    while (fromHead.hasNext() && fromHead.next() == null) {
        fromHead.remove();
    }
 
    Collections.reverse(values);
 
    Iterator<BigInteger> fromTail = values.iterator();
    while (fromTail.hasNext() && fromTail.next() == null) {
        fromTail.remove();
    }
 
    return values.contains(null);
}

We take the initial list and then remove all the null values from the beginning of it, then reverse the list and remove all the values from the end.

We then check if there's a null value and if there is then it would indicate there is indeed a gap in the list.

To write this function in Clojure we can start off by using the 'drop-while' function to get rid of the trailing nil values.

I started off with this attempt:

(defn has-gaps? [list]
    let [no-nils] [drop-while #(= % nil) list]
  no-nils)

Unfortunately that gives us the following error!

Can't take value of a macro: #'clojure.core/let (NO_SOURCE_FILE:16)

It thinks we're trying to pass around the 'let' macro instead of evaluating it – I forgot to put in the brackets around the 'let'!

I fixed that with this next version:

(defn has-gaps? [list]
    (let [no-nils] [drop-while nil? list]
  no-nils))

But again, no love:

java.lang.IllegalArgumentException: let requires an even number of forms in binding vector (NO_SOURCE_FILE:23)

The way I understand it the 'let' macro takes in a vector of bindings as its first argument and what I've done here is pass in two vectors instead of one.

In the bindings vector we need to ensure that there are an even number of forms so that each symbol can be bound to an expression.

I fixed this by putting the two vectors defined above into another vector:

(defn has-gaps? [list]
    (let [[no-nils] [(drop-while nil? list)]]
  no-nils))

We can simplify that further so that we don't have nested vectors:

(defn has-gaps? [list]
    (let [no-nils (drop-while nil? list)]
  no-nils))

The next step was to make 'no-nils' a function so that I could make use of that function when the list was reversed as well:

(defn has-gaps? [list]
    (let [no-nils (fn [x] (drop-while nil? x))]
  (no-nils list)))

I then wrote the rest of the function to reverse the list and then check the remaining list for nil:

(defn has-gaps? [list]
    (let [[no-nils] [(fn [x] (drop-while nil? x))]
          [nils-removed] [(fn [x] ((comp no-nils reverse no-nils) x))]]
  (some nil? (nils-removed list))))

The 'comp' function can be used to compose a set of functions which is what I needed.

It seemed like the 'nils-removed' function wasn't really necessary so I inlined that:

(defn has-gaps? [list]
    (let [no-nils (fn [x] (drop-while nil? x))]
  (some nil? ((comp no-nils reverse no-nils) list))))

The function can now be used like this:

user=> (has-gaps? '(1 2 3))
nil
user=> (has-gaps? '(nil 1 2 3))
nil
user=> (has-gaps? '(1 2 3 nil))
nil
user=> (has-gaps? '(1 2 nil 3))
true

I'd be intrigued to know if there's a better way to do this.

I initially incorrectly thought the 'contains?' function was the one that I wanted:

(contains? '(1 nil 2 3) nil)
=> false

I thought it would work the same as the Java equivalent but that function actually checks whether a key exists in a collection rather than a value. It's more useful when dealing with maps.

There's more discussion on the consistency of the API on the mailing list.

Luckily the documentation guides us towards the 'some' function:

My first attempt was to write an anonymous function to check if there was a 'nil' in the list:

(some #(= % nil) '(1 nil 2 3))
=> true

(some #(= % nil) '(1  2 3))
=> nil

fogus showed me an even better way by making use of the built in 'nil?' function:

(some nil? '(1 nil 2 3))

Another approach would be to make use of the Java 'contains' method as Philip Schwarz pointed out:

(.contains '(1 nil 2 3) nil)
=> true

I noticed that when you use Java methods in Clojure with collections then the result will either be 'true' or 'false' whereas when you use Clojure built in functions then it's more likely to be 'true' or 'nil'.

I guess this is linked to the idea that 'nil' is false in Clojure so it doesn't make much difference what the return value is.

When I'm using a language I've got into the habit of just trying out the API in the way that I expect it to work rather than paying a lot of attention to what the API documentation says.

I think this is something I'll need to work out to avoid much frustration!

The meaning of parentheses

Much like Keith Bennett I'm not used to parentheses playing such an important role in the way that an expression gets evaluated.

As I understand it if an expression is enclosed in parentheses then that means it will be evaluated as a function.

For example I spent quite a while trying to work out why the following code kept throwing a class cast exception:

(if (true) 1 0)

If you run that code in the REPL you'll get the following exception because 'true' isn't a function and therefore can't be applied as such:

java.lang.ClassCastException: java.lang.Boolean cannot be cast to clojure.lang.IFn (NO_SOURCE_FILE:0)

If we don't want something to be treated this way then the parentheses need to disappear!

(if true 1 0)

Truthyness

Somewhat related to the above is understanding which expressions evaluate to 'true' or 'false'.

I'm told there are some edge cases but that as a general rule everything except for 'false' and 'nil' evaluates to true.

I think that's an idea which is more common in languages like Ruby but I'm not yet used to the idea that we can something like this and have it execute:

(if "mark" 1 0)

In C# or Java I would except to have to compare "mark" to something in order for it to evaluate to a boolean result.

It seems like a really neat way to reduce the amount of code we have to write though so I like it so far.

Character Literals

I've been working through Mark Volkmann's Clojure tutorial and in one example he defines the following function:

(def vowel? (set "aeiou"))

I wanted to try it out to see if a certain character was a vowel so I initially did this:

=>(vowel? "a")
nil

"a" is actually a string though which means it's an array of characters when what we really want is a single character.

I thought the following would be what I wanted:

(vowel? 'a')

Instead what I got was the following exception:

java.lang.Exception: Unmatched delimiter: )

This one just turned out to be a case of me not reading the manual very carefully and actually the following is what I wanted:

=> (vowel? \a)
\a

It's a simple function to countdown from a given number to zero and then return that sequence.

This was one of the examples from the book:

(defn countdown [result x] 
  (if (zero? x)
    result
    (recur (conj result x) (dec x))))

That function could then be called like this:

(countdown [] 5)

I wanted to see what the function would look if we didn't have the empty vector as a parameter.

From playing around with F# and Scala my first thought would be to write the function like this:

(defn count-down [from]
  (defn inner-count [so-far x]
    (if (zero? x)
      so-far
      (inner-count (conj so-far x) (dec x))))
  (inner-count [] from))

As the book points out a bit further on, Clojure doesn't perform automatic tail call optimisation so we end up with a stack overflow exception if we run the function with a big enough input value.

Clojure does optimise calls to 'recur' so it makes more sense to use that if we want to avoid that problem.

This is an example which makes use of that:

(defn count-down [from]
  (defn inner-count [so-far x]
    (if (zero? x)
      so-far
      (recur (conj so-far x) (dec x))))
  (inner-count [] from))

Looking through the Clojure mailing list at a similar problem I noticed that one of the suggestions was to arity overload the function to include an accumulator.

(defn count-down
  ([from]
    (count-down [] from))
  ([so-far from]
    (if (zero? from)
      so-far
      (recur (conj so-far from) (dec from)))))

Written this way it feels a little bit like Haskell or Erlang but probably not idiomatic Clojure.

Anyway on the next page Halloway shows a better way to do this with much less code!

(into [] (take 5 (iterate dec 5)))

I noticed that in Scala the idea of using 'take' and 'drop' on streams of values seems to be quite popular so I'm intrigued as to whether I'll find the same thing with Clojure.