Mark Needham

Thoughts on Software Development

Clojure: Language as thought shaper

with 3 comments

I recently read an interesting article by Tom Van Cutsem where he describes some of the goals that influence the design of programming languages and one which stood out to me is that of viewing ‘language as a thought shaper’:

Language as thought shaper: to induce a paradigm shift in how one should structure software (changing the “path of least resistance”).

To quote Alan Perlis: “a language that doesn’t affect the way you think about programming, is not worth knowing.”

The goal of a thought shaper language is to change the way a programmer thinks about structuring his or her program.

I’ve been rewriting part of the current system that I’m working on in Clojure in my spare time to see how the design would differ and it’s interesting to see that it’s quite different.

The part of the system I’m working on needs to extract a bunch of XML files from ZIP files and then import those into the database.

From a high level the problem can be described as follows:

  • Get all files in specified directory
  • Find only the ZIP files
  • Find the XML files in those ZIP files
  • Categorise the XML files depending on whether we can import them
  • Add an additional section to good files to allow for easier database indexing
  • Import the new version of the files into the database

Clojure encourages a design based around processing lists and this problem seems to fit that paradigm very neatly.

We can make use of the ->> macro to chain together a bunch of functions originally acting on the specified directory to allow us to achieve this.

At the moment this is what the entry point of the code looks like:

(defn parse-directory [dir]                                                                                                                                                                           
  (->> (all-files-in dir)                                                                                         
       (filter #(.. (canonical-path %1) (endsWith ".zip")))                                                                                                                                                                 
       (mapcat (fn [file] (extract file)))                                                                         
       (filter (fn [entry] (. (entry :name) (endsWith ".xml"))))                                                                                                                                                 
       (map #(categorise %))))

The design of the Scala code is a bit different even though the language constructs exist to make a similar design possible.

The following are some of the classes involved:

  • ImportManager – finds the XML files in the ZIP files, delegates to DocumentMatcher
  • DeliveryManager – gets all the ZIP files from specified directory
  • DocumentMatcher – checks if XML document matches any validation rules and wraps in appropriate object
  • ValidDocument/InvalidDocument – wrap the XML document and upload to database in the case of the former
  • ValidationRule – checks if the document can be imported into the system

It was interesting to me that when I read the Scala code the problem appeared quite complicated whereas in Clojure it’s easier to see the outline of what the program does.

I think is because we’re trying to shoe horn a pipes and filters problems into objects which leaves us with a design that feels quite unnatural.

I originally learnt this design style while playing around with F# a couple of years ago and it seems to work reasonably well in most functional languages.

Be Sociable, Share!

Written by Mark Needham

July 10th, 2011 at 10:21 pm

Posted in Clojure

Tagged with

  • Anonymous

    import scala.collection.JavaConverters._

    case class CategorizedFile(f: File, tag: String)

    def extract(f: File) = Seq(f) //not implemented extract xmls from zip
    def categorise(f: File) = CategorizedFile(f, “tag”) //not implemented categorize

    val dir = “startDir”

    for{ file <- new File(dir).listFiles
         if file.getName.endsWith(".zip")
         xml <- extract(file)
       } yield categorise(xml)

    If you want to model it in a complicated way you can do this in scala.  But you can also "pipe" and filter in a for-comprehension.
    Of course you can also use the funny scalaz operators, that help you with error reporting etc..

  • Julian Birch

    This is exactly why I try to program in Clojure as well: it helps break your knee-jerk reactions on how to code something.

    A question about style, though:  Why do you prefer writing
    (map #(categorise %))
    (mapcat (fn [file] (extract file)))

    (map categorise)
    (mapcat extract)

    I would have regarded the latter as more clear.  Mind you, if there’s a problem with Clojure, it’s that it exaggerates my tendency to play code golf…

  • @187f43006a70762c7bda32bc86e1b3ac:disqus yeh I figured you could probably write it exactly the same way in Scala but it was interesting to notice that it wasn’t the way we immediately thought to write it. I think we might go and change that once we’ve got the Clojure version working.

    @google-f7af9c8034d5070fb2e8565c79c604b7:disqus no reason actually. A colleague was showing me his Clojure code and he writes it the same way as you. As you say – much clearer. I think I’ll write mine like that from now on, thanks for the advice!