Mark Needham

Thoughts on Software Development

Clojure: Not so lazy sequences a.k.a chunking behaviour

with 3 comments

I’ve been playing with Clojure over the weekend and got caught out by the behaviour of lazy sequences due to chunking – something which was obvious to experienced Clojurians although not me.

I had something similar to the following bit of code which I expected to only evaluate the first item of the infinite sequence that the range function generates:

> (take 1 (map (fn [x] (println (str "printing..." x))) (range)))
(printing...0
printing...1
printing...2
printing...3
printing...4
printing...5
printing...6
printing...7
printing...8
printing...9
printing...10
printing...11
printing...12
printing...13
printing...14
printing...15
printing...16
printing...17
printing...18
printing...19
printing...20
printing...21
printing...22
printing...23
printing...24
printing...25
printing...26
printing...27
printing...28
printing...29
printing...30
printing...31
nil)

The reason this was annoying is because I wanted to shortcut the lazy sequence using take-while, much like the poster of this StackOverflow question.

As I understand it when we have a lazy sequence the granularity of that laziness is 32 items at a time a.k.a one chunk, something that Michael Fogus wrote about 4 years ago. This was a bit surprising to me but it sounds like it makes sense for the majority of cases.

However, if we want to work around that behaviour we can wrap the lazy sequence in the following unchunk function provided by Stuart Sierra:

(defn unchunk [s]
  (when (seq s)
    (lazy-seq
      (cons (first s)
            (unchunk (next s))))))

Now if we repeat our initial code we’ll see it only prints once:

> (take 1 (map (fn [x] (println (str "printing..." x))) (unchunk (range))))
(printing...0
nil)
Be Sociable, Share!

Written by Mark Needham

April 6th, 2014 at 10:07 pm

Posted in Clojure

Tagged with

  • Beau Fabry

    Lazy sequences (and lazy evaluation in general) play *really* badly with side-effects. There’s a chapter in SICP about it. Must remember unchunk.

  • http://www.markhneedham.com/blog Mark Needham

    @beaufabry:disqus yeh I was told on twitter that you shouldn’t mix lazy evaluation with side effects. Is it purely because of the chunking thing or are there other reasons why it’s a bad idea?

  • Beau Fabry

    Pragmatically I think it’s fine in clojure because we only have lazy sequences not full lazy evaluation, so know where to watch out. But if it were a completely lazy-evaluated language then having side-effects would make the code too hard to reason about. Consider

    “`
    (def foo (atom 1))

    (let [arg1 (reset! foo 2)
    arg2 (reset! foo 3)]
    (println (str @foo arg2 arg1))
    “`