technical blog

25 May 2013

Comparing Clojure Serialization Libraries

I recently had cause to investigate serialization options for Clojure data structures. I came up with four contenders: Carbonite, Deep Freeze, Nippy and the standard EDN reader from Clojure 1.5.1. Out of curiosity, I also decided to benchmark Cheshire’s JSON and SMILE serialization, to see how it compared.

Right off the bat I had to discard Deep Freeze, as it managed to fail on some very simple data structures:

(defn dp-round-trip [x]
  (deep-freeze/thaw-from-array (deep-freeze/freeze-to-array x)))

(dp-round-trip {:x '(1 (2)) :y '(3 (4))})
=> {:y (3 (1 (2))), (4) :x}

But fortunately all the other libraries worked without issue. I benchmarked them with Criterium, and you can see the results below:

Unsurprisingly Clojure’s EDN reader and pr-str function were the least performant, but the gap wasn’t exactly huge either, so it was better than I was expecting.

Also somewhat unsurprisingly, the Cheshire SMILE binary format came out ahead of the Clojure binary serialization libraries. Since JSON/SMILE supports a smaller subset of data structures, this was about what I had expected.

There wasn’t much to choose between the Clojure binary serialization libraries, Carbonite and Nippy, in terms of performance. Carbonite seemed to edge out by a few microseconds in the majority of tests, but there was no significant difference. However, in terms of ease of use, Nippy’s syntax has the edge:

;; Nippy
(nippy/freeze-to-bytes data)

;; Carbonite
(let [registry (carbonite.api/default-registry)]
  (carbonite.buffer/write-bytes registry data))

Carbonite has also been forked several times, and doesn’t appear to have an official fork anymore. The most up-to-date one was a repository maintained by Sam Richie from Twitter.


So my final choice was naturally Nippy, which as well as being quick, supports an impressive array of data structures with a very straightforward syntax. However, if I only wanted to serialize very simple data structures, and didn’t mind losing distinctions between keywords and strings, I’d have to think hard about using Cheshire’s SMILE encoding.