25 May 2013
Comparing Clojure Serialization Libraries
I recently had cause to investigate serialization options for Clojure data structures. I came up with four contenders: Carbonite, Deep Freeze, Nippy and the standard EDN reader from Clojure 1.5.1. Out of curiosity, I also decided to benchmark Cheshire’s JSON and SMILE serialization, to see how it compared.
Right off the bat I had to discard Deep Freeze, as it managed to fail on some very simple data structures:
(defn dp-round-trip [x]
(deep-freeze/thaw-from-array (deep-freeze/freeze-to-array x)))
(dp-round-trip {:x '(1 (2)) :y '(3 (4))})
=> {:y (3 (1 (2))), (4) :x}
But fortunately all the other libraries worked without issue. I benchmarked them with Criterium, and you can see the results below:
Unsurprisingly Clojure’s EDN reader and pr-str function were the
least performant, but the gap wasn’t exactly huge either, so it was
better than I was expecting.
Also somewhat unsurprisingly, the Cheshire SMILE binary format came out ahead of the Clojure binary serialization libraries. Since JSON/SMILE supports a smaller subset of data structures, this was about what I had expected.
There wasn’t much to choose between the Clojure binary serialization libraries, Carbonite and Nippy, in terms of performance. Carbonite seemed to edge out by a few microseconds in the majority of tests, but there was no significant difference. However, in terms of ease of use, Nippy’s syntax has the edge:
;; Nippy
(nippy/freeze-to-bytes data)
;; Carbonite
(let [registry (carbonite.api/default-registry)]
(carbonite.buffer/write-bytes registry data))
Carbonite has also been forked several times, and doesn’t appear to have an official fork anymore. The most up-to-date one was a repository maintained by Sam Richie from Twitter.
Summary
So my final choice was naturally Nippy, which as well as being quick, supports an impressive array of data structures with a very straightforward syntax. However, if I only wanted to serialize very simple data structures, and didn’t mind losing distinctions between keywords and strings, I’d have to think hard about using Cheshire’s SMILE encoding.