Saturday, October 3, 2009

I/O write speed benchmark

At work we had some performance issues with one of our production boxes. I thought it was related to the I/O on the veritas file system which we have to use for high availability. In order to prove it I needed a tool to write 1K block and flush it, fast. I've searched around on the web and could not find anything that measures only writes, so I wrote one in Clojure. I had to make sure that it is fast and does not generate to much garbage in the main tight loop. New transient was a perfect choice for it. Transients only available in Clojure 1.1.0, so you will need to get the latest alpha.

The main loop is pretty straight forward.
(defn write-to-file [#^String file]
  (with-open [s (FileOutputStream. file)]
    (loop [i 0 r (transient [])]
      (if (< i 1e3)
        (recur (inc i) (conj! r (with-time
                                    (doto s
                                      (.write test-row)
                                      (.flush)))))
        (persistent! r)))))

Here is the gist of the rest.



I've used incanter to plot the data.
And this is one of the plots that I've got, when I ran it on my MacBook Pro.
As you can see there is quite a bit of jitter and that will affect latency. On our production Linux servers after tuning the file system, those jitters almost disappeared. The next step is to use Real-Time OS and RTSJ.

No comments: