Annual Update on State of Java JSON data binding performance
Yes, 'tis the season for performance measurements again: last time I
covered this subject about a year ago with "JSON
data binding performance (again!)".
During past 12 months
many of the tested libraries have released new versions; some with high
hopes for performance improvements (Gson 1.6, for example, was rumored
to have faster streaming parser). So it seems prudent to have another
look at how performant are Java JSON data binding libraries currently
available.
1. Libraries tested
Libraries tested are the same as last time; with versions:
- Jackson 1.6 (was 1.2)
- Gson 1.6 (was 1.4)
- Json-tools (core, 1.7) (no change)
- Flex-json 2.1 (was 1.9.1)
(for what it's worth, I was also hoping to include tests for "json-marshaller", but lack of documentation coupled with seeming inability to parse directly from stream or even String suggested that it's not yet mature enough to be included)
2. Test system
I switched over to the light side at home (replaced my old Athlon/Linux box to a small spunky Mini-Mac, yay!), so the test box has a 2.53 GHz Intel Core 2 Duo CPU. This is somewhat box; it seems to be about 4x as fast for this particular task. Test is single-threaded, so it would be possible to roughly double the throughput with different Japex test setup; however, test threads are CPU-bound and independent so seems to be little point in adding more concurrency.
Test code is available from Woodstox SVN repository (under "staxbind" folder), and runs on Japex. Converters for libraries are rather simple; data consists of medium-sized (20k) documents for anyone interested in replicating results.
3. Pretty Graphs
Ok, here is the main graph:
and for more data, check out the full results.
4. "But what does it MEAN?"
As with previous tests, upper part of double-graph just indicates amount of data read and/or written (which is identical for first three, but flex-json seems to insist including some additional type metadata), which can be ignored; the lower-graph indicates through-put (higher bar means faster processing) and is the interesting part.
There are three tests; read JSON (into data object(s)), write JSON (from data object(s)) and read-then-write which combines two operations. I use last result, since it gives reasonable approximation for common use with web services where requests are read, some processing done, and a response written.
From graph it looks like results have not changed a lot; here is the revised speed ratio, using the slowest implementation (Gson) as the baseline:
| Impl | Read (TPS) | Write (TPS) | Read+Write (TPS) | R+W, times baseline |
|---|---|---|---|---|
| Jackson (automatic) | 7240.202 | 9161.873 | 4023.464 | 14.75 |
| FlexJson | 721.743 | 1402.848 | 462.594 | 1.69 |
| Json-tools | 524.119 | 1007.068 | 341.123 | 1.25 |
| GSON | 714.106 | 462.935 | 272.637 | 1 |
5. Thoughts?
Not much has changed; Jackson is still an order of magnitude faster than the rest, and relative ranking of the other libraries hasn't changed either.
