Saturday, July 10, 2010

Every day JSON work: comparing JSON documents for equality ("are these JSONS same?")

One JSON task that may seem trivial but is not is that of checking whether two given JSON documents are equal: that is, have same structure and values, and ordering of array values is same. It is not sufficient to use String equality comparison because while equality of Strings does mean contained JSON is equal, differing Strings does not mean they are not equal, since:

  • It is possible to quote characters differently using \uXXXX mechanism
  • Order of properties of JSON objects is undefined (at conceptual level); meaning that physical ordering may differ while conceptually JSON is still identical

So how should one compare equality? With Jackson, simplest way is to construct JSON tree (JsonNode) for documents to compare and just use JsonNode.equals()! Like so:

  ObjectMapper mapper = new ObjectMapper();
  JsonNode tree1 = mapper.readTree(input1);
  JsonNode tree2 = mapper.readTree(input2);
  boolean areEqual = tree1.equals(tree2);

I realized that this (fact that JsonNode.equals() works "as expected", doing deep value comparison) is not documented outside of Javadocs (it will be now, I will add something at FasterXML Jackson Wiki). But hopefully it was already known by experienced Jackson users. Either way, here's how JSON content comparison can be done easily and reliably.

Monday, May 17, 2010

Finally: Java JSON Schema validator; based on Jackson

Ok here are some good news for Java developers who use (or might like to use) JSON, but are bothered by lack of data format validation options: Nicolas Vahlas has written the first Java JSON Schema validator, and it is available from Gitorious as project json-schema-validator.

I have not yet have time to dig deep into it, but there all signs for it being so-called Good Stuff. Not just because it is based on Jackson -- although proper reuse of existing solid components is a general good sign -- but because description gives an idea of author being someone actually knows what he is doing.

So please check it out if said functionality seems at all interesting: the best way to ensure it becomes a first-class tool (and maybe even help JSON Schema standard improve along the way) is to use it, give feedback, and get the whole flywheel-of-virtue (aka virtuous cycle) thing going on. That's how things like Jackson and Woodstox became good: feedback is the amplifier of open source productivity.

Ok, enough raving: I'm off to get the sources for bit of closer look. :-)

Thursday, May 13, 2010

Happy First Birthday, Jackson the JSON processor!

Time sure flies when you are having fun: I just realized thatJackson JSON processor had its first birthday, if we use its 1.0 release's date (of May 9th, 2009) as the reference point.

And how far has Jackson community come? Here are some interesting statistics, mostly provided by good old Google Analytics:

  • 10,000 monthly visits to Jackson wiki; with 33,000 page views (getting close to half million page views per year)
  • 5,000 monthly visits to Jackson Javadocs; with 17,000 page views (200k on annual basis)
  • close to 4,500 downloads (== visits toJackson Download page) -- meaning about 50k direct downloads on annual basis; and many more if access via Maven was included (wonder how could this be tracked)

It is fascinating how far and how fast Jackson has gone: currently its downloads beat Woodstox by factor of 4:1, for example, although Woodstox has been the leading high-performance Java XML parser for years now -- difference has to do with both declining popularity of XML (compared to JSON), and because Woodstox is more commonly bundled with frameworks, whereas Jackson is quite pleasant to use directly. Furthermore looking at trends, it looks like "Jackson traffic" has doubled or tripled over past 8 months or so.

Of course, traffic numbers are not quite as interesting as more applicable things -- number of users, and even more importantly, amount of functionality Jackson offers. But still, above numbers give some guidance as to growing popularity of Jackson, as well as of JSON as THE dominant data format on Java platform.

Wednesday, March 17, 2010

Jackson 1.5: Polymorphic Type Handling, first steps

Now that Jackson 1.5 is out (ready for download?), it's good to start reviewing some candies from the goody bag. While 1.5 is, technically speaking, a "minor" release it is nothing but grand judging by number of added features .
In a way Jackson mirrors development of JDK: Jackson version 1.2 and 1.5 have contained tons of new stuff, whereas 1.3 and 1.4 have been more about incremental steady improvements and bug fixes.

Let's start with the top requested new feature: Polymorphic Type Handling.

1. Polymorphic Types: not just for serialization any more!

First things first: polymorphic types just refer to cases where there is a single declared property type (say, Animal), but multiple concrete subtypes (like Dog and Cat). Basic OO stuff, implemented in Java by sub-classes and interface implementation. Nothing unusual there.
Challenge is actually that of mapping this cleanly to data formats: as with relational model (SQL) and hierarchic model (XML), JSON has no native concept of inheritance (XML has neither, in itself; XML Schema does add support however).

Jackson has actually supported serialization of all types, including polymorphic subtypes, since first data binding enabled versions (0.9.5). This because Jackson by default uses actual runtime type when determining serializer to use (although it can be configured to only static, declared type, too) -- which is different from what JAXB does -- and thus has no trouble writing out all properties sub-type has.

But the problem is, without additional type information, it is not possible to determine intended subtype when deserializing, when there are multiple subtypes to choose from: when property has declared type of 'Animal', there is no way to know whether it should be reconstructed as "Dog" or "Cat". This is where PTH implementation comes in.

Obvious answer is that additional type information must be added in serialized JSON data to allow this later reconstruction. But there are multiple considerations as to how this should be done, including:

  • What kind of type information should be included (Java class name? Logical type name? something else?)
  • How (where) should this type information be included (as a property? using wrapper constructs?)
  • Do we always have to include this type information? (doing so adds unnecessary bloat in cases where type information is redundant)

Jackson 1.5 tackles these (and some other minor concerns) in an efficient, configurable, and I hope, elegant way.

2. Enough theory: show me the code!

Ok: chances are that you are not all THAT interested in details of how implementation was tricky, or what philosophical considerations went on in determining what to implement and how. So let's start with some toy code.

Considering case of a polymorphic class; say... Person base class, and Employee and Customer sub-classes (I have written an "Animal example" at FX wiki already, let's get some variety), and using a wrapper class such as:

public class Contact {
 public Person initiator; // who initiated contact
 public Person receiver; // with whome?
}

and Person (etc) classes as follows

public abstract class Person {
 String name;
}

public class Employee {
 long employeeId;
}

public class Customer {
 String ssn;
 ServiceLevel serviceLevel;
}

What do we need to do to make Contact properly deserializable? By default, no additional type information would be added, and deserialization would fail

There are two main ways to do this, covering slightly different use cases, and offering different levels of configurability.

3. Type Info, method #1: explicit annotations

First method is to add explicit @JsonTypeInfo annotation in the declared type (or its supertypes):

@JsonTypeInfo(use=JsonTypeInfo.Id.CLASS, include=JsonTypeInfo.As.PROPERTY, property="@class")
public abstract class Person {
  //...
}  

which would basically enable addition of type information for any instances of Person, as well as its subtypes. With above configuration -- include fully-qualified Java class name as type identifier; include it as main-level additional (meta-)property using property name "@class" -- output might look something like:

{
  "@class" : "com.fasterxml.sample.Employee",
  "name" : "Billy Bacon",
  "employeeId" : 126509
}

(with additional type information bolded)

So far so good: it is bit extra work to sprinkle annotations (which can be added via mix-ins, of course), but nothing too bad. As long as you know types in advance. And you can configure handling pretty well (check out Javadocs for more details for @JsonTypeInfo, related; or if I get that far, later entries on advanced topics)

4. Type info, method #2: default typing

But there is another way, which requires much less code, although offering bit less configurability. Specifically, you can enable so-called "default typing" for ObjectMapper by:

  mapper.enableDefaultTyping(); // defaults for defaults (see below); include as wrapper-array, non-concrete types
  mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL, JsonTypeInfo.As.WRAPPER_OBJECT); // all non-final types

which would then apply default type information (type id is always fully-qualified Java class; inclusion method is customizable but defaults to "wrapper array") to all types (classes) that fulfill applicability criteria: by default it would be Object.class and non-concrete types (abstract classes, interfaces). So in our case this would work, given that Person is an abstract class: and if it wasn't, we could use "ObjectMapper.DefaultTyping.NON_FINAL" to include type information on all non-final classes.

5. Is that All?

No, not at all. This is just the simplest way to start safely handling polymorphic types like, say, List<Object>.

What else there is to find is good subject for follow-up entries, but here are some teasers:

  • Java class name is not a good choice when you have non-Java clients or services to talk to -- if so, you can use "logical type name" (see @JsonTypeName and @JsonSubTypes for more info); much like what JAXB does. Or, even further, you can have fully customized type id converters! (as long as they can convert to/from JSON Strings)
  • If you don't like using main-level property names (maybe there's a conflict), you can use other styles; "wrapper Object", "wrapper Array"
  • If you like default typing, but don't like its selection criteria, you can create your own: just sub-class default TypeResolverBuilder, assign it to your ObjectMapper.
  • Similarly, you can define fully custom type information handler, using @JsonTypeResolver annotation -- this will be nicely integrated with the whole system, you only have to provide actual inclusion, but it will be called for any existing types.

But enough for now... more on 1.5 later.

Wednesday, March 10, 2010

Users of Jackson, Unite!

I know this is something that users of Jackson JSON library must have been craving for longest time: their Very Own Social Network (no? you'd rather have handling of cyclic types? d'oh!)
Wait no longer: enter jackson-users.ning.com. Place for developers who know the Right Tool for slicing and dicing JSON on Java platform (or possible, any platform).

So what's the point? Besides my own curiosity (and, shall we say, desire to "eat your own dog food"; check out author's profile for more info) -- which was a big factor -- I thought it might be one more way to get more of community feel. Mailing lists are good, blogs and wikis too, but a gathering place with bit of all of them might be even better.
We shall see how it all works out -- and thanks to lower threshold for participating (since it's easier to give access, let members edit and add things), you can be important part of making it Work Right.

See you there: I'll try sending a nice Virtual Gift for member number 100... :-)

ps. What about blogs over there, and here? My current thinking is that "Jackson Users blog" will focus strongly on Jackson (JSON, data binding, frameworks that use it) issues; whereas CowTown blog will have wider coverage. So it should be more of specialization versus general interest.

Groovy/Gaelyk + Google App Engine + Jackson == Nice Restful Framework

Here's another interesting article: "RESTing with JSON in Gaelyk". Looks like Jackson usage on "Google platforms" (Android, GAE) is becoming more of a maintsream thing. This is great for multiple reasons, and multi-platform support is one of (many) goals of Jackson.
But just having goals matters little if they are not met. Fortunately, thanks to active user/adopter base, goals are being verified over time; and where there are problems, they usually get resolved in timely matter.

Friday, February 26, 2010

Jackson, compliments du jour, en francaise

I wish my french skills were little bit more refined (6 months of suggestopedic teaching, 2 hours per week apparently is not enough to get more than general idea of text :) ). But from what I gather, Android pour l’entreprise – 6 – Oubliez Gson, Jackson rocks my world! seems to have generally positive outlook on Jackson for JSON processing on Android platform (oubliez meaning "forget"... it's good that cognac ratings help my language skills here!). And apparently much of this is due to Android VM (Dalvik) being somewhat sensitive to GC-induced stress; so Jackson's focus on efficiency (not just speed, but focus on simplicity of code, trying to avoid extraneous intermediate storage and code) really pays off.

It is great that a library can be versatile enough to perform well on wide set of platforms; and it is absolutely marvellous that there are users who put Jackson to good use, and let others know what works and what doesn't.

Anyway, thought I'll share this; Android developers in practicular might find this interesting. Also: author of the article has suggested couple of good improvements to Jackson, too, to make things work even better in future.

Tuesday, December 22, 2009

Another Jackson adopter: Spring-json

As has been reported earlier, Spring 3.0 (and Spring-json module, specifically) now has a Jackson-based JSON view variant: see MappingJacksonJsonView and spring-json view comparison. Comparison seems reasonable, mentioning annotation-based configurability and performance as strong points of jackson-based view.

UPDATE, 28-Dec-2009: One more for the road: Restlet is also including Jackson-based extension.

Sunday, December 20, 2009

Jackson 1.4: more control over writing JSON, improved interoperability

First things first: if you haven't noticed yet, Jackson 1.4.0 was just released.

This release focuses mainly on writer (serialization) side, but there are also continuing improvements to interoperability. I will review main improvements below.

1. JSON generation improvements

1.1 Ignoring properties

A new annotation, @JsonIgnoreProperties, allows:

  • omitting serialization of listed properties: @JsonIgnoreProperties({ "secretField", "internalProperty" }); listed properties will not be included in JSON output
  • omitting listed properties from being deserialized; if encountered they are just ignored even if there is a setter for them (regardless of whether setter is marked to be ignored or not)
  • ignoring all unknown properties for annotated class during deserialization (similar to disabling DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES), but only affects instances of annotated class. This is done with property "ignoreUnknown": @JsonIgnoreProperties(ignoreUnknown=true) (note: has no effect on serialization)

1.2 JsonView

New @JsonView annotation allows defining logical views for serialization: sets of properties to be written out for given view.

Let's consider a simple example, where we want to control amount of information written out, based on, say, user's credentials. To define 3 classes of properties, we can define views (more about identification below):

  class Views { // container for View classes
static class Public { }
static class ExtendedPublic extends PublicView { }
static class Internal extends ExtendedPublicView { }
  }
  
And to define access levels for our info class, we would do
public class PersonalInformation { // Bean that uses Views to define subsets of properties to include // Name is public @JsonView(Views.Public.class) String name; // Address semi-public @JsonView(Views.ExtendPublic.class) Address address; // SSN only for internal usage @JsonView(Views.Internal.class) SocialSecNumber ssn; }

Given this set up, we would define View to use for serialization by:

  objectMapper.writeValueUsingView(out, infoInstance, Views.Public.class); // short-cut
  // or full version:
  objectMapper.getSerializationConfig().setSerializationView(Views.Public.class);
  objectMapper.writeValue(out, beanInstance); // will use active view set via Config
  // (note: can also pre-construct config object with 'mapper.copySerializationConfig'; reuse configuration)

Which in this particular case would only contain "name" property. If we had used view Views.ExtendedPublic.class, we would have gotten 2 fields; and with Views.Internal.class, all 3.

Views are identified by classes: you can either create specific marker classes, or use existing classes. Views use inheritance indicated by class structure: such that a view is considered a sub-view of another view if it extends that view. Child views include properties that parent views include.

For more description see JsonView wiki page.

1.3 Ordering properties output

Another new annotation, @JsonPropertyOrder, allows defining complete and partial field orderings:

  • You can define explicit ordering by listing properties as annotation value: @JsonPropertyOrder({ "id", "name" }) would ensure that "id" and "name" are output before any other properties during serialization
  • You can specify that anything not explicitly ordered will be output in alphabetic order: @JsonPropertyOrder(alphabetic=true)
  • Without these settings order is undefined because JVM does not expose order of underlying fields or methods (but see note below)

Beyond these definitions, 1.4 also guarantees that properties used with @JsonCreator annotations (constructors, factories) are serialized before other properties, unless there are explicitly ordered properties (which will have priority). This change was to optimize @JsonCreator property usage: ideally these properties should be readable before other properties -- although JSON logical model does not provide for such guarantee, Jackson will try to do its best to make ordering optimal.

2. Interoperability improvements

Goal of interoperability improvement is to make Jackson a "universal" JSON data binding tool on JVM. That is: we hope to make Jackson usable from other JVM languages, not just Java -- already one can use it from quite a few (reported to work from Groovy, Clojure), and hopefully supporting others like Scala in near future (Scala lists are not well handled, yet) -- as well as interoperate nicely with most common data libraries.

To this end, there is now default support (i.e. no need for custom converters or mix-in annotations) for following data types:

  • DOM (xml) trees: properties declared as DOM Document, Element and Node will now be properly serialized to, and deserialized from JSON Strings. Useful if you want to embed XML as JSON properties (sometimes good for interoperability)
  • Joda DateTime type, and mechanism to easily add more types as needed (file a Jira request if you need more!)
  • Handling of javax.xml types that some platforms lack (Android and GAE have had some issues) much improved so that they are dynamically and reliably added, if underlying types are present.

Last point should also make it yet easier to make Jackson run on new "subset platforms"; containers that support subset of JDK 1.5 (or have issues with some parts).

3. Plans for 1.5

So what next? 1.4 release can be thought of a "minor" minor release, similar to 1.3; compared to fundamentally new functionality of 1.2 (mix-in annotations, JsonCreators), 1.3 and 1.4 have consisted of smaller (but more numerous) evolutionary incremental improvements. In many ways, Jackson releases have mirror JDK releases, come to think of that.

Anyway: the next Big Thing will be "Polymorphic Deserialization", which is by far the most requested feature. That is, ability to deserialize instances of correct types, even in absence of static type information (declared type of, say, List<Object> could still be deserialized to contain whatever actual type of serialized instance was). Getting this done is important in itself, but the most important aspect in my mind is to Do It Right. This should not be a stop-gap solution, or something to rewrite in near future. It should be comprehensive, flexible and robust solution to the non-trivial problem. And plan is to do just that; now that other queued blocking issues (like that of finall getting much-requested JsonView done) have been dealt with.

Wednesday, December 16, 2009

More Jackson adoption: Mule/iBeans

As per this announcement, Mule is another major framework that has officially adopted Jackson for its JSON processing needs ("first we take JAX-RS... then we take ESB!"). That should be good for everyone involved. And more work for the development, polishing and fixing things that new flow of users brings in; as well as plenty more exposure for the project & processor.
That should keep the project honest, relevant, and hopefully producing diamond(s) -- without high pressure, all you'd have would be lump of coal. :-)

On a sort of related note; the latest tally of number of contributors (individuals named on 'release-notes/CREDITS') is 50. Not too shabby -- Jackson will pretty soon bypass Woodstox in most regards (see anearlier blog entry for context); probably not by LOC, but in most other measures.

Related Blogs

(by Author (topics))

Powered By

Powered by Thingamablog,
Blogger Templates and Discus comments.

About me

  • I am known as Cowtowncoder
  • Contact me at@yahoo.com
Check my profile to learn more.