CowTalk

Users of Jackson, Unite!

Wed, 10 Mar 2010 17:39:41 -0800

I know this is something that users of Jackson JSON library must have been craving for longest time: their Very Own Social Network (no? you'd rather have handling of cyclic types? d'oh!)
Wait no longer: enter jackson-users.ning.com. Place for developers who know the Right Tool for slicing and dicing JSON on Java platform (or possible, any platform).

So what's the point? Besides my own curiosity (and, shall we say, desire to "eat your own dog food"; check out author's profile for more info) -- which was a big factor -- I thought it might be one more way to get more of community feel. Mailing lists are good, blogs and wikis too, but a gathering place with bit of all of them might be even better.
We shall see how it all works out -- and thanks to lower threshold for participating (since it's easier to give access, let members edit and add things), you can be important part of making it Work Right.

See you there: I'll try sending a nice Virtual Gift for member number 100... :-)

Groovy/Gaelyk + Google App Engine + Jackson == Nice Restful Framework

Wed, 10 Mar 2010 17:34:06 -0800

Here's another interesting article: "RESTing with JSON in Gaelyk". Looks like Jackson usage on "Google platforms" (Android, GAE) is becoming more of a maintsream thing. This is great for multiple reasons, and multi-platform support is one of (many) goals of Jackson.
But just having goals matters little if they are not met. Fortunately, thanks to active user/adopter base, goals are being verified over time; and where there are problems, they usually get resolved in timely matter.

Jackson, compliments du jour, en francaise

Fri, 26 Feb 2010 21:18:38 -0800

I wish my french skills were little bit more refined (6 months of suggestopedic teaching, 2 hours per week apparently is not enough to get more than general idea of text :) ). But from what I gather, Android pour l’entreprise – 6 – Oubliez Gson, Jackson rocks my world! seems to have generally positive outlook on Jackson for JSON processing on Android platform (oubliez meaning "forget"... it's good that cognac ratings help my language skills here!). And apparently much of this is due to Android VM (Dalvik) being somewhat sensitive to GC-induced stress; so Jackson's focus on efficiency (not just speed, but focus on simplicity of code, trying to avoid extraneous intermediate storage and code) really pays off.

It is great that a library can be versatile enough to perform well on wide set of platforms; and it is absolutely marvellous that there are users who put Jackson to good use, and let others know what works and what doesn't.

Anyway, thought I'll share this; Android developers in practicular might find this interesting. Also: author of the article has suggested couple of good improvements to Jackson, too, to make things work even better in future.

Fool's Gold, Standard(s)

Mon, 22 Feb 2010 23:52:11 -0800

Here's something new: some good reading ("Ron Paul's money plan is far from golden") at CNN (sic!): this time about nostalgic folly of returning to the "gold standard". It is surprising that someone whose intellectual aspirations are bit above those of his supporters (ok, granted, that's a low bar), one would be so mistaken about realities of tying national currency into amount of precious metal(s) central bank physically has. Maybe this is why central banks are generally lead by people with economic education and experience, and not physicians.

I mean, yes, from laymanperspective, it would seem nice if that green paper that gets printed on would actually have collateral. But impracticality of full collateralization should be obvious: you don't need much of a thought-exercise to see how and why it would fail; and from that point on, to backtrack and see why this realization (when shared by people who control flow of money) means that attempt would be a self-fulfilling failure. And if we were unlucky, slowly cooking but colossal-cluster-magnitude failure.

In addition to the great depression that is obviusly mentioned in the articles, proponents of "strong currency" managed to starve millions of people to death during late 1800s. I am most familiar with a somewhat starvations in Finland (there were 2 instances): globally speaking these were just blimps on radar (sice the whole country population was barely in millions), but death rate from starvation actually exceeded that of world wars... and all that so that central bank could protect value of currency, by not loaning money (or subsidize seeds), managing to keep central bank in black, and peasants hungry or dead. Famine was orginally triggered by weather, of course, but the catastrophe could have been averted by government action. And in similar vain, in more recent memory, depression of early 90s (in Finland) was also deepened by later crop of strong currency proponents, who tried (ultimately in vain) to keep the currency strong by trying to avoid devaluation. In the end they had to let it float anyway (causing run-off devaluation by something like 30% in a week), but so late that much of damage was already done. Fortunately no one starved to death on account of this failure, although unemployment rate tripled closer to 20%.

I am sure there are many more examples; and some EU countries are currently experiencing related challenges (now that they are forced to exercise certain discipline after screwing up their finances before realizing it must be done).

These examples are closely related to "gold standard" part, in that there is simplistic view of nations having to balance their check books on very short term. This is neither practical nor beneficial. And trying to force it to be done does not make it any more practical, beneficial or wise.

And yet -- it seems that principled fools never let facts get in a way of intuitive theories. So I am just waiting for a grand unified theory that binds together ideas of tax-cut for riches, return to the gold standard, and the idea that poor people caused depression (due to welfare costs allegedly being a major contributor to this whole meltdown -- don't ask me how the mechanism is supposed to play; apparently this claim is getting some consideration in tea bagger circles).

NatGeo & Places of Amazing Beauty, well beyond what you could imagine: Hebrides

Mon, 15 Feb 2010 22:30:41 -0800

Ok, it would probably be time to actually write about something technical -- like, say, ultra-modern Polymorphic Type Handling system thatJackson 1.5 will have ("Lexus of Java Data Binders!") -- but after cranking out three-digit number of entries last year (at least 50% of them technical), I will cut me some slack here. Still, without typing some prose I am worried that my fingers might evolve away, so let's consider something beyond realm of technical stuff.

As usual my inspiration comes from a high-quality affordable US magazine called National Geographic. That's hardly news. But after reading about all these exotic remote locations like Madagascar, or that magical forest/mountain area in China of which name now escapes me, I was somewhat surprised to find that Hebrides (that set of islands on one side of Scotland) not only produces heavenly malt whiskies, but also sports sceneries more celestial than Sports Illustrated swimsuit edition. And it's not just that I am getting old: beauty may lie in eye of beholder, but there are certain things that are beautiful regardless of their age (or observer's age). Some landscapes of nature are like that, with or without curvature.

So if you get a chance, have a look at article "Edge of the World", and especially the slide show of photos that accompany it. And picture yourself in one of settings, perhaps sipping a warm glass of Talisker or Laphroaig (latter when visiting, say, Fingal's Cave; former at around those abandoned stone buildings at Hirta, St. Kilda?).

If it wasn't for the fact that kids would probably bore to death in 10 minutes flat, I would be ready to migrate to Hebrides. At least until getting there, and sun sets down & wind sets in and all those little practical details. But in my imagination I am already packing!

Good News, 2009/Jan: Farming Cool Again, Urban-Agro saving Detroit?

Sun, 10 Jan 2010 16:19:13 -0800

Color me goofy, but I believe I have written something relevant to this latest interesting article in Fortune, "Can Farming Save Detroit?"... lessee.

Ah, yes, back in October I did point to a related Sci-Am article ("The Rise of Vertical Farms" in "Good news is news too...") that presented and discussed the concept of ultra-modern urban farming. And now Fortune has something more concrete to write about, regarding planned development in Detroit, of all places. It (Urban agriculture in general and the particular project in specific) actually seems to make lots of sense -- I did not know, for example, that there are no chain super markets in Detroit; and conversely there is lots of vacant land, mostly (formerly) residential (meaning no toxic industrial waste). There's no point in just repeating all potential benefits here: just suffice it to say that if done correctly, it could be a significant win-win-win situation, from environmental as well as quality-of-food aspect. That it would be done by someone rather unexpected is probably a good thing in itself: seems like lately people that are most unexpected to stand up and do something do just that (like mr. Boone with wind mills etc. etc.). Desire to leave positive legacy is a strong driving force, and continues to influence development in US: it's not just Ivy League colleges that get founded by elderly billionaries (and yet some numbskulls are trying to kill inheritance tax -- how freakishly stupid is that? -- but I digress). Saving the world does kind of top the list of things to do, if you want to leave such a legacy.

In addition to being interesting in and of itself, I find it fascinating how ideas enter mainstream gradually. I am pretty sure that Time and Newsweek pick this up in a month or two; then followed by broadcast news (.... sloooowly), and eventually daily print papers (once everyone is about aware of the thing). I guess this is one more thing to read one of these affordable high-quality magazine US market is (still) blessed with, like Scientific American, Fortune and National Geographic Magazine (and there are plenty more -- these just happen to be ones I have time to read): you get to learn about important ideas, concepts and developments slightly ahead of most others. And if you are even more time-constrainted than I am, well, you can just skim Time or Newsweek, and still be well ahead the curve.

Of course, it could also be that in a decade or two we may be reading articles like this one same way we do now for all those "by 2000, everyone has a personal rocket ship and eats food pills for lunch" future visions... we'll see. It's just that there are actual major problems with current agricultural state of the sart; and I am not thinking of left-wings "agri-biz is bad" angle, but rather more concrete problems of us running out of phospates for fertilization (it is a severely limited natural resource, turns out); loss/compaction of top soil (may need no/low-tilling techniques; but current food crops are not optimized for those); and the perennial problem of over-population and continued world-wide population growth. Oh, and also rabid opposition by well-meaning environmentalists against useful gene-manipulation and breeding techniques.

So, we shall see. We live in very interesting times. As always.

Upgrading from Woodstox 3.x to 4.0

Thu, 31 Dec 2009 22:33:18 -0800

It has now been almost one year since Woodstox 4.0 was released.
Given this, it would be interesting to know how many Woodstox users continue using older versions, and how many have upgraded.

My guess (somewhat educated, too, based on bug reports and some statistcs on Maven dependencies) is that adoption has been quite slow. I think this is primarily due to 3 things:

Older versions work well, and fulfill all current needs of the user
New functionality that 4.0 offers is not widely known, and/or is not (currently!) needed
There are concerns that because this is a major version upgrade, upgrade might not go smoothly.

I can not argue against (1): Woodstox has been a rather solid product since first official releases; and 3.2 in particular is a well-rounded rock solid XML processor (if you are using an earlier version, however, at least upgrade to latest 3.2 patch version, 3.2.9!).
And with respect to (2), I have covered most important pieces of new functionality, Typed Access API and Schema Validation.

But so far I have not written anything about incompatible changes between 3.2 and 4.0 versions. So let's rectify that omission.

1. Why Upgrade?

But first: maybe it is worth iterating couple of reasons why you might want to upgrade at all:

You might want to validate XML documents you read or write against W3C Schema (aka XML Schema). Earlier versions only allowed validating against DTDs and Relax NG schemas
If you want to access typed content -- that is, numbers, XML qualified names, even binary content, contained as XML text -- new Typed Access API simplifies code a lot, and also makes it more efficient.
Latest versions of useful helper libraries like StaxMate require Woodstox 4.0 (StaxMate 2.0 needs 4.x, for example)
No new development will be done for 3.2 branch; and eventually not even bug fixes.

Assuming you might want to upgrade, what possible issues could you face?

2. Backwards incompatible changes since 3.2

Based on my own experiences, there are few issues with upgrade. Although the official list of incompatibilities has a few entries, I have only really noticed one class of things that tend to fail: Unit tests!

Sounds bad? Actually, yes and no: no, because these are not real failures (ones I have seen). And yes, since it means that you end up fixing broken test code (extra overhead without tangible benefits). But this is one of challenges with unit tests: fragility is often desireable, but not always so.

Specific problem that I have seen multiple times is related to one cosmetic aspect of XML: inclusion of white space with elements.

Woodstox 3.2 used to output empty elements with "extra" white space, like so:

but 4.0 will not add this white space:

(this is a new feature as per WSTX-125 Jira entry)

and so some existing unit tests for systems I have worked on compare literal XML for output tests. This is not optimal, but it is bit less work than writing tests in more robust way, to check for logical (not physical) equality. So whereas they formerly assume existence of such white space, tests need to be modified not to expect it (or allow either way).

3. Other challenges?

Actually, I have not seen any actual problems, or other cosmetic problems. But here are other changes that are most likely to cause compatibility problems (refer to the full list mentioned earlier for couple of changes that are much less likely to do so):

"Default namespace" and "no prefix" are now consistently reported as empty Strings, not nulls (unless explicitly specified otherwise in relevant Stax/Stax2 Javadocs). Usually this does not cause problems, because Stax-dependant code has had to deal with inconsistencies with other Stax implementations; but could cause problems if code is expecting null.
"IS_COALESCING" was (accidentally) enabled for Woodstox versions prior to 4.0. This was fixed for 4.0 (as per Stax specification), but it is possible that some code was assuming on never getting partial text segments (if developer was not aware of Stax allowing such splitting of segment, similar to how SAX API does it.

4. Upgrade or not?

I would recommend investigating upgrade; if for nothing else, because of maintenance aspect. Pre-4.0 versions will not be actively maintained in future. But it is good to be aware of what has changed, and of course having good set of unit tests should guard against unexpected problems.

And hey, it's soon 2010 -- Woodstox 3.2 is soooo 2008. :-)

How much does Free cost again?

Tue, 22 Dec 2009 21:53:38 -0800

One of supposedly hard question is what is the actual cost of freedom of speech. It can be debated endlessly, and is often considered priceless (Mastercard trademarks not-with-standing). Hard question, sure, and something I generally don't spend much time thinking about.

And then again, sometimes it is rather easy to know the exact cost. Like in case of my little blog: it is US$ 9.95.

That ultra fancy commenting interface that you see next to entries (and elaborate insanely complicated machinery that powers in in the backend), provided by kind folks of Haloscan, has been free (of cost) so far. But it was very recently brought to my attention that there is now a simple choice for me, a happy user of said system, to make: I am free to either (a) pay up (annual fee), or (b) "vacate the premises" (but with graceful allowance of letting me keep stuff I brought along, i.e. export comments!). What's not to like? What a delightful jule-time offer!

Choice is bit tricky: on one hand, philosophically I do not object to paying for services I use (or offer for others to use). But on the other hand, I really REALLY dislike impression of a bait-and-switch scheme in operation. Worse, given the short transition period -- unless I totally missed earlier notices, just a couple of weeks -- makes this feel rather rushed, "an offer you can't refuse". So in many ways I do feel like just taking my crap (no offense to respected authors of high-quality commentary here) to someplace else. Perhaps even spending lotsa time (which is money) to build a simple replacement for the system. Heck, I have been paid to write such systems. Might as well do some pro bono work for my own blog. But then again... I am lazy and time-constrained.

So: question is; is this dinky little commentary system worth 10 bucks a year? Especially when advertising income from all the sidebar ads is within that same order of magnitude, possibly less.

ps. Anyone know an actual free plug-in comment interface? Send me a note if you do! (or if you must... leave a comment... but if so, before Jan 02, 2010 :-) )

Another Jackson adopter: Spring-json

Tue, 22 Dec 2009 19:58:37 -0800

As has been reported earlier, Spring 3.0 (and Spring-json module, specifically) now has a Jackson-based JSON view variant: see MappingJacksonJsonView and spring-json view comparison. Comparison seems reasonable, mentioning annotation-based configurability and performance as strong points of jackson-based view.

UPDATE, 28-Dec-2009: One more for the road: Restlet is also including Jackson-based extension.

Jackson 1.4: more control over writing JSON, improved interoperability

Sun, 20 Dec 2009 17:33:48 -0800

First things first: if you haven't noticed yet, Jackson 1.4.0 was just released.

This release focuses mainly on writer (serialization) side, but there are also continuing improvements to interoperability. I will review main improvements below.

1. JSON generation improvements

1.1 Ignoring properties

A new annotation, @JsonIgnoreProperties, allows:

omitting serialization of listed properties: @JsonIgnoreProperties({ "secretField", "internalProperty" }); listed properties will not be included in JSON output
omitting listed properties from being deserialized; if encountered they are just ignored even if there is a setter for them (regardless of whether setter is marked to be ignored or not)
ignoring all unknown properties for annotated class during deserialization (similar to disabling DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES), but only affects instances of annotated class. This is done with property "ignoreUnknown": @JsonIgnoreProperties(ignoreUnknown=true) (note: has no effect on serialization)

1.2 JsonView

New @JsonView annotation allows defining logical views for serialization: sets of properties to be written out for given view.

Let's consider a simple example, where we want to control amount of information written out, based on, say, user's credentials. To define 3 classes of properties, we can define views (more about identification below):

  class Views { // container for View classes
static class Public { }
static class ExtendedPublic extends PublicView { }
static class Internal extends ExtendedPublicView { }
  }
  
And to define access levels for our info class, we would do

  public class PersonalInformation { // Bean that uses Views to define subsets of properties to include
// Name is public
@JsonView(Views.Public.class) String name;
// Address semi-public
@JsonView(Views.ExtendPublic.class) Address address;
// SSN only for internal usage
@JsonView(Views.Internal.class) SocialSecNumber ssn;
  }

Given this set up, we would define View to use for serialization by:

  objectMapper.writeValueUsingView(out, infoInstance, Views.Public.class); // short-cut
  // or full version:
  objectMapper.getSerializationConfig().setSerializationView(Views.Public.class);
  objectMapper.writeValue(out, beanInstance); // will use active view set via Config
  // (note: can also pre-construct config object with 'mapper.copySerializationConfig'; reuse configuration)

Which in this particular case would only contain "name" property. If we had used view Views.ExtendedPublic.class, we would have gotten 2 fields; and with Views.Internal.class, all 3.

Views are identified by classes: you can either create specific marker classes, or use existing classes. Views use inheritance indicated by class structure: such that a view is considered a sub-view of another view if it extends that view. Child views include properties that parent views include.

For more description see JsonView wiki page.

1.3 Ordering properties output

Another new annotation, @JsonPropertyOrder, allows defining complete and partial field orderings:

You can define explicit ordering by listing properties as annotation value: @JsonPropertyOrder({ "id", "name" }) would ensure that "id" and "name" are output before any other properties during serialization
You can specify that anything not explicitly ordered will be output in alphabetic order: @JsonPropertyOrder(alphabetic=true)
Without these settings order is undefined because JVM does not expose order of underlying fields or methods (but see note below)

Beyond these definitions, 1.4 also guarantees that properties used with @JsonCreator annotations (constructors, factories) are serialized before other properties, unless there are explicitly ordered properties (which will have priority). This change was to optimize @JsonCreator property usage: ideally these properties should be readable before other properties -- although JSON logical model does not provide for such guarantee, Jackson will try to do its best to make ordering optimal.

2. Interoperability improvements

Goal of interoperability improvement is to make Jackson a "universal" JSON data binding tool on JVM. That is: we hope to make Jackson usable from other JVM languages, not just Java -- already one can use it from quite a few (reported to work from Groovy, Clojure), and hopefully supporting others like Scala in near future (Scala lists are not well handled, yet) -- as well as interoperate nicely with most common data libraries.

To this end, there is now default support (i.e. no need for custom converters or mix-in annotations) for following data types:

DOM (xml) trees: properties declared as DOM Document, Element and Node will now be properly serialized to, and deserialized from JSON Strings. Useful if you want to embed XML as JSON properties (sometimes good for interoperability)
Joda DateTime type, and mechanism to easily add more types as needed (file a Jira request if you need more!)
Handling of javax.xml types that some platforms lack (Android and GAE have had some issues) much improved so that they are dynamically and reliably added, if underlying types are present.

Last point should also make it yet easier to make Jackson run on new "subset platforms"; containers that support subset of JDK 1.5 (or have issues with some parts).

3. Plans for 1.5

So what next? 1.4 release can be thought of a "minor" minor release, similar to 1.3; compared to fundamentally new functionality of 1.2 (mix-in annotations, JsonCreators), 1.3 and 1.4 have consisted of smaller (but more numerous) evolutionary incremental improvements. In many ways, Jackson releases have mirror JDK releases, come to think of that.

Anyway: the next Big Thing will be "Polymorphic Deserialization", which is by far the most requested feature. That is, ability to deserialize instances of correct types, even in absence of static type information (declared type of, say, List<Object> could still be deserialized to contain whatever actual type of serialized instance was). Getting this done is important in itself, but the most important aspect in my mind is to Do It Right. This should not be a stop-gap solution, or something to rewrite in near future. It should be comprehensive, flexible and robust solution to the non-trivial problem. And plan is to do just that; now that other queued blocking issues (like that of finall getting much-requested JsonView done) have been dealt with.