CowTalk

(warning: this is another rant. Sorry!)

Here's another improvement in my daily life: after more than a year of space-shuttle-lift-off noise, short-but-brutish uptimes, and countless curses, family's lean old Dell XP "work"station is out for good. Its only agreeable attribute was its slim neat looks (and sort of neat mechanism used for case, allowing its easy opening -- too bad there's not much to do even if you can open it easily). Good riddance, music to my ears.

The replacement, HP Pavilion Slimline, actually looks every bit as good as its predecessor. But otherwise the two are polar opposites: new box is quiet, as reliable as expected (i.e., "just works"), and its only design flaw is that it comes with an OS written by a company based in Redmond. But that I can live with, since it's not my work machine. :-)
And even Windows seems to have improved a bit between versions (new one has Windows 7, previous one whatever preceded Vista).

Anyway: I just thought I'll share my distaste with Dell products (maybe I should actually include "not-so-good" lists on my semi-professional home page?) now that I am getting rid of them.

It all started couple of years ago, when I decided to stop wasting my time on building my own PCs from components (which made sense after college, could save some money). I figured that with time I spent building PCs, and then troubleshooting problems with components, it just didn't make a whole lot of sense. And so I thought I'd go with something that other customers in general had found usable: back then Dell had highest customer ratings of all PC companies; and save for one friend of mine (who had already fought with Dell's phone "support" people, due to problems with memory chips that were failing; and that no amount of rebooting would ever fix), I wasn't aware of huge problems with the company or its products.

What I found out by experience makes me suspect that the company that had gotten good reviews had been abducted by aliens, and replaced by an ersatz replica or something. Correlation between happy customers and company that produced crap I bought just is not there. I am not talking about customer support (no point calling them wrt. badly designed piece of hardware, IMO, it is not not something a script-reading underpaid remote helper can help a lot with), but rather about quality of hardware. My experience beyond PC fiasco was that their products are competitively priced, but have low quality. For example, laser printer that I bought to replace trusty old Apple writer (which, after having bought second hand, served us for 8 years; for total lifetime of probably 15 years; and would have worked well but I couldn't find new toner cartridges for reasonable prices any more!) was inexpensive, and worked fine for a while. Like, maybe a year. And then broke down. The only thing left are LCD monitors, which I have to admit were reasonably priced, and still work. In fact both are still in active use. So I guess they do produce something other than lemons.

Thinking about that last sentence: I guess I could put my feelings into fitting slogan: Dell -- General Motors of Computers.
Feel free to quote.

ps. I am happy to admit that after kicking that incompetent CEO of theirs out, HP seems to have done nice comeback. Good for them, and us.

Posted by Tatu Saloranta at Friday, March 26, 2010 10:37 PM
Categories: Philosophic, Rant
| Permalink |Comments | links to this post

(note: semi-professional refers to contents and NOT to construction of page itself -- page can be considered totally amateurish in its design and implementation -- but things contained are related to my Professional development, including its career-development aspects)

Ok here is something old recycled as something new: behold, Tatu's Semi-Professional home page at FasterXML wiki.

The idea is to combine information on state of public projects I work on and public material I would like to share with others, in a somewhat compact and less transient form. Blogs are good for publishing some information; but over time things tend to bury under other things. Plus, form of articles, as well as regular Wiki pages, are more for slightly deeper content. I needed a simple link collection, like book index. So that's what this aims to be, personal-but-public index. Like what home pages used to be, sort of... and hey, you can probably spot the Vintage 90s sign of "Work in Progress" attitude to writing and updating the thing. Who'd have thank that 90s became retro so fast.

Anyway: although reading through the page should be challenge to anyone who can read my blog entries, here's yet more condensed take on contents; index of the index (IoI?). What we got is:

• Summary of (open source) projects I am closely involved with; along with near-term plan
• List of lists, "Best of"; things I think others might be interested in

And that's it for now. Will refactor as needed (I'm sure I'll have to reinvent multi-level indexing structure soon); add things that seem worth adding.

Posted by Tatu Saloranta at Tuesday, March 23, 2010 7:53 PM
Categories:
| Permalink |Comments | links to this post

Lately I have gotten to use Git as it is the production SCM of choice for my new employer (having recently taken over from Subversion). This is generally a good thing: while I had learnt to like p4 after using it for a while (and especially after reading "Practical Perforce"), and appreciate svn's finally replacing cvs as the "default open-source SCM, there is something to be said about efficient distributed SCM system. And there is lots of excitement behind svn: consensus is building that Git will become the de facto SCM for open-source development.

For the most part Git has proven to be worth the hype it has generated and working with it has been easy. But the first real speed bump for me was trying to get branches to work. Most documentation focuses on working with local branches (which are usually "personal", single user branchs), which have their uses for development by large teams, or heavily shared codebases. Nothing wrong with that.

But for my own uses day-to-day uses with open source projects, it is so-called remote branches (ones that live on shared "master" repository) that matter more. Specifically, branches used for "legacy" versions, needed for adding patches for versions other than the most recently released stable: having "1.3" and "1.4" branches while trunk/mainline/master being used for upcoming "1.5" version (for example).

And this is where things get bit unintuive and complicated. Fortunately, after googling for an hour, I found this article, which gave me specific recipe I needed:

# 1. Create the remote branch
git push origin origin:refs/heads/new_feature_name

# 2. Make sure everything is up-to-date
git fetch origin

# 3. Then you can see that the branch is created.
# (this should show ‘origin/new_feature_name’)
git branch -r

# 4. Start tracking the new branch
git checkout --track -b new_feature_name origin/new_feature_name

# 5. Make sure everything is up-to-date
git pull

plus absolute necessary follow-up for "Cleaning up Mistakes":

# If you make a mistake you can always delete the remote branch
git push origin :heads/new_feature_name

and that's it. I hope. If not, I'm sure to gripe about that in future. :-)

Posted by Tatu Saloranta at Monday, March 22, 2010 10:54 AM
Categories: General
| Permalink |Comments | links to this post

I don't know how I have managed to miss the next "Simple Big Thing", but I noticed that something called WebSockets has been around for months. And yet I only now bumped into (via announcement for Jetty 8, which boasts support for Servlet 3 API as well as WebSockets). Better late than never I guess.

The idea with WS is very simple, and even somewhat elegant: start up with a regular HTTP connect with hand shake, and then convert it to what amounts to a socket. And use simple properties of UTF-8 encoding to provide simple and inexpensive framing mechanism. Start and end markers used are bytes 0x00 and 0xFF which can not be included in UTF-8 encoded textual data (along with couple of other bytes); and this mostly guarantees that you can nicely included structured textual data using XML or JSON. In fact, elegance of the solution reminds me of UTF-8 encoding which also is surprisingly well designed and simple. In fact, ability of WebSockets to do what it is doing is yet another testament for pure goodness of UTF-8 encoding.

Anyway, this is just a heads up, just in case some of you may have missed this important development. Believe me, this is going to be a big thing very soon. Comet begone; WebSockets is the way forward for efficient low-latency stateful interaction!

Posted by Tatu Saloranta at Sunday, March 21, 2010 12:23 PM
Categories: Performance
| Permalink |Comments | links to this post

Lately (as in this week) I have had to do some work with relational databases (argh!) -- nothing too bad, pretty basic stuff, something where BDB would work mostly fine, but where accessibility dictates more shareable alternative.
But for testing purposes, I much rather have something fully controllable from within JVM instead of having to deploy a repliace of DB that will be used for production.

Fortunately there are multiple viable choices; and just based on personal preferences I decided to try out H2.

1. Detour: why H2?

Part of my reasoning is due to seemingly healthy activity on mailing lists; and another part is the feature set (which is extensive). But ultimately it mostly comes down to the fact that project and its author seem very much like archetypical examples of Good Open Source Project (and its leader); specifically project that has a single author that

• has strong vision regarding what the product should be about, and
• has actually "done it before"; either implemented a similar product before (in this case, hypersonicSQL, and even more) or worked in domain, and
• is a competent developer all-around, outside his/her specific domain

But I digress. I just thought I'll mention how much good leadership means for Open Source projects, and why it is perhaps the best indicator for quality of project output (this is why Saxon is so good; why Jersey kicks ass; and so forth); much more so than branding (yes, being a "Google project" gives you plenty of insta-karma, but means surprisingly little with regards to quality).

2. Ok, so How Do I Start the Thing?

Anyway: back to what I was doing... a t first I spend quite a bit of time trying to read through documentation (which does exist; not in abundance, but in large enough quantities) trying to figure out how to start up database in embedded, in-memory mode. From feature set, it is clear that one can run H2 in embedded mode; and further, fully in-memory (as an alternative to disk-based persistence). But somehow I failed to initially realize how simple this is, and tried to figure out with Server/Service (TCP, HTTP or PG == Postgres) is needed.

Turns out the right answer is "none of above". In fact, you don't need to start anything. Rather: "if you call it, it will be there".
Literally: database will spring to existence if you try connecting to it via JDBC, using special URL.

... d'oh! But of course. :-)

3. Really? Are you kidding me?

Nope.

Once you wrap your brains around this Zen-y idea, it starts to make more sense (and also suggest at one issue to overcome -- database will also, by default, be gone when the last connection closes -- this can be easily resolved, more on this later)

So, just to help anyone else who might just be googling for the answer; it is JDBC connection String that matters. I use something like:

  jdbc:h2:mem:test;DB_CLOSE_DELAY=-1

(where 'test' is logical name of the database, needed to allow multiple connections)

so, for example:

---

 DataSource ds = JdbcConnectionPool.create("jdbc:h2:mem:test;DB_CLOSE_DELAY=-1", "user", "password");
 Connection conn = ds.getConnection();
 conn.createStatement().executeUpdate("CREATE TABLE data ("
   +" key VARCHAR(255) PRIMARY KEY,
   +" value VARCHAR(1023) )");
 // ... populate with data, test etc
 conn.close();

---

and that's all. No need to start anything else up unless you actually want DB to serve external connections. Oh, and if you do: yes, you can actually expose embedded and/or in-memory databases (these are orthogonal choices!) to external clients too.
Connecting is done either using straight JDBC, or using simple connection pool H2 comes equipped with, which actually seems functional enough for most needs. I think I like this little (?) database already; it makes writing unit tests a breeze, obviously. But I suspect it could also act as somewhat production ready light-weight - yet - non-toy SQL database. We shall see.

Oh, and for more quick-n-dirty tips, just check out H2 Cheat Sheet. It's some condensed good stuff.

Posted by Tatu Saloranta at Saturday, March 20, 2010 6:43 PM
Categories:
| Permalink |Comments | links to this post

Here's another detour from the usual technical content. Just for fun, let's talk about some lighter subject: subjective feeling of rewards, in context of some bigger thing like, say, US health-care reform.

As background, I admit that while I have significant interest in on-going process of getting universal health-care available here in the United States, I don't have much to say about goal itself or road there. Much has been said; most of essential aspects, and ton of crap beyond that. So it is enough to state that my father is an M.D (outside of US); that I have been familiar with amounts US overpays for its health-care (100-150% higher per capita than any other country -- this has been widely known since 90s); and that I am aware of how much good that spending has done to national health, statistically (wrt. infant mortality, life expectancy -- very little). And from this you can deduce what I think is the right way forward.

But regardless of my personal opinions, there are still things I find interesting about the meta-process. Here are some easy predictions to make, assuming reform bill passes (which seems quite probably at this point):

• Sky is not going to fall: it is enough to look back in history for all significant changes, from end of Prohibition, to US social security system bootstrapping to civil liberties, to welfare system reform. Things are not going to be altogether different, even if things got royally clustered on short term (wrt. systemic changes)
• Rate of improvement is going to be slow: system doesn't even kick in for couple of years; but beyond that, well, there is always huge inertia with bigger changes

These are trivially obvious things. But there is one more easy prediction to make, something based on basic behavioral psychology, that I find more interesting:

• People who are rooting for the reform will find passage of the Health-care bill VERY rewarding, especially so on short term.

Why so? Isn't it "known truth" that this bruising long process takes joy out of victory? That there's long process of healing yadda yadda?

No: absolutely on contrary! It has been proven time and again that the reward one gets from achievements is proportional to effort that was needed to get it done (in case of outside observers, effort may well just consist of following up on events and of emotional attachment, worry and so on -- nonetheless more involvement than with most political causes)

So if the reform had passed easily, it would have gotten at most lukewarm feeling of accomplishment, soon to be diluted by other matters. Bigger the fight and fiercer the resistance, the sweeter is overcoming of these obstacles.

And guess what? Both sides know this, at some instinctive level. Why they don't seem to know it at conscious level I don't know. Or perhaps it just seems better not to spell it out. But to figure this out, you just have to read, say, "Predicably Irrational" (chapter about trying to buy season tickets from no-lifers who have spent almost a year doing stupid stunts, to get the tickets; and value them almost an order of magnitude higher than ones who did NOT get the tickets, after similar amount of effort), or about any other recent pop behavioral science book.

Actually, given this, I'm pretty sure Obama team knows this; and possibly made good use of it during campaigning (they won it, and supporters got nice kick out of prolonged battle -- this, too, was researched [I saw a reference from SciAm, forget where the study was published]). They seem well-versed in theory (and perhaps practical art) of basic behavioral patterns.

Maybe it wouldn't be a bad idea to host parties like ones they have after one's national team has won soccer world cup or something. :-)

ps. Of course, if the effort was to fail, its opponents would get similar highs, enjoy the rewards. Hitting the brick with your fist only hurts if the dang thing does NOT break.

Posted by Tatu Saloranta at Friday, March 19, 2010 10:22 PM
Categories: Philosophic
| Permalink |Comments | links to this post

Annotations are a great feature that JDK 1.5 brought along. Along with enumerations and generics it vastly improved Java usability. And compared to generics, annotations are rather easy to understand and use.

But considering how good annotations are in general, there are two irritating aspects -- like two sore thumbs -- that sort of ruin perfect harmony between my Java-dev happiness, and Java annotations:

• Inability to use plain old null as annotation value: no, you absolutely can not define null as annotation property value; neither explicitly, nor implicitly (can only omit annotation properties with default value, and default value can not be null)
• Inability to use "any enum value" as value (possibly with generic annotations

Of these, latter is annoying, and prevents some use cases (say, ability to define an enum and use it as ordered constants for versioning via annotations), but I can see why there may be technical limitations. Enumeration implementation is bit convoluted when you get down to low-level details (just for fun, try to figure out what is the REAL type of enumeration values, via Introspection... think they are fields? think again!)
But former... whoever thought that null would be useless as annotation property value made a huge disservice to Java developers. Null is absolutely crucial value to have. It's like having numeric system without value zero.

So why does this matter? Basically, not having null as an option means that one has to create null surrogates: values that imitate nulls. For example, assume that you want to allow adding custom handlers via annotations (for, say, JSON serializer to use for a type): you can define an annotation property with value of Class<IntendedType> (yes, generics work just fine here). But you can not easily make its use optional: so, if you do not want to define such handler (common case), there is no easy way to do it.

The usual work-around, then, is to do two things:

1. Create a dummy class (abstract class MyAnnotation.NoHandler extends Handler { } )
2. Assign this dummy class as the default value for said property

This works, but is ugly. And gets even uglier when annotation property has String type -- that's where those funky "##default-name##" constants for JAXB come from.

And all this instead of the obvious Right Solution: allowing annotation properties to have value of null. It would be interesting hear plausible explanations for this omission.

Posted by Tatu Saloranta at Thursday, March 18, 2010 10:03 PM
Categories: Java
| Permalink |Comments | links to this post

Now that Jackson 1.5 is out (ready for download?), it's good to start reviewing some candies from the goody bag. While 1.5 is, technically speaking, a "minor" release it is nothing but grand judging by number of added features .
In a way Jackson mirrors development of JDK: Jackson version 1.2 and 1.5 have contained tons of new stuff, whereas 1.3 and 1.4 have been more about incremental steady improvements and bug fixes.

Let's start with the top requested new feature: Polymorphic Type Handling.

1. Polymorphic Types: not just for serialization any more!

First things first: polymorphic types just refer to cases where there is a single declared property type (say, Animal), but multiple concrete subtypes (like Dog and Cat). Basic OO stuff, implemented in Java by sub-classes and interface implementation. Nothing unusual there.
Challenge is actually that of mapping this cleanly to data formats: as with relational model (SQL) and hierarchic model (XML), JSON has no native concept of inheritance (XML has neither, in itself; XML Schema does add support however).

Jackson has actually supported serialization of all types, including polymorphic subtypes, since first data binding enabled versions (0.9.5). This because Jackson by default uses actual runtime type when determining serializer to use (although it can be configured to only static, declared type, too) -- which is different from what JAXB does -- and thus has no trouble writing out all properties sub-type has.

But the problem is, without additional type information, it is not possible to determine intended subtype when deserializing, when there are multiple subtypes to choose from: when property has declared type of 'Animal', there is no way to know whether it should be reconstructed as "Dog" or "Cat". This is where PTH implementation comes in.

Obvious answer is that additional type information must be added in serialized JSON data to allow this later reconstruction. But there are multiple considerations as to how this should be done, including:

• What kind of type information should be included (Java class name? Logical type name? something else?)
• How (where) should this type information be included (as a property? using wrapper constructs?)
• Do we always have to include this type information? (doing so adds unnecessary bloat in cases where type information is redundant)

Jackson 1.5 tackles these (and some other minor concerns) in an efficient, configurable, and I hope, elegant way.

2. Enough theory: show me the code!

Ok: chances are that you are not all THAT interested in details of how implementation was tricky, or what philosophical considerations went on in determining what to implement and how. So let's start with some toy code.

Considering case of a polymorphic class; say... Person base class, and Employee and Customer sub-classes (I have written an "Animal example" at FX wiki already, let's get some variety), and using a wrapper class such as:

---

public class Contact {
 public Person initiator; // who initiated contact
 public Person receiver; // with whome?
}

---

and Person (etc) classes as follows

---

public abstract class Person {
 String name;
}

public class Employee extends Person {
 long employeeId;
}

public class Customer extends Person {
 String ssn;
 ServiceLevel serviceLevel;
}

---

What do we need to do to make Contact properly deserializable? By default, no additional type information would be added, and deserialization would fail

There are two main ways to do this, covering slightly different use cases, and offering different levels of configurability.

3. Type Info, method #1: explicit annotations

First method is to add explicit @JsonTypeInfo annotation in the declared type (or its supertypes):

@JsonTypeInfo(use=JsonTypeInfo.Id.CLASS, include=JsonTypeInfo.As.PROPERTY, property="@class")
public abstract class Person {
  //...
}  

which would basically enable addition of type information for any instances of Person, as well as its subtypes. With above configuration -- include fully-qualified Java class name as type identifier; include it as main-level additional (meta-)property using property name "@class" -- output might look something like:

{
  "@class" : "com.fasterxml.sample.Employee",
  "name" : "Billy Bacon",
  "employeeId" : 126509
}

(with additional type information bolded)

So far so good: it is bit extra work to sprinkle annotations (which can be added via mix-ins, of course), but nothing too bad. As long as you know types in advance. And you can configure handling pretty well (check out Javadocs for more details for @JsonTypeInfo, related; or if I get that far, later entries on advanced topics)

4. Type info, method #2: default typing

But there is another way, which requires much less code, although offering bit less configurability. Specifically, you can enable so-called "default typing" for ObjectMapper by:

  mapper.enableDefaultTyping(); // defaults for defaults (see below); include as wrapper-array, non-concrete types
  mapper.enableDefaultTyping(ObjectMapper.DefaultTyping.NON_FINAL, JsonTypeInfo.As.WRAPPER_OBJECT); // all non-final types

which would then apply default type information (type id is always fully-qualified Java class; inclusion method is customizable but defaults to "wrapper array") to all types (classes) that fulfill applicability criteria: by default it would be Object.class and non-concrete types (abstract classes, interfaces). So in our case this would work, given that Person is an abstract class: and if it wasn't, we could use "ObjectMapper.DefaultTyping.NON_FINAL" to include type information on all non-final classes.

5. Is that All?

No, not at all. This is just the simplest way to start safely handling polymorphic types like, say, List<Object>.

What else there is to find is good subject for follow-up entries, but here are some teasers:

• Java class name is not a good choice when you have non-Java clients or services to talk to -- if so, you can use "logical type name" (see @JsonTypeName and @JsonSubTypes for more info); much like what JAXB does. Or, even further, you can have fully customized type id converters! (as long as they can convert to/from JSON Strings)
• If you don't like using main-level property names (maybe there's a conflict), you can use other styles; "wrapper Object", "wrapper Array"
• If you like default typing, but don't like its selection criteria, you can create your own: just sub-class default TypeResolverBuilder, assign it to your ObjectMapper.
• Similarly, you can define fully custom type information handler, using @JsonTypeResolver annotation -- this will be nicely integrated with the whole system, you only have to provide actual inclusion, but it will be called for any existing types.

But enough for now... more on 1.5 later.

Posted by Tatu Saloranta at Wednesday, March 17, 2010 8:07 PM
Categories: Java, JSON
| Permalink |Comments | links to this post

Somehow I have managed to miss obvious competition that is creeping up for crown of best simple non-distributed persistent (disk-based) key-value store. So far choice has been (or so I thought) mostly between BDB flavours: BDB JE (Java Edition) and its slightly older big brother, BDB-C ("native"). And about only other choice would have been a Java clone, JDBM (which regrettably hasn't really woken from its hibernating state, despite interest).

But it turns out that there are many more choices out there, solid, mature, high-performing choices. Specifically:

• Tokyo Cabinet (TC)
• CDB (Constant DB -- and probably a play on fact that 'C' follows 'B' in alphabet), and especially (?) its Java version, sg-cdb
• Redis

The three are not exactly equivalent, actually: only Tokyo Cabinet can be seen as exact replacement of BDBs. CDB, for example, is a read-only store. And Redis is an in-memory store, but with memory backup (snapshot, or journaling), and slightly expanded value handling semantics (with some set and list value primitives, for aggregate operations).

This diversity is actually a good thing, however; as per "right tool for the job" thinking, they all should have specific sweet spots regarding usage. In fact, I could see each of them excel in slightly different tasks. And BDB, too, is probably still the best choice for some use cases; specifically BDB-JE that has pretty good behavior under heavy concurrency scenarios, thanks to its design.

But what would those optimal tasks be? That is something that would require actually using them, or at least playing with them, perhaps building simple benchmarks. That sounds like a good idea for some prototyping, for near-term future. :-)

UPDATE (16-Mar-2010): Looks like Redis is getting some more tailwind, as perthis announcement (Redis author joins VMWare, will work on Redis). Very cool!

Posted by Tatu Saloranta at Thursday, March 11, 2010 10:26 PM
Categories:
| Permalink |Comments | links to this post

I know this is something that users of Jackson JSON library must have been craving for longest time: their Very Own Social Network (no? you'd rather have handling of cyclic types? d'oh!)
Wait no longer: enter jackson-users.ning.com. Place for developers who know the Right Tool for slicing and dicing JSON on Java platform (or possible, any platform).

So what's the point? Besides my own curiosity (and, shall we say, desire to "eat your own dog food"; check out author's profile for more info) -- which was a big factor -- I thought it might be one more way to get more of community feel. Mailing lists are good, blogs and wikis too, but a gathering place with bit of all of them might be even better.
We shall see how it all works out -- and thanks to lower threshold for participating (since it's easier to give access, let members edit and add things), you can be important part of making it Work Right.

See you there: I'll try sending a nice Virtual Gift for member number 100... :-)

ps. What about blogs over there, and here? My current thinking is that "Jackson Users blog" will focus strongly on Jackson (JSON, data binding, frameworks that use it) issues; whereas CowTown blog will have wider coverage. So it should be more of specialization versus general interest.

Posted by Tatu Saloranta at Wednesday, March 10, 2010 5:39 PM
Categories: Java, JSON
| Permalink |Comments | links to this post

Here's another interesting article: "RESTing with JSON in Gaelyk". Looks like Jackson usage on "Google platforms" (Android, GAE) is becoming more of a maintsream thing. This is great for multiple reasons, and multi-platform support is one of (many) goals of Jackson.
But just having goals matters little if they are not met. Fortunately, thanks to active user/adopter base, goals are being verified over time; and where there are problems, they usually get resolved in timely matter.

Posted by Tatu Saloranta at Wednesday, March 10, 2010 5:34 PM
Categories: Java, JSON
| Permalink |Comments | links to this post