CowTalk

1. Generics in Java: love and hate

Ever since Java 1.5 introduced generic types, Java developers have had strained relationship with them. On one hand, they are clearly a nice addition for static type safety of collection types; as well as make generic dispatching patterns (and fluent-style construction-by-copying-methods) possible. But on the other hand there are tricky issues introduced; mostly stemming from the infamous Type Erasure (see Java Generics FAQ if you are not familiar with it).

Generic types are especially problematic for framework and library developers. This is because although type erasure is not total -- Fields, Methods and Super-types have generic type information available from within class definitions (see "Super Type Tokens to Rescue!" for an explanation) -- available non-erased type information is offered in a nearly inedible form, as instances of "java.lang.reflect.Type" (which is implemented by Class.class, amongs other types).

2. Superficial issue: bad object hierarchy, modelling

The first obvious issue with Type is that it is not much more than a marker type, and exposes little in way of common functionality between implementations. So the very first thing one has to do is to upcast it to one of subtypes; and this suggests (rightly so) that object model is not very good. The reason for such awkward type hierarchy is probaby backwards-compatibility: as Java 1.5 had to bolt-in Type to be a supertype of Class, and Class had been extensively used by JDK, it may have been difficult to create any meaningful interface type to use.

But as awkward as it is to do instanceof's and upcasting, this is not the real big problem. There are some frameworks that try untangling traversal of this ugliness (like Kohsuke's Tiger Types); and coming up with a better type hierarchy is not particularly difficult.

3. The REAL problem: 'Type' only contains partial type definition

To illustrate the actual problem, let us consider following types:

  public abstract class Wrapper<T>
  {
    public T value;
  }
  public abstract class ListWrapper<E> extends Wrapper<List<T>> { }

  public class MyStringListWrapper extends ListWrapper<String> { }

Quick: what is type of field "value" of type MyStringListWrapper?

For seasoned Java veterans answer should come easy: it is of type "List<String>". For code that tries to determine type, an obvious procedure would be:

1. Locate java.lang.reflect.Field representing field "value" from Wrapper.class
2. Get its generic type using "field.getGenericType()"

Simple? Not so fast. What gets returned is an instance of Type; and more specifically and instance of java.lang.reflect.TypeVariable.
And what does TypeVariable give us? At most, upper and lower bounds (if we had "T extends B" or "T super S"), and... name. Bummer. Not much to go about at all.

The next obvious idea is to check out who declared the field (field.getDeclaringClass()), and see if we could somehow figure it out.

Turns out we can not: class "Wrapper.class" has no idea -- all it knows is that there is a type parameter T. Worse, while we can figure out super types (someClass.getGenericSuperType()), there isn't way to do the opposite as the class may be extended by multiple subtypes; and because thanks to Type Erasure, there will only ever be just one instance of any given class, no matter how many times it is extended with varying type parameters.

The real problem, then, is that we just do not have enough context to reliably resolve type parameters for given Methods, Fields or Constructors. In this case we would need "MyStringListWrapper.class"; from which point we could (with some work... non-trivial, but doable) unravel actual full type signature.

4. Solution: we need (more) context

From above it should be obvious that it is not enough to just hand a java.lang.reflect.Type value and expect it to tell the whole story. What is needed is context that represents classes, and more importantly, class-supertype relations where remainders of generic type information are hidden. Given this information it is possible -- although not trivially simple -- to reconstruct the full type definition of a member.

5. Detour: why do so many frameworks get this wrong?

Before presenting something better, I want to point out something interesting: most existing frameworks and APIs seem to operate under misunderstanding that it is enough to just pass java.lang.reflect.Type value and be done with it. JAX-RS, for example, is a really nice REST(-like) API (with good free implementations); but it passes serialization/deserialization values types as java.lang.reflect.Types (possibly together with Class that is not context but just type-erased equivalent of value; which does not help a lot with resolution).

I guess the idea may have been that perhaps one should have custom implementations of Type values (which is some work as there are no public default implementations) which can then contain information. This is theoretically possible, but very much impractical -- the gap between Type you get from Method, Field or Constructor is not enough, as you need to traverse type hierarchy; and THEN create custom implementations of GenericType... and then it just _might_ work.

But I digress; let's get back to solving the problem of properly resolving generic type information.

6. A library to handle generic type resolution: Java ClassMate

As part of implementing Jackson JSON processor, I had to solve the problem of resolving generic types of class members. It took until version 1.6 to get all (?) edge cases completely cracked, but at this point I think everything is working correctly, based on understanding the complex rat's nest of Java type information. Given this (and persistent requests from my fellow open source authors to write something like "generic Mr Bean" package), I figured that maybe I could actually write a good library that solves this problem, as well as some additional questions (yes -- there are plenty of additional problems needed when auto-discovering properties of POJOs -- but more on this on follow-up articles).

So: this is where my newest open source library -- Java ClassMate -- will hopefully make the world slightly less brutal place for Java framework developers.

To solve case I presented above, you would need to do 2 things:

1. Resolve POJO type -- in this case, MyStringListWrapper -- to fully resolve type hierarchy (including type parameter bindings)
2. Resolve class members, hierarchically.

Two-step processing was chosen (instead of beginning to end) for efficiency reasons, and because there are use cases where only first part is required (for example, to just find parameterization of generic types -- i.e. "I have this Map type; what is the key type?"

I will not go too deep into full functionality of the package: but here is piece of code needed to handle example case above:

---

  // First: need to resolve actual POJO type
  TypeResolver typeResolver = new TypeResolver(); // TypeResolvers are thread-safe, reusable
  ResolvedType pojoType = typeResolver.resolve(MyStringListWrapper.class);
  // and then resolve members (fields, methods);
  MemberResolver memberResolver = new MemberResolver(typeResolver); // likewise, reusable
  // for now, use default annotation settings (== ignore), overrides (none), filtering (none)
  ResolvedTypeWithMembers bean = memberResolver.resolve(mainType, null, null);
  // and then find field we are interested in
  for (ResolvedField field : bean.getMemberFields()) {
    if ("value".equals(field.getName()) {
       ResolvedType fieldType = field.getType();
    }
  }

---

ResolvedType, in contrast to java.lang.reflect.Type, has fully resolved generic type parameter information; along with some other niceties such as optional aggregation of annotations (for example, methods can "inherit" annotations from overridden version of the method from super-class or interface).

In a way, ClassMate proposes a replacement of existing JDK type hierarchy, with methods that allow constructing property type information from available "raw" information. This includes not only ability to pass raw classes (in which case generic type MUST come from super-type definitions) but also programmatically constructing types (given raw class and generic type parameterization explicitly; or by using "GenericType" which uses "Super Type Token" pattern).

And this will actually be enough to figure out as much generic type information that there is to find, and write libraries that handle these types as expected; even when presented with advanced multi-level type parameterization.

7. Still There'll Be More

(but fear not, I will neither blacken your christmas, nor do anything to your door)

Since ClassMate is still in its pre-1.0 state, there are things left to complete; and maybe API can be simplified. But I would welcome all potential users to check it out at this point, since this would be perfect time to make sure use case you have is supported. I will try to write more about actual usage on my blog; ideas for things to write about and questions on how to handle a related use case would be most welcome.

Posted by Tatu Saloranta at Tuesday, December 21, 2010 8:15 PM
Categories: Java, Open Source
| Permalink |Comments | links to this post