Wednesday, December 22, 2010

Experiments with advertising, Adsense vs Adbrite, experience so far

It has been a while -- almost 6 months, to be precise -- since I decided to see if there is more to on-line advertising than venerable Google AdSense. So it is time to see if I have learnt anything.

1. Summary: gain some, lose some

An overall verdict is pretty much inconclusive: I like some aspects (more control, less fluctuation with revenue); but from strictly monetary view point, change is mixed bag. Fortunately revenue we are talking about is in "trivial" range -- enough so that it does not round down to zero, but not enough to pay for hosting at current rates. So I can freely do whatever I want without risking losing any "real money". I might as well just gain StackOverflow credits.

2. Positive

Overall the main positive aspect for me is the feeling of empowerment: AdBrite gives more control for publisher, from controlling what to display to defining minimum bids, and even allowing fallbacks (typically, to, what else but AdSense!). I like this a lot, and would assume it is a non-trivial competitive advantage as well: whereas for me control is more of a nice-to-have, I know for a fact that bigger "serious" publisher REALLY want to have more control. This is most important for publishers with valuable brand to take care of.

Another smallish positive thing is that since most advertisements are cost-per-display (aka cost-per-thousand == CPM), and NOT cost-per-click (CPC), revenue stream is steadier. With AdSense, your revenue typically fluctuates wildly, unless you get lots of direct placements.

3. Negative

On downside, "guaranteed" revenue from CPM is not particularly high. In fact, little CPM that I have seen from AdSense for others sites (not this blog) is typically in the same range as what I can get from AdBrite for majority of views (AB does have wider range of CPMs, based on viewer profiling); and even if readers are often skimpy clickers, whenever there are clicks it is typically worth more than two or three thousand CPM views. So overall it is possible that AdSense might actually pay more, over time (with caveat from above that either way, very little money will change hands :-) ).

4. Other

Oh. One thing I was hoping to see was wider selection of interesting ads to display; not just same old, same old. This may or may not be true: I think that overall selection may be wider (just from looking at all the ads that get displayed, via publisher management console), but selection for individual profile is still rather limited. So I don't know if it's very different from what Google would give. I guess it makes sense, in a way, that algorithms tend to over-fit ads with (IMO) too little randomity. But I really would like some more variation, personally.

5. Conclusions

It has been merely quite interesting ride. Perhaps I should check out potentially other choices? Too bad most alternatives seem to be just obnoxious irritating block-the-whole-page scams, or things that try to take over links, images; things that I would personally hate to see. I have no plans to introduce anything like that. But as usual, I am open to things that fit in well enough; something similar to AdSense or AdBrite ad systems, I guess.

Tuesday, December 21, 2010

Why 'java.lang.reflect.Type' Just Does Not Cut It as Complete Type Definition

1. Generics in Java: love and hate

Ever since Java 1.5 introduced generic types, Java developers have had strained relationship with them. On one hand, they are clearly a nice addition for static type safety of collection types; as well as make generic dispatching patterns (and fluent-style construction-by-copying-methods) possible. But on the other hand there are tricky issues introduced; mostly stemming from the infamous Type Erasure (see Java Generics FAQ if you are not familiar with it).

Generic types are especially problematic for framework and library developers. This is because although type erasure is not total -- Fields, Methods and Super-types have generic type information available from within class definitions (see "Super Type Tokens to Rescue!" for an explanation) -- available non-erased type information is offered in a nearly inedible form, as instances of "java.lang.reflect.Type" (which is implemented by Class.class, amongs other types).

2. Superficial issue: bad object hierarchy, modelling

The first obvious issue with Type is that it is not much more than a marker type, and exposes little in way of common functionality between implementations. So the very first thing one has to do is to upcast it to one of subtypes; and this suggests (rightly so) that object model is not very good. The reason for such awkward type hierarchy is probaby backwards-compatibility: as Java 1.5 had to bolt-in Type to be a supertype of Class, and Class had been extensively used by JDK, it may have been difficult to create any meaningful interface type to use.

But as awkward as it is to do instanceof's and upcasting, this is not the real big problem. There are some frameworks that try untangling traversal of this ugliness (like Kohsuke's Tiger Types); and coming up with a better type hierarchy is not particularly difficult.

3. The REAL problem: 'Type' only contains partial type definition

To illustrate the actual problem, let us consider following types:

  public abstract class Wrapper<T>
{ public T value;
public abstract class ListWrapper<E> extends Wrapper<List<T>> { } public class MyStringListWrapper extends ListWrapper<String> { }

Quick: what is type of field "value" of type MyStringListWrapper?

For seasoned Java veterans answer should come easy: it is of type "List<String>". For code that tries to determine type, an obvious procedure would be:

  1. Locate java.lang.reflect.Field representing field "value" from Wrapper.class
  2. Get its generic type using "field.getGenericType()"

Simple? Not so fast. What gets returned is an instance of Type; and more specifically and instance of java.lang.reflect.TypeVariable.
And what does TypeVariable give us? At most, upper and lower bounds (if we had "T extends B" or "T super S"), and... name. Bummer. Not much to go about at all.

The next obvious idea is to check out who declared the field (field.getDeclaringClass()), and see if we could somehow figure it out.

Turns out we can not: class "Wrapper.class" has no idea -- all it knows is that there is a type parameter T. Worse, while we can figure out super types (someClass.getGenericSuperType()), there isn't way to do the opposite as the class may be extended by multiple subtypes; and because thanks to Type Erasure, there will only ever be just one instance of any given class, no matter how many times it is extended with varying type parameters.

The real problem, then, is that we just do not have enough context to reliably resolve type parameters for given Methods, Fields or Constructors. In this case we would need "MyStringListWrapper.class"; from which point we could (with some work... non-trivial, but doable) unravel actual full type signature.

4. Solution: we need (more) context

From above it should be obvious that it is not enough to just hand a java.lang.reflect.Type value and expect it to tell the whole story. What is needed is context that represents classes, and more importantly, class-supertype relations where remainders of generic type information are hidden. Given this information it is possible -- although not trivially simple -- to reconstruct the full type definition of a member.

5. Detour: why do so many frameworks get this wrong?

Before presenting something better, I want to point out something interesting: most existing frameworks and APIs seem to operate under misunderstanding that it is enough to just pass java.lang.reflect.Type value and be done with it. JAX-RS, for example, is a really nice REST(-like) API (with good free implementations); but it passes serialization/deserialization values types as java.lang.reflect.Types (possibly together with Class that is not context but just type-erased equivalent of value; which does not help a lot with resolution).

I guess the idea may have been that perhaps one should have custom implementations of Type values (which is some work as there are no public default implementations) which can then contain information. This is theoretically possible, but very much impractical -- the gap between Type you get from Method, Field or Constructor is not enough, as you need to traverse type hierarchy; and THEN create custom implementations of GenericType... and then it just _might_ work.

But I digress; let's get back to solving the problem of properly resolving generic type information.

6. A library to handle generic type resolution: Java ClassMate

As part of implementing Jackson JSON processor, I had to solve the problem of resolving generic types of class members. It took until version 1.6 to get all (?) edge cases completely cracked, but at this point I think everything is working correctly, based on understanding the complex rat's nest of Java type information. Given this (and persistent requests from my fellow open source authors to write something like "generic Mr Bean" package), I figured that maybe I could actually write a good library that solves this problem, as well as some additional questions (yes -- there are plenty of additional problems needed when auto-discovering properties of POJOs -- but more on this on follow-up articles).

So: this is where my newest open source library -- Java ClassMate -- will hopefully make the world slightly less brutal place for Java framework developers.

To solve case I presented above, you would need to do 2 things:

  1. Resolve POJO type -- in this case, MyStringListWrapper -- to fully resolve type hierarchy (including type parameter bindings)
  2. Resolve class members, hierarchically.

Two-step processing was chosen (instead of beginning to end) for efficiency reasons, and because there are use cases where only first part is required (for example, to just find parameterization of generic types -- i.e. "I have this Map type; what is the key type?"

I will not go too deep into full functionality of the package: but here is piece of code needed to handle example case above:

  // First: need to resolve actual POJO type
TypeResolver typeResolver = new TypeResolver(); // TypeResolvers are thread-safe, reusable
ResolvedType pojoType = typeResolver.resolve(MyStringListWrapper.class);
// and then resolve members (fields, methods);
MemberResolver memberResolver = new MemberResolver(typeResolver); // likewise, reusable // for now, use default annotation settings (== ignore), overrides (none), filtering (none) ResolvedTypeWithMembers bean = memberResolver.resolve(mainType, null, null); // and then find field we are interested in
for (ResolvedField field : bean.getMemberFields()) {
if ("value".equals(field.getName()) {
ResolvedType fieldType = field.getType();

ResolvedType, in contrast to java.lang.reflect.Type, has fully resolved generic type parameter information; along with some other niceties such as optional aggregation of annotations (for example, methods can "inherit" annotations from overridden version of the method from super-class or interface).

In a way, ClassMate proposes a replacement of existing JDK type hierarchy, with methods that allow constructing property type information from available "raw" information. This includes not only ability to pass raw classes (in which case generic type MUST come from super-type definitions) but also programmatically constructing types (given raw class and generic type parameterization explicitly; or by using "GenericType" which uses "Super Type Token" pattern).

And this will actually be enough to figure out as much generic type information that there is to find, and write libraries that handle these types as expected; even when presented with advanced multi-level type parameterization.

7. Still There'll Be More

(but fear not, I will neither blacken your christmas, nor do anything to your door)

Since ClassMate is still in its pre-1.0 state, there are things left to complete; and maybe API can be simplified. But I would welcome all potential users to check it out at this point, since this would be perfect time to make sure use case you have is supported. I will try to write more about actual usage on my blog; ideas for things to write about and questions on how to handle a related use case would be most welcome.

Monday, December 13, 2010

Amazon Web Service (AWS), WikiLeaks: series of unfortunate events

As a current Amazon Web Service customer (as well as ex-employee of Amazon) I was sad to see reports of AWS mishandling of its WikiLeaks hosting.
My main objection is not regarding whether AWS should host the content or not, and I understand that due to self-service nature sometimes termination need to occur after customer relationship has been initially established. But the way termination came about was a complete cluster and really makes me wonder if I want to continue using AWS or even recommend it to others.

As far as I understand, basic facts are that:

  1. WikiLeaks started hosting content with AWS
  2. AWS was contacted by Posturing and Angry US politician(s) who wants to fight WikiLeaks using intimidation tactics ("you are either with us, or you are with.... terrorists!"). Sort of like, you know, people who use terror as a weapon to further their agenda.
  3. Shortly afterwards AWS terminated hosting of said content citing "probable cause for copyright infringement", without actual request for doing so (pro-actively) -- essentially claiming WikiLeaks was "guilty until proven innocent", but without giving them a chance to present any proof.

Now: the way I see it, one of two things happened to effect step 3: either Amazon agreed to do what Lieber"man" et al asked (but lied about not having done that); or Amazon wanted to pro-actively tackle issue they knew would become problematic (opportunistic), using some suitable weasel-word section from the contract.

What should have happened is simple: AWS should have done nothing before officials presented them with a court order or valid cease-and-desist letter (or whatever equivalent is done for Patriot Act requests); and if that happened, publicly announce what they did and why. This is what other companies have done (Google, Yahoo). Or in cases of copyright infringement, similar demand by (alleged) copyright holder, accompanied with court order or whatever DMCA requires. One would think this would be easy for government to do as well for content it has produced.

So why did this not happen? Since I have no idea what sort of backchannel communication resulted in what happened, best I can do is speculate. My two favourite suggestions are that either someone called favors; or that some mid-level manager made a panic decision.

Painful, very painful to watch. It's as if someone gave themselves a wedgie just to prevent bullies from doing it...

Related Blogs

(by Author (topics))

Powered By

About me

  • I am known as Cowtowncoder
  • Contact me
Check my profile to learn more.