Wednesday, November 14, 2007

Walls and bridges

Last time I told you about the FunctionChain utility I wrote on top of google-collections. But I discovered a very strange problem, so I have posted a fixed version and I'd like to tell you about the problem, since I learned a lot while dealing with it.

I got a Function instance and I want to find out whether its apply method accepts nulls. In order to do that, I need to check the existence of @Nullable annotation on the first parameter of the apply method. The annotation's retention policy is run-time, so the task sounds pretty trivial, right? Wrong!

How do I get hold of the apply method? At first I thought function.getClass().getMethod("apply", Object.class) should do the job, you know, because of erasure. And indeed a method is always returned, only it was sometimes missing the annotations. This was driving me crazy, until I looked at all the methods of function using reflection. Guess what - there are sometimes multiple apply methods. How could it be - it's an anonymous inner class implementing the Function interface alone...? And then the flashback hit me - right, synthetic bridge methods! If I defined my function to transform Integer to String, there would be a String apply(Integer) method in addition to Object apply(Object). And if I defined my function to transform an Object to Class (equivalent of Object.getClass()) then I got 2 apply methods taking Object, one returns an Object and the other one a Class. Unbelievable, ha?!

Another thing worth noting, is that the annotation is present only on the original method, not on the bridge method. I have browsed the web and found 2 references to similar problem here and here. Both Michael Ernst and Rob Harrop think that not copying annotations to bridge method is a javac bug, but Sun Bug Database has no trace, so I'm going to submit one.

Ok, at least I knew now what's going on, but how do I find out the right method? I decided that the only way of doing it is figuring out the first parameter in the Function generic definition. I used Neal Gafter's super-type-tokens trick with some minor enhancements - array support and ability to process arbitrary number of parameters. It allows me to find out the classes that substitute generic parameter definitions, and once I get the first one - it's the apply method parameter class, so I'll use it to look up the method. The apply method return type should not be a problem, because according to getMethod JavaDoc the method with a more specific return type will be found (and I can't imagine how there can be more than two methods with same parameter in our case).

I had to change Neal's code a bit, because while he's using getClass().getGenericSuperclass() I had to use getClass().getGenericInterfaces() and to find the one representing Function. I noticed that interfaces need to be manually collected from all super-classes, because each class would only return the ones it explicitly implements, and (usually) not the ones that the super-class implements. Bummer... To add insult to injury - there may be several implemented Types that represent a Function - for example if the parent implements Function<T,String> and child implements Function<Integer,String>, those would be 2 different types, and I, of course, am looking for the most specific one. LinkedHashSet did the job. Phew.

You can look at the source code here, let me know what you think. All in all, it turned out to be much harder than I originally expected, and I am wondering if google-collection folks would be willing to add similar functionality to their package... after all run-time retention of @Nullable is useless without it.


Neal Gafter said...

It is a bug that the runtime reflection libraries are too weak to describe the program's actual structure - which methods override which other methods from the programmer's perspective. The user annotated the apply(String) method, not apply(Object), so apply(String) is where the annotation should appear. The strategy people are currently using works most of the time, but not always. Javac processes the same class files but understands the program's structure and the relationship between the types, so it is possible to just ask "please return the final overrider of this (possibly generic) interface method in this given class". The reflection APIs don't provide access to this computation.

It would be a good idea to add something to the reflection APIs so that you can ask which (non-synthetic) method in a class type implements a given method in an interface type. That should properly handle generics and bridge methods. I think this would solve the problem without duplicating data in the class file.

Yardena said...

Hi Neal,
First of all I am flattered that you found time to look at this :-)

Secondly, I agree that reflection corresponds to bytecode and not to the original source, and that's Ok. I want to clarify though that when I said "copying" I didn't mean literally duplicating annotation data, what I was asking for is when user requests annotations of a synthetic method, return annotations of the corresponding non-synthetic method. Whether they would be copied or "linked" is an underlying implementation detail in my opinion. Here's why I expect such behavior - I would want to see annotations and message body treated the same way, because when using Dependency Injection or Aspects for example, annotations are used to construct method behavior alongside its body. Since reflective invocation of the bridge method redirects to the actual implementation, I would expect the same behavior with regards to annotations. That's my line of thinking, anyway.
BTW I have opened a request for enhancement which was assigned internal review ID 1110336, but have not yet heard back from Sun.

Alienisty said...

It would probably be enough if the Method class could return a reference to the bridged Method when isBridge() returns true.