Red and Sensual Java

Sayeret Lambda

2010-03-03T12:20:00.007+02:00

After delivering the keynote at JavaEdge2009, Ted Neward asserted that Israel is ready for the "polyglot" era. Ronen, Ophir and I decided to verify this assertion, so we are happy to announce the creation of "Sayeret Lambda", the Israeli "Lambda Lounge"! The group contains researchers, consultants, students, and industry professionals, interested in Scala, Clojure, Erlang, Lisp, Prolog, Smalltalk, Ruby, Fan, Groovy and so on...

Our first meeting is going to take place on March 10th, and is dedicated to Lisp and Clojure. In April we are going to host Gilad Bracha's talk about Newspeak. Both talks will be in Hebrew.

Programming language geeks in Israel are welcome to join us!

Lessons we learn, lessons we teach

2009-05-19T19:59:00.027+03:00

Lately I've been a little involved in both (learning and teaching that is), and wanted to share some thoughts on the subject.

Do you like science fiction? I used to like it (when I was younger, so much younger than today). Asimov, Bradbury and such. It's especially fun when the story mixes past, future and present; reality and imagination. What fascinates me most in these stories, is how the author is making deep and painful observations about reality through moving the characters into a completely unrealistic setting - future comes handy, because it is hardest to imagine . It's just amazing how, by disguising things beyond recognition, the author breaks the thought conventions and emotional associations built in our brain, and doing that lets us understand the reality beyond what we were able to before. John Lennon said "nothing you can see that isn't shown", right, but you can imagine what you don't see, and that's how you "know the unknown" a little bit better.

There's this recurring theme in sci-fi stories "smart man builds machine, machine becomes smarter than man, machine rules man". Is it a real threat? If so, how can we prevent it? Should we stop building smart machines? Is high-tech going to destroy mankind? I don't think so. For one, if I did think so, I would quit and become an organic farmer, and I haven't done so... so far. But the question does bother me. And the answer is, I think, that if we want to continue building smarter machines, we absolutely need to keep building smarter people, or we'll end up like in those sci-fi stories.

The main purpose of education is often perceived as passing on what we already know to the next generation, so they don't waste time on rediscovering it. But it's only secondary. The real goal should be passing on the ability to explore the unknown and solve what previous generation haven't solved. Because if students can think at least as well as their teachers, rediscovering something will be just a small detour for them on the road to Knowledge. But without a developed mind they have no legs to walk that road. And it's a steep road up, so when they stop, they (inadvertently perhaps, but) inevitably slip to the bottom. So how do we nourish the ability to think?

It's not what we know, it is how we learned it.

When asking ourselves how to teach, we should first turn to introspection - how did we learn? If the way we've been taught made us discover all those great things that we are so anxious to pass on, why don't we teach what we've been taught? Sure, we will throw in a bit or two of what we've discovered, but basically why don't we build from the same grounds? They say it about parenting - if you like the way you were raised, you'll likely be a good parent, because you'll repeat what you saw. However, if you look around in what's going on in education system, in almost every level, you see the curriculum constantly changing, "updating", "modernizing" etc. "It was true then, it's wrong now. We don't need it." Why? Because we have the technology? Big mistake. Ancient Greeks, Hebrews, Egyptians, not having the technology, weren't even tiny bit stupider than us. Without the wheel, there would be no Internet, speaking of which, who invented the Net? But no, we don't need old men, old books and such, religion, history... we have TV commercials to teach us how to live, we have the technology. And once technology costs money, suppliers of technology don't want us thinking independently, because, Google forbid, we may decide we don't need it! In computer science, this is the nightmare sci-fi writers were warning us against, and we should at least try to prevent it from happening.

We can't go forward without understanding the past. Students can't possibly understand Java and Object Orientation before they understand procedural programming, functions, math, logic. Then, when (and if) we show them objects, let's show how they came about, and not a popular imitation. If we have a great book and curriculum that generations of computer scientists and engineers grew on, why are we throwing it away? Let Java, Python and their patrons wait. They will lay their heavy paws on the students in just very few years anyway and will turn them in Dilberts, Wallies, and Alices, converting large XML files to long stack traces. Let's give freedom of thought and curiousity a chance to grow just a little bit in students' minds, so at least some of it can survive through corporate development.

It's not what a technology does, it is how and why it works

We're all the time obsessively looking for solutions to our problems. We barely stop and analyze them, until the solution itself becomes our biggest problem. I watched this QCon presentation a while back, and it was a deja vu in many senses. I encountered problems like that, and I even solved them in somewhat similar way. I may be wrong, but what I get from the presentation on the technical level, is that sometimes Object Orientation as we know it (C#, Java), with all the patterns and practices and such, does not solve our problem. The problem in the presentation reminds me of the expression problem - data and operation-set need to evolve, how do we express the relationships? The proposed solution (although I may be getting it wrong) is to have interface per operation; then the implementation of the interface, using reified generic type parameter, stores the type of object it applies to; then at start-up something wires together data types and implementations; then it all becomes a big happy family of multi-methods, operation implementation chosen via dynamic dispatch on the data type. And an old saying goes "when the problem is hard enough, you will find yourself re-inventing Lisp to solve it". It's interesting how Udi describes arriving at this design - a team of people were struggling with the problem for years, until an "old programmer" came to the project retrospective meeting and said "make your roles explicit". Udi took the guy aside and made him explain. Then Udi (and his team, I suppose) implemented it, and now reported the success at QCon. Who was that "old geezer"? What was he? Udi didn't say, in the presentation he is portrayed as a little green troll. Anyway, why do I care who that mysterious character was? Because I didn't come to computer science for some wise Merlin to tell me where the Holy Grail is, I want to be that Merlin! Unfortunately, I am probably not smart enough, but someone else is. That's why I think we need more wanna-be Merlins; wanna-be Kings we already have plenty.

Since I became interested in functional programming, for example, my Java coding style changed significantly. I thought I knew how to use Generics, only after a taste of OCaml and Benjamin Pierce's writings I realized how little I knew about types. I thought I knew how to write object-oriented programs, only after playing with Smalltalk I realized what object oriented really meant. I may not be able to use Haskell for the day job, but the concepts of immutability, closures, function composition, laziness, are helpful no matter what language I use. Let's look at one example - the (in)famous problem of object-oriented design: does square extend rectangle, or in other words does circle extend ellipse? In both cases the former has 2 distinct properties (edge sizes or focal points) and the latter has only 1. Bob Martin discusses the problem in his article "Design Principles and Design Patterns". Does he offer a solution? No - "design by contract" and Eiffel and some hand-waving. On the other hand, understanding types helps, because then I can "design for subsumption": I ask myself - if Ellipse is a type that describes all ellipses in the world, do they all have 2 distinct focal points? No. Then maybe I shouldn't have methods to get and set them. Maybe my API should try to follow the definition of ellipse more precisely. That surely helps to design good APIs. Furthermore, magically, once we turn the shapes to immutable, the problem almost goes away. If Rectangle has a method Rectangle transform(x,y) that produces a new Rectangle with given sizes, then Square can inherit it with no problem, it would simply produce a Rectangle, not a Square when x != y. Same trick would work for Ellipse. After all, shapes are math definitions, why should they be mutable?! See, a bit of "functional" thinking solved the problem. And the moral - understanding the classic foundation of computer science is necessary for programmers.

Reality distracts clarity of thought.

If there is one more thing that we can learn from these sci-fi stories, it's that detaching things from reality may actually increase our ability to grasp them. However education in recent years is insisting on "examples", or worse - "realistic examples", or even worse "examples of being used in the industry". I am not saying the above is worthless, or unnecessary, but it should not be overrated. At some point education ministry decided to teach elementary school math with actual objects - sticks and such, rather than teaching kids the abstract idea of numbers, and it was a disaster. Math, in general, cannot be taught by following "real world" intuition. Nor logic. Physics has evolved way beyond relatively "intuitive" mechanics. So why are so many educational institutions chasing "real life" technologies, at the expense of classics, and ignoring the "too innovative to be popular"? I know why, of course, - money, pressure from the industry, pressure from students who want real jobs after they graduate. But resisting that pressure is absolutely necessary, for the sake of future generations, to save our civilization! Luckily we still have some universities in Northern Europe :-)

I noticed an interesting phenomena with students - when they are "fresh" and don't carry a baggage of "field experience" (C, Java, curly braces etc.), it is easier for them to take a "different" point of view, to understand more abstract ideas. Generics is one example. Teaching them to experienced Java programmers is extremely hard. But with undergraduates it is, surprisingly, much easier. So why don't we teach Haskell for types, Smalltalk for objects, and maybe C for low-level stuff? Then when they meet Java, or any language they will likely encounter in the industry, it will be a piece of cake to learn. Furthermore, some of them will be able to design the next Java!

Java nested classes - tips and tricks

2009-05-19T16:51:00.014+03:00

I had to prepare this anyway, so I thought I might as well post.

First of all to get the terminology straight: normal classes and interfaces are top-level. But class or interface can also be nested, if it is defined inside another class or interface. Nested classes originate in Beta programming language, and are available in Java since version 1.1. Non-static nested classes are called inner. Inner classes can be members (declared immediately inside outer class definitions), like this:

class Outer { class Inner { ... } }

Inner classes can also be declared within methods and other blocks, then they are called local, like:

class Outer { void foo() { class Local { ... } ... }

A more popular breed of local inner classes are anonymous classes:

class Outer { void foo() { ... new Bar(...) { ... } ... }

For visual impression - check out this diagram.

Tip - construction: Creating new instance of nested or inner class within the outer class is simple. From outside it's a bit trickier, suppose we have a class like this:

class Outer {
static class Nested { ... }
class Inner { ... }
}

We can reference and instantiate the classes like this:

Outer.Nested nested = new Outer.Nested(...);
Outer out = ...
out.Inner in = out.new Inner(...); //translated by javac to new Inner(out, ...)

The magic behind inner class constructor is that compiler implicitly adds Outer parameter to all Inner constructors, and passes the enclosing instance when constructor is invoked. From then on, inner class instance (for its entire lifetime) holds a strong reference to the enclosing instance.

Tip - instanceOf: If we have two distinct instances Outer out1, out2 then out1.Inner and out2.Inner denote the same class, but Inner instances will refer to a different enclosing instances. This is different from Scala and Newspeak, where inner class is distinct for every enclosing instance.

Tip - access enclosing instance: To access the instance of outer class from within a contained inner class use Outer.this. Nested/inner class methods/fields hide outer class ones, to access outer class elements prefix them with Outer class name, e.g. Outer.staticMethod(...) or Outer.this.anyMethod(...).

Tip - nesting and inheritance: Generally, method lookup rules in Java nested classes follow "comb semantics" - first search inheritance hierarchy, then enclosing lexical scopes. This behavior can introduce some wierd puzzlers, like #9 here. In Newspeak the enclosing scope is considered before inheritance, which makes it easier to follow from programmer's perspective.

Tip - generics: Generics type parameters of enclosing class (or method) can be used within inner classes.

Tip - interfaces: Interfaces may have nested classes (necessarily and implicitly static), it may be particurlaly useful for declaring nested enums.

Tip - statics: Inner class cannot have static declarations in it, except compile-time constants. To overcome this, static declarations canbe moved to the top-level class.

Trick - loading: Nested class is treated just as any other class by the JVM, e.g. it is not loaded/initialized until used. This fact is used to implement thread-safe lazy singletons using the Holder pattern.

Tip - final: Anonymous inner classes and local classes can access variables in the surrounding scope only if the variables are final:

void invokeAnon(final int number) {
   final String word = “hello”;
   someObject.pass(new Runnable() {
   public void run() {
   System.out.println(word.substring(number));
   }
   });
}

Trick - double braces: Double brace initialization is a trick of putting initialization block inside anonymous inner class declaration, like:

   final Map numbers = new HashMap(){{
   put("one", 1);
   put("two", 2);
   put("three", 3);
   //...
   }};

Trick - tokens: A cool Generics trick that uses local class to capture type parameters is super-type-token, a.k.a. Gafter Gadget.

Tip - reflection: Starting from Java 5 a bunch of methods have been added to reflection with regards to nested classes. For example, Class#getEnclosingClass() method will let you find out the enclosing class for an inner class, for example:

class Enigma {
final static Class MY_CLASS = new Object(){}.getClass().getEnclosingClass();
}

Prolog: Last, but not least JLS is the ultimate resource for finding out more.

Community Choice Award

2009-05-08T12:13:00.011+03:00

I nominated Newspeak Programming Language for "Most Likely to Change the Way You Do Everything" Community Choice Award. I don't know how exactly they choose winners, apparently:

The first phase will be to nominate finalists for each of the Categories. Nominations will be accepted at ... between May 6, 2009 at 10:00 a.m. PDT and May 29, 2009 at 4:00 PDT. Among the nominees for each of the Categories, the finalists for the Awards will be chosen. Voting for the final winners will commence at ... on June 22, 2009 at 10:00 a.m. PDT and end on July 20, 2009 at 4:00 pm PDT. ... The odds of winning in any category are dependent upon the total number of eligible nominations received.

Anyway, clicking the orange bot on Newspeak sourceforge page might help the odds - I ask all my readers to contribute a click for a good cause!

Thank you.

UPDATE: Why does Newspeak deserve it? Newspeak is a class-based dynamically-typed object-orientated language that revives the ideals of Smalltalk and Self. It incorporates many great ideas, but one of the major innovations is its modularity support.

No other language or framework today provides comprehensive solution for creating modular software. Some languages support hierarchical code organization, there are tools that build components, tools that manage dependencies between components, yet another set of tools and formats deal with module deployment, there are platforms and tools that facilitate versioning and patching, hot and cold updates. Newspeak has it all - the language, development environment, and platform together provide easy and intuitive end-to-end modularity support.

Newspeak supports both mixin-based inheritance and class nesting - modules are top level classes, while all other classes are nested in them. Dependencies between modules are specified using constructors, the absence of global (static) scope enforces complete isolation of modules and prevents creation of incidental or implicit dependencies. Since all objects communicate via virtual method invocations, there is no hard wired dependency on a particular module implementation. Everything is virtual, including the parent-child relationship between classes, which allows for great flexibility and extensibility. The platform supports construction, serialization and loading of module instances, and therefore effectively supports building and deploying applications without the need for any external tools (even though some of this is still under development). Dynamic platform underneath Newspeak has rich meta-programming support and allows querying module definitions, extending modules and supports hot (incremental) updates. Multiple versions of the same module can coexist without interference. Security is maintained by capability-based model where access to resources is guarded by capability objects (also under development). Modules may access their execution environment (the virtual machine, or platform) and through it interact with external resources. Newspeak is also network-aware, and is designed to support distributed component management using service objects.

Newspeak is open-source, it was not widely, but successfully, used in an industrial environment, until financial situation deteriorated and corporations turned their back on funding innovation. There are several publications, and more on the way, conference presentations are received with great excitement. Newspeak is modern, it combines "best-of-breed" ideas of computer science and decades of Smalltalk and Java practical experience. It is easy to learn and very pleasant to code in (and not just for Smalltalkers and programming languages afficonados, but also for averagely skilled Java programmers like yours truly). Newspeak philosophy is inspiring. The people who work on it are extremely smart, but also nice and cool guys... Need I say more?

Oh Null, Null

2009-03-31T15:06:00.013+03:00

Introduction

Some say it’s one of the worst inventions in history of programming languages, some say it is number one cause of errors in Java programs. Anyway you look at it - null pointer (in all its incarnations) is a sensitive subject. I wonder why there is no song about it, there are several funny re-works of Beatles songs with geeky lyrics, like Unix Man (Nowhere Man, “he’s a real Unix man sitting in his Unix LAN, making all his UNIX .plans for nobody…”), Write In C (Let It Be), Yesterday (“yesterday, all those backups seemed a waste of pay…”), Something ("Something in the way it fails, defies the algorithm logic...") and a not so funny Eleanor Rigby. I’d like to write a new song, “Null” using the “Girl” tune. Only my inspiration ended after a single verse:

If there’s anybody going to listen to my story,
All about the Null who brought the fall.
It’s a value that you want so much, it makes you sorry,
Still you don’t regret a single call.
Oh Null, Null…

I also have a middle verse inspired by Smalltalk, where the equivalent of null is nil, a keyword that returns an object of class UndefinedObject. It has no methods, except isNil and notNil that return true and false respectively, so nil responds to any other method invocation with “DoesNotUnderstand: <method name>”, and in my experience, at least, pretty much equivalent to NPE in Java. Interestingly enough, though, Objective-C turned nil into a “black hole” that returns self upon any invocation.

Nil’s a kind of null, she is not answering your calls,
you feel a fool (fool fool).
When you think the code looks good, she answers “it’s not understood”,
she’s cruel (cruel cruel).
Ah null, null…

From Beatles to Elvis

One situation where nulls are very annoying, is when we have a “chain” of method invocations, such as account.getOwner().getAddress().getStreet(). If any of the methods return null, we get NullPointerException from the next method invocation. I previously suggested a solution to the problem, but now (due to popular demand) there is a proposal for Java 7 language enhancement, called The Elvis Operators. One of the stronger arguments for this solution is that it is available in other languages like C# and Groovy, even if not everybody is entirely happy with it. Perhaps this is subjective, but after all, programming language design is about human perception and feelings, as much as about computer capabilities. So, I admit: I feel uneasy with this new way of method invocation.

The Demeter ~~Law~~ Suggestion

One of the more controversial guidelines of object-oriented programming, Demeter Law is related to “Tell Don’t Ask” and usually summarized as “only talk to your close friends”. It prohibits series of method invocations like the one described above; in fact, more than one dot in series of method invocations is disallowed (unless the chain originates in a this object’s member (?)). There’s a good description of the “The Law” in Pragmatic Programmers article; it was originally discovered and named by Karl Lieberherr, who created the Demeter Project and wrote the book named “Adaptive Object-Oriented Software - The Demeter Method”. Martin Fowler demoted the Law to Suggestion, and in general, Demeter managed to generate a lot of controversy and debates inside both Java and Ruby communities, the debates ranging from intellectual (almost academic) polemics, to flamed battles between consulting firms over preferences in programming styles, music and hairdo. Yes, really. Anyway, if you believe in Demeter, our use-case for “null-safe” method invocations is flawed to begin with. But do we believe in Demeter? It took me a while to form an opinion.

1) I do believe in “Tell, Don’t Ask” and the internal iterator (as presented in Pragmatic Programmers article) to be preferable over external iterator (when applicable, of course).

“Tell Don’t Ask” begs for closures in the language. For those who say “closures are not object-oriented”, Smalltalk not only supports closures, and with minimal syntactic overhead, but basic things like if and while are designed around closures and would not be possible otherwise.

But "Tell Don't Ask" goes much deeper and wider. For example let's look through Demeter's eyes on service locator vs. injection: locator is wrong, because you go to a factory to get a factory to get a service. Injection is right, because you pass yourself to injector, which sets your the dependencies directly. How about ORM? How much grief is in loading strategies, because we do getContainer().getItem().getProperty(), vs. calling database.execute(query)? Food for thought.

2) Now the paper boy example – this one I do not buy. First of all, we don’t pay for things in cash these days – we give the seller our credit card. Not very secure? Maybe. But much more handy.

People should generally stop obsessing around security in software, I think. At one point a highly ranked architect in the firm I worked for said “we can’t deploy any software in the browser because it is not secure”. Yes, it is not secure, it’s a problem and people are working on it. And yet, even as it is, is it secure for the jeweler store to put some jewels in the front window? Why not keep them all in the backroom safe? Because they won’t have buyers, that’s why. What good is all the security, if you go out of business?

Back to paper boy, the example also does not hold if you are a large organization – the receiver of the service is not the one who authorizes the payment, and not the one who handles the payment. The supplier may wish for “direct communication” as much as he wants, but the rules of corporate procurement are quite different. Then again, it may be silly to attack the example…? Well, the thing is that I hope to show later that you can’t follow Demeter and scale, just like the example doesn’t.

3) Now there’s “hard to mock” argument. Ok, at risk of starting a flame war here, I mean: I love unit-testing just like the next guy, and I know how useful it is, but let’s not get carried away here.

Unit-testing, just like static types/compilation, or static analysis, is a means to achieve code quality, not the goal in itself. If it helps you – great, if it gets in your way – ditch it. I think the “100% test coverage” is a fallacy. Dijkstra said “tests only prove presence of bugs, never their absence”, which is even truer for unit-tests. (I heard “we have unit-tests, we don’t need QA” argument once or twice. Yeah, right.)

Anyway, “hard to test” may be a smell, but unless the suspicion can be substantiated – sorry, circumstantial evidence not accepted.

4) So even though I partially agree that “Demeter violation” is a symptom of a problem, the solution proposed by Demeter is, in my opinion, simply absurd. The grand huge fat mega-façade? Dots replaced with underscores? It cannot scale and it is sweeping design problems under the carpet. I mean, give me hierarchies of nested objects that support subsumption, like Newspeak modules – now we are talking. But mega-façades? No thanks.

So there are good things that result from minding Demeter, but not following the law per se.

a) For example, “don’t return a java.util.List from your API when you don’t want clients adding elements to the list” – that’s basics of any sound API design – return the minimum interface that the client needs, Josh Bloch has been saying this repeatedly.

b) We should consider to proxy the returned object (perhaps with our own (inner) class), and then we’ll have control over what the returned object’s methods do.

c) Applied to DI – “do not depend on module A just to give you access to another module B, instead depend on B directly”, is also a reasonable rule of thumb.

But let’s now go back to our beloved nulls.

Power to the method - let the declaration site decide

Demeter or not, methods are meant to do things. I would argue that typical methods should not be assumed to be wrapping field access, or performing trivial computations. Yeah, I know we all do POJOs, which are really POCS, Plain Old C Structs in disguise. (And what makes them structs are frameworks that assume a dumb accessor and mutator, and freak out if they are not.) I believe that most methods should do something, so invoking them on a null target should yield an error, not some ambivalent null… So I am not too happy with changing the programming language to accommodate to behavior that should have been atypical.

Now if some methods are “dumb” and have reasonable defaults (like null for property accessor in null object, or 0 for size/length of a null collection/array, or null for a null string upper case transformation), then the method implementer should decide – not the caller. And again, it’s not problem-free, but worth exploring. Somewhat similar behavior has been proposed by Jacek here.

Suppose at the method declaration we add a default clause: either default <value>, or default <expression>, or default <code-block>, with the return type of value/expression/code-block similar to method return type, and parameters and exceptions (in 2 latter cases) similar to the method’s ones. BTW default is an existing Java keyword, used in switch statements.

Now examples:

public int size() { return this.size; } default 0;

public boolean isEmpty() { return this.size == 0; } default true;

public Person getOwner() { return this.owner; } default null;

public Iterator iterator() { return … } default Collections.emptyList().iterator();

public Something produce(Param1 p1, Param2 p2) throws E1 {

//normal method code here

} default {

//default calculation, similar to static method, can use p1 and p2, throw sub-type of E1

//needs to return Something

}

The location of the “default clause” can be after method body (and if we stick to coding convention of keeping default it on the same line as the closing curly brace, I think it is preferable), but it can sit also between end of method parameter list and the throws clause, or between the throws and the method body start. I think tail location is preferable, because it reminds of the switch statement.

Another variation that increases similarity to switch may be adding a colon (default : ) and requiring return keyword, like default : return null; instead of default null. Also, the support for “defaults” can be added gradually, with only values in the first stage and more later if it proves to be successful. Also, alternative to default may be case null : … .

The method would be compiled now additional synthetic static method, e.g. ___size(), with the same signature and throws clause, the synthetic method returning the “default” body provided by the user.

Unfortunately, though, call sites are affected by such method definition and compiled to something like:

SomeType object;

if (object != null) {

object.m()

} else {

try {

SomeType.___m();

} catch (NoSuchMethodError e) {

throw new NullPointerException();

}

So the “default” would be calculated based on declaration type of the object, just like any static method invocation. Why catching NoSuchMethod? Well, what if SomeType class is recompiled alone, and the default clause, and with it the ___m() method, are gone? We want NPE to be thrown, not NSME.

If, on the other hand, we add the default behavior without recompiling clients, they’ll be still throwing NPEs, I think we would not want all method invocations to become complex byte-code like the one described above… If I understand Jacek’s proposal correctly, unlike me, he suggests every method invocation to go through a static method.

It would be interesting to see to what extent JIT could optimize code like that when it learns that object is not null… could it throw away everything but object.m()? This certainly requires more investigation.

Maybe, babe, boom

Another option would be to turn to Functional Programming for an answer. This is one of the things known in FP world for ages: Maybe in Haskell, or Option in Scala and F#. Here’s a nice description by Debasish Gosh, and if you want to learn about Monads, James Iry’s blog or a Haskell book would be a good place to start. But we don’t have Monads in Java, Stephan Schmidt tried to simulate Maybe with Iterable, and the result is cute, very cute even, but lacks the monadic awesomeness of flatMap, as others have pointed out. Hm, but wait, we do have a monad in Java. That’s exceptions. The naïve translation of Maybe in Java would be:

String street;

try {

street = account.getOwner().getAddress().getStreet();

} catch (NullPointerException e) {

street = null;

}

Ok, it’s not so clean and pretty as pattern match, but… well, that’s Java. No, wait, smart and experienced programmers like us – we know that we shouldn’t use exceptions that way, because it is expensive! Hm, and why is it? Because filling in the stack trace, actually, otherwise it’s rather cheap. But we don’t need stack trace here, only… the exception is the standard NPE, we cannot subclass it and override fillInStackTrace(). Bummer.

Using exceptions for control flow is debatable. I am not a proponent of this style, just for the record. I am simply exploring options… what if the JVM “magically” knew not to fill in stack trace when it’s not used? Exceptions would become cheaper and whole new programming style in Java would emerge...?

Some of the Java 7 language enhancement proposals are to do with exceptions, multi-catch for example, I am all for it BTW. Now what if we add another flavor to catch and introduce “no-instance” catch, such as:

try {

//…

} catch (NullPointerException) {

//…

}

Note that there is no e, no exception instance, no way to re-throw or print – clearly no need for a stack trace. What if JVM kept a bit-map of exception types in “no-instance” try/catch blocks on the stack, and threw these exceptions without filling in the stack trace? After all, we are guaranteed that they are caught and no “phantom” exception like this escapes to the user… these exceptions could be pooled too. Then we could say

String street;

try {

street = account.getOwner().getAddress().getStreet();

} catch (NullPointerException) {

street = null;

}

And if we had closures, we could even make it prettier. We could create a utility method like

public static <T> T nullsafe({ => T} expression) {

return nullsafe(expression, null);

}

public static <T> T nullsafe({ => T} expression, T default_value) {

try {

return expression.invoke()

} catch (NullPointerException) {

return default_value;

}

It would be invoked like this:

String street = nullsafe({ => account.getOwner().getAddress().getStreet() });

In all its glorious maybiness and with no, or little, performance overhead?! :-) But hey, I said I don’t like clients deciding on the value that the method returns… true, which is why I am not entirely happy with any of the proposals. But at least here we don’t change the language for the sake of null alone – the “exceptional” invocation is explicitly made with a catch clause or via a special library call.

Epilog

All this just for the sake of intellectual exercise, so don't take too seriously. :-)

Update:

So why I don't like any of the solutions, not even the ones I proposed? The "default clause" is essentially another static method. It may even work for interfaces, with a slight twist. The real problem is that static cannot be overridden, unlike the method it is "attached to", so the default clause and the actual method body will not correspond, so we basically gained nothing.

As for the instance-less exception, call me old-fashined, but I think that exceptions are for exceptional things, and this is stretching the hacks around them just one bit too far.

So I'll stay with plain old null for now, thank you.

The return of forgotten design patterns

2009-01-09T17:35:00.008+02:00

Some design patterns are used all the time and their names are known to all - like facade, factories and proxies. Some design patterns are more popular than they should be. But some, although rarely mentioned by their name, have been recently "rediscovered". I'm talking about Flyweight and Memento.

Flyweight
This one basically lets us share n instances between m concurrent clients when m > n. It separates "intrinistic state", the normal class members, and "extrinistic state", which is maintained via parameter passing and return values. Make your intrinistic state immutable, and you can share the same instance between multiple clients. Cool, ha? Priceless. Look at message passing concurrency instead of shared memory concurrency, REST, transactionless architecture... All these treasures actually follow the same spirit as Flyweight.

Memento
This pattern suggests that if you want to save your object state, you better export it in a new dedicated memento object and store the memento. Then restore your object from the memento.

This is like serialization, only serialization didn't follow the pattern, unfortunately.

Josh Bloch suggests we do it manually with so called "Serialization Proxies" - see item 78 in chapter 11 of 2nd edition of "Effective Java". Here's a slide from JavaOne 2006 preso:

The book lists more advantages of the pattern, like improved security (see item 76 - danger of hackers acquiring references to private fields using de-serialization), ability to de-serialize a different class from the originally serialized instance (RegularEnumSet and JumboEnumSet example) etc.The name "memento" isn't mentioned though.

Now imagine persistence architectures actually using intermediate memento objects... instead of modifying actual objects bytecode, breaking encapsulation with access to their private fields, imposing constraints like public empty constructors and so on... Maybe we would have been better off with mementos...?

Static types are from Mars, Dynamic types are from Venus

2009-01-09T13:22:00.006+02:00

Venus and Mars

static - associated with logic and acting by the rules, strong, efficient, usually responsible for safety and order enforcement; but non-compromising (for better or worse), non-adaptive, weak in communication skills.

dynamic - associated with beauty and elegance, light, good in communication skills, can usually be easily made to do what you want them to, change all the time and adapt to change well; but unpredictable, act on intuition rather than logic, often seen as less efficient and weaker.

Types and components

2009-01-07T19:10:00.026+02:00

Just some thoughts following a recent conversation I had. Don't we always want static types? If we can detect errors in our program, why not do it as early as possible? Sure. So when is "as early as possible"? I think the answer depends on what our program is - is it one monolithic piece or a component?

In the past monolithic software was prevalent, but today I would bet that most software is meant to be a component. Just look at open-source as it used to be before Java - you'd usually download the C source-code, change it if you need, build it all on your machine, and create one big executable. (And BTW - congratulations, you are a geek.) Cross platform Java changed the situation. Now open-source projects give you a jar to download, you put it on your classpath and you are ready to go. This lowered the bar for open-source adoption, and contributed (aside less restrictive licenses and other factors) to baby-boom of open-source frameworks and tools. Nowadays a Java developer cannot even imagine having no 3rd party jars in the classpath!

Static type check validates our software component against other components in the compilation environment. Does it match the runtime environment? What about different configurations of the runtime environment - there are tests and real deployments, and various types of deployments, and any given installation environment can change over time - new components being added, other updated or removed? How do we guarantee that compile time checks still hold? The short answer is - we can't. We need dynamic type safety anyway. Now let's examine the added value and the price (yes, there is one!) of deeply static types.

On code organization level, we try to reduce the dependencies to bare minimum - hide classes behind interfaces that we hope will remain more stable. The problem is that number of interfaces and factories in our application grows, while pursuing modularity we sacrifice simplicity... So maybe the problem is in the name? Some go as far as add support for structural types, minimizing dependency to a single field/method signature (not a problem-free solution, but there are interesting refinements). All this may help, but doesn't really solve the problem.

Another aspect we need to deal with is building and packaging the software. Here we enter the world of dependency management, the world of "make", Ant, Maven, repositories, jar versions; if it's a large enough and complex enough software we work on, simply speaking - we enter the world of pain. I still find it strange that we haven't found a better way.

As for application deployment and its problems, we'll get back to it later. But the truth is that no matter how hard we try, we can't guarantee there will be no errors when we deploy our software, so ... JVM doesn't trust us and gives us verification.

Simply put, when class is compiled, some of its requirements from other classes are captured and encoded into the bytecode. Then JVM would check them when the class is loaded, and reject the class if they can't be met. (This is really an over-simplified description of a complex algorithm, which also takes time to execute, despite optimization efforts on JVM side.) So this isn't really a dynamic check, it's something in between - names in our class get linked when it is loaded. In the classic Java SE class-loading scheme, where components are basically a chain, this scheme should work. But if we want real components, ones we can add, override, replace or remove while program is running - sweet turns sour. Our interfaces and factories have names, and classes that represent them need to reside in some "common vocabulary" usually loaded by the parent classloader, because it's not only the class bytecode that matters, but also who loaded what. Since we are talking actual classes, not their names, once we loaded two components, they cannot change their protocol of communication without reloading their parent, they also can't use a different version of a sub-component that the parent component has referenced.

In JEE that sort of things is necessary, that's why classloading in JEE is a terrible mess, not only it does not follow any specification, but it is different in almost each and every app server (wasn't there supposed to be portability?!) If you ever used commons-logging in a JEE app, you probably know what I mean. Maybe it got fixed lately, I don't know, but the Tech Guide for commons-logging is an ode to classloader frustration.

Back to deployment: whenever there are some sort of dynamic components - JEE, Spring or OSGi, there is always reflection. And most of the time there's lots of XML too. It's an escape route from static types. I attended Alef Arendsen's session at JavaEdge that presented OSGi and SpringSource. I carefully watched Alef juggle between XML, source and console like a child who watches a circus magician trying to uncover his tricks. But I didn't quite figure out the magic. And that was a whole session just for HelloWorld. I know Spring folks are doing best they can, and they're smart and all... but comparing to Smalltalk, I wasn't quite impressed. As for other solutions, although I haven't tried this out, there's Guice/OSGi integration without XML and with dynamic proxies and on-the-fly bytecode generation with ASM, but there's some overhead for the user, because it requires intermediate objects for services. So this way or the other, looks like JVM platform is holding us back.

Verification is addition, not replacement of dynamic checks. So what we get is basically a triple check of correctness (javac, verifier, dynamic) but loss of flexibility - we are interfering with components runtime life-cycles. If the invocation target is resolved just in time when the call is made, nothing precludes the target component from being reloaded between calls. But with preemptive validation, we get a static dependency tree at runtime, classes wired with each other "too early" and for good, which makes reloading a component very hard (although people keep trying). The reason for "early linking" is also performance, but late binding doesn't mean that the runtime platform can't do any optimization heuristics... but they'll have to be dynamic optimizations in the style of JIT. Will invokedynamic bring the salvation?

It seems when we are talking about multiple components, "statically typed platform" does not quite do the job. Static type check may mean a lot inside a component, but as for inter-component communication they are not only useless, but harmful. People sometimes dismiss dynamic types, because they think "it's like static types, but without static types". What they may not realize is that you are not just loosing, you are gaining something with dynamic types. You get late binding and meta-programming, and in a multi-component environment, it means a whole lot!

And that's when we are talking "inside the platform" components developed in the same language. Once you work with a system that runs on a different platform or developed in a different language - our type system doesn't normally stretch across the communication boundary. The other system may not even have static types, and since we are only as strong as the weakest link, our static types don't really help us. I think every time we try to encode types into communication between systems we end up with a monster like CORBA or Web Services. But there's another (unfortunately popular) extreme of just sending a string over and hoping for the best - with no checking on our side at all. Then we are relying on the other system to stop us from doing damage, and there's no way to correctly blame the component that made an error - was it a wrong string or an unexpected change on the other side? I think that ideally type or contract checking and conversions can be done dynamically on both sides, and not as part of the protocol. This results in light and flexible data-exchanging protocols (like HTTP or ATOM) which are easier to work with and I think will win in the end. On the more theoretical level I like this model for intercommunication and of course there are Aliens, that model external system as a special object in our system.

So as far as I see - components simply require a dynamic environment, they may be statically checked inside, but act as "dynamic" to the outside world. Sort of hard skeleton and soft shell. Indeed soft parts are much easier to fit together and less breakable, due to flexibility - this is used often in mechanical engineering and in nature, so why not in software?

How not to implement Comparable

2008-12-11T17:02:00.005+02:00

I have already blogged about the danger of numeric overflows. I recently came across this example in Java course materials (!!!):

public class Foo implements Comparable<Foo> {
private int number;
public int compareTo(Foo o) {
  if (this == o) return 0;
  return number - o.number;
}
}

How do you think Integer.MAX_VALUE compares to negative numbers? It will appear smaller. This reminds me of even worse case we encountered in a real codebase. Look at this:

public class Foo implements Comparable<Foo> {
private long id;
//...
public int compareTo(Foo o) {
  if (this == o) return 0;
  return (int)(id - o.id);
}
public boolean equals(Object o) {
  return o != null && (o instanceOf Foo) && compareTo((Foo)o) == 0;
}
}

How do you think new Foo(8325671243L) and new Foo(25505540427L) compare? They are equal, but I will do it ala Weiqi Gao and leave you to find out why... :-)

Static initializers - update

2008-12-11T16:01:00.004+02:00

After some interesting comments to my previous write-up I decided to make a follow-up post with some additional details.

Here is the code from Effective Java:

public class Person {
 private final Date birthDate;
 //...
 private static final Date BOOM_START;
 private static final Date BOOM_END;

 static {
   Calendar gmtCal = Calendar.getInstance(TimeZone.getTimeZone("GMT"));
   gmtCal.set(1946, Calendar.JANUARY, 1, 0, 0, 0);
   BOOM_START = gmtCal.getTime();
   gmtCal.set(1965, Calendar.JANUARY, 1, 0, 0, 0);
   BOOM_END = gmtCal.getTime();
 }

 public boolean isBabyBoomer() {
   return birthDate.compareTo(BOOM_START) >= 0 &&
     birthDate.compareTo(BOOM_END) < 0;
 }
}

and here's how I would have changed it:

public class BabyBoom {
 private static final BabyBoom boom = new BabyBoom();
 private Date start = null;
 private Date end = null;

 private BabyBoom() {}

 public static BabyBoom getInstance() {
   if (boom.start == null || boom.end == null) {
     Calendar gmtCal = Calendar.getInstance(TimeZone.getTimeZone("GMT"));
     gmtCal.set(1946, Calendar.JANUARY, 1, 0, 0, 0);
     boom.start = gmtCal.getTime();
     gmtCal.set(1965, Calendar.JANUARY, 1, 0, 0, 0);
     boom.end = gmtCal.getTime();
   }
   return boom;
 }

 public boolean contains(Date birthDate) {
   return birthDate.compareTo(start) >= 0 && birthDate.compareTo(end) < 0;
 }
}


public class Person {
 private final Date birthDate;
 //...
 public boolean isBabyBoomer() {
   BabyBoom boom = BabyBoom.getInstance();
   return boom.contains(birthDate);
 }
}

I didn't synchronize getInstance, because initializing the dates twice does no harm, so it's not worth the price of synchronization. However I did check that both fields are initialized before returning the object in getInstance.

A case against static initializers

2008-12-07T22:05:00.005+02:00

"Effective Java" is an excellent book. I recently bought the 2nd edition, and it is absolutely fabulous, priceless. However after quite some time in the industry, I've learnt not to take any advice blindly. First edition was also excellent, but several of the items were revisited since then. So here's an item that I have mixed feelings about - Avoid creating unnecessary objects. The advice is to use static initializers for the expensive computation.

Static initializers are double-edged sword. It's like with the stock exchange in times of crisis - for a particular individual it may be a good idea to sell the stock, but the trouble is that everybody's doing it, and in the end everybody's losing big-time. Same applies to static initializers. One or two may seem harmless, but they add up and together create a big problem. The fact that static initializers ~~are~~ (edit) may be invoked at program start-up affects everybody and since they potentially interfere with classloading, it's ~~very hard~~ (edit) harder to debug them if anything goes wrong.

Here is how it usually gets out of hand: people start with initializing static members in static blocks. Map of values, sort of configuration details. That alone sounds harmless. But soon comes the time when the values in a map need to be read from a properties file, so here we got IO within static block. Uh oh, better catch these exceptions. Before you notice the whole thing turns into a puzzler. Don't believe me? Here's a real problem I had.

Server in Java, client is either applet or JNLP. On certain machines, only one of them can run. You run server first - client never comes up, no error whatsoever. You reboot and connect as client to another machine - no problem. But if you try to start up the server locally - silent death. A team in India spends months on it. In vain. The whole release is detained, escalation to senior management. The thing ends up on my desk after a ruthless blame game between teams. Long story short: it's a DirectX problem. Why the **** does the server need DirectX? Ah. What's next after reading defaults from a properties file? Reading them from the database. Oh, but it's a different process, and the database needs to be up. So we not just connect to database from initialization block, we wait for the database process to be up. Great idea. How? We follow "best coding practices" and reuse: find a poller utility somewhere in the JDK. Apparently there is a java.awt.Timer. Why not? Great idea. Apparently, a touch of one AWT class causes a bunch of other AWT classes to load, which in turn loads DirectX and OpenGL dlls. And guess what - Windows on some machines has a nasty bug, that only allows to load them once per machine, regardless of the user. And when another user tries to do it - the loading gets stuck. And our server is of course a system process, while the client belongs to the logged in user.

Since it was a last minute fix, we solved it with some JVM flags that disabled DirectX and OpenGL. The problem was not the fix, but the diagnosis. If it was part of the regular code, it would have been easy to connect with a debugger, see what call gets stuck, investigate it from there. But as it was part of start-up, people didn't know where to look. Not to mention the man-months accumulatively spent by developers who waited for the server to restart when testing.

So... what's the lesson here? Life is better without static, avoid it as much as you can.

Java... tea?

2008-12-07T14:24:00.000+02:00

No clone for you!

2008-12-07T12:40:00.011+02:00

Java Practices call Cloneable/Object#clone API pathalogical and suggest to avoid it. I have to agree. Here's my little story.

I have a method that receives a certain object, which is Cloneable. Now I want to actually clone that object, so I am calling the clone method. Makes sense, no? Apparently not. My slightly outdated version of IDEA was confused, just as I was, and did not complain. But Javac was cruel and uncompromizing. No clone for you.

Because Cloneable does not require a clone() method, and Object#clone() can be protected. WTF? My objects subclass different not-necessarily-cloneable classes, so they can't have a common cloneable superclass. So I'll define my own ReallyCloneable interface that has a public clone method, sounds simple enough...? Not!

public interface ReallyCloneable extends Cloneable {
    public Object clone() _
}

Now I realize there's CloneNotSupportedException to throw. A checked exception. I have 2 options: be a good girl, adhere to best practices and clone the signature of Object#clone, or ... not. Let's explore both options.

public interface ReallyCloneable extends Cloneable {
    public Object clone() throws CloneNotSupportedException;
}

Now all my classes (let's assume they are simple enough to contain only primitives, Strings and arrays of the above) implement ReallyCloneable and add those 3 lines:

public Object clone() throws CloneNotSupportedException {
    return super.clone();
}

But whoever clones my objects also needs to add:

try {
    //call clone
} catch (CloneNotSupportedException e)  {
    //handle it... how???
}

Because implementing Cloneable and all the rest means that JVM will not necessarity throw CloneNotSupportedException, but as far as Javac is concerned, it still may. So no matter what, the caller needs to prepare himself for the worst. Then why bother with interfaces and all that, I ask? People say Java is verbose. I don't mind verbose, but bloated with boiler-plate to this extent?!

The other option is not to declare the exception in ReallyCloneable. This is possible because overriding rules do not require to throw the exceptions of the overridden method.

public Object clone() {
  try {
    return super.clone();
  } catch (CloneNotSupportedException e) {
    //hm...? 
  }
}

Even though JLS allows it, my IDE and static analysis tools will complain. So I have to either reconfigure them or add some "escape" annotations. More typing... By now I feel like a typomaniac. And wait, I should handle the exception - I don't want to swallow it. Besides, what if someone really objects to being cloned? There is a need for a runtime exception.

public class CloneFailedException extends RuntimeException {
public CloneFailedException() {
super();
}
  public CloneFailedException(CloneNotSupportedException e) {
    super(e);
  }
}

and

public Object clone() {
  try {
    return super.clone();
  } catch (CloneNotSupportedException e) {
    throw new CloneFailedException(e); 
  }
}

Now at least the callers don't have to catch. But they will still have to cast, because there is no way to express a self type with Java's static types system. Sigh.

And all this why? and for what? Couldn't we just have Cloneable with a public clone and a RuntimeException to throw if we didn't want to be cloned? Simple as that?

This is yet another example of static type system's failed attempt to protect us from ourselves.

Brains, bucks and programming languages

2008-11-29T17:15:00.002+02:00

The title is supposed to be a paraphrase of "sex, drugs and rock'n'roll" in a geeky context.

This autumn I went to see Paul McCartney in concert - a lifetime dream come true. For most people Paul McCartney is first of all an ex-Beatle. Indeed, during the concert he played many classic Beatles tunes to please the audience. And the audience was very pleased. Then, he cashed in the multi-million-dollar cheque and went back to England to do what he really likes - which at this point seems to be composing experimental electronic music. To me it looks pretty fair.

Recently I listenned to James Gosling's keynote at the JVM Language Summit. I actually enjoyed the presentation very much. One of the things he said, was something like "My dream would be to implement Fortran over JVM ... ah, but I have a day-job". Now, not that JVM really needs a Fortran IMO. But think about it for a second. How many people in the world can design a programming language? How many of them can design a good programming language? And a popular one? Java is more popular than Beatles. Uhm, well... even if it's not, you get the idea. Now what can be more important for James Gosling to do during his day job than design a programming language of his choice, I should ask his employer? What? Throwing T-shirts at JavaOne attendants? No, really. Why is it that James Gosling can't do anything he freaking likes for the rest of his life?

I think something in our business is unfair. I am not saying Microsoft model is right, I am very pro open-source and free software and all that. But I'm confused - something about it isn't right. Large IT companies make loads of money, and waste a lot of it on complete crap - I've seen this from inside. So how come Gilad Bracha cannot find funding for Newspeak development? This is totally surreal!

There goes another angry post.

DSL - fuel for life

2008-11-29T14:25:00.015+02:00

Do you feel that your programming language is too bloated? I do, and I know I am not alone.

Let's take a look at Java. You may ask - what do you mean, when you say Java? Ah, good question. There is Java, the language, as described in the spec. Java the Standard. Don't you wonder where's the new edition of the book, BTW? Anyway, then there's mini-edition, enterprise edition, real-time Java... there's a huge stack of official/certified technologies (e.g. all the JEE stuff - is JSP or JSF Java? is JPQL?), for which there are often multiple vendors. That's not all, there are all the popular open-source frameworks that don't bother getting Sun's approval, and yet they possess lion-share of the market (e.g. Eclipse, Spring). There's no chance to even keep track of all this, forget mastering. And yet, I don't feel that I have all I need. I have all that, and yet I can't write a descent program the way I'd like to, because I am missing some core features. What? Let's see - how about proper modularity, closures, tuples, local type inference, properties...

Java made the grade in expanding layer by layer - to the extent where it reminds me the state of the Earth in Wall-E. If you haven't seen the movie, I'll just say that the Earth drowned in human-produced garbage, and the garbage made it inhabitable for anything organic. The garbage in Java makes it impossible for the core, organic features of the language to grow. In the movie, people are leaving, and robots stay back to clean up - now you make the analogy.

As a Java programmer I really identify with Wall-E. Moving tons of garbage around, day after day, in a desperate attempt to clean up the world - something obviously impossible. And man, I'd love to be that flying-iPod-looking Eve from outer space. She's so strong, so modern, so clean and shiny... And she has a mission!

If Wall-E is my Java, then my Eve is no doubt Newspeak.

So what does it mean in a wider sense of programming languages? I think good language should have a small core, as little redundancy as possible. And it should grow from then on. Here's Guy Steel at OOPSLA '98:

I think that the best way of growing is via internal domain specific languages, DSLs, - you don't change the core language, yet you cover more and more domains. Anders Hejlsberg also talked about it at the recent PDC conference - adding features as a DSL; and then, if the DSL is very successful and popular, add syntactic sugar to the core language, as they did with LINQ. I don't think that you need that latter part if your language is well suited for DSLs to begin with. So the language should have a small core and be well suited for DSLs. Java isn't DSL-friendly. Scala is much better, e.g. the Actors library. Haskell, Ruby and of course Smalltalk are really good at it.

The last part is getting rid of garbage as you grow, and again, Gilad has an idea how to do that.

I could go on forever and ever about DSLs, showing examples like parser combinators, unit-test and SQL libraries and so on. Martin Fowler is writing a book. So, instead of boring you with repeating what's been said many times, and enumerating lots of references, I will just finish with this cutest quote:

DSLs are for making languages bear-able.

For I am a bear of very little brain and long words confuse me. [Milne 1926]

The premise of this subject is that computers should adapt to the ways of people, and not the other way around.

Types and other virtues

2008-10-23T01:42:00.004+02:00

Here's a thought. Why do Java Generics attract so much bad vibes? If we're talking about types again, gotta visit our old friends - ML, Haskell. They are doing fine, they've got it all figured out. Hindley-Milner etc., they've proved themselves to be right.

But them and Java, it's apples and oranges. So to bring them to the same arena with object oriented languages, let's see how are they doing on the front of extensibility? Well, there are some recent advancements, polymorphic variants in OCaml, extensible records in Haskell, but it's not like the other stuff they've figured out ages ago, there's a slight hint of hesitation here.

So having types and being object oriented at the same time - maybe it's just, hm, non-trivial. If Martin Odersky says it's hard for the compiler (and he's a genius!) - then it really must be. Now look at it as a reverse Turing test - if the machine can't figure it out, how the **** are we supposed to? Which means we need an escape route. If declaring the right type is hard, we need an easy way out.

Pre-generics Java didn't bother itself too much with complex static types, the trick was to fall back on the dynamic types whenever in doubt: arrays - ArrayStoreException, collections - ClassCastException. Some say it's broken. It's a hole in a fence, but it's a way out . "Oops"

Then Generics came and fixed the hole in the fence. But, without realizing, they violated the status-quo, the eco-system, they stepped on a butterfly. Suddenly, the brain hurts, and the only way out is dumping all the angle brackets and jumping over the fence. It's a "No Exit"!

What does Scala do? Ah! First of all Scala has a much more advanced type system to start with. But the real trick is implicits. If it's too hard to declare the right type, just imperatively describe the conversion. Done deal, here's your way out. It's pretty cool, but implicits have their share of bad vibes too. Can't live with them, can't live without them, I guess.

So how about implicits for Java 7? No, actually how about Scala for Java 7? It's been done before, you know...

The Great Divide

2008-10-05T23:53:00.007+02:00

What hasn't been said about static vs. dynamic types in programming languages? Read on at your own risk, because here I go again...

When you think of static types what comes to mind? Haskell? OCaml? Scala? Dear friend, you are better than most of us, but you have clicked the wrong URL. Peace. See you in another post...

Did you say Java? Still with me? Good. Listen, now when the others have gone, just between you and me, the guys from the previous paragraph - they're on to some good stuff. Check it out, you won't regret it. But don't quit your day job, not just yet. It's a bit complicated, but did you ever witness extreme programming methodology implemented in a big corporation? No? Then picture this: Elbonians take over Dilbert's firm and make everybody do XP. They even send pointy-haired boss to a Certified Scrum Master course. Get the outcome? It can only end like the implementation of Carl Marx's ideas in Russian countryside. I am trying to say - there are ideals, and there is reality. In reality, Haskell programs have bugs too.

So Java, you say. How do you feel about dynamic types? Cool? Get out of here. No really, it's no fun preaching to the converted. See you!

Oh no, heaven forbid, you won't touch them with a stick. You're my guy then. So let's rewind to Java 1.4 days, after all many Java developers still use 1.4 and many others look back at it with nostalgia. Are you one of them? Ok. So what about pre-generics collections, do you think they are statically typed? Hmmm... And what percentage of your code involves collections? So this code was not entirely statically checked. Now add all the reflection stuff...

But then of course came Generics. And suddenly Java is much more complex. How do I make my code compile, gee, wildcards, captures... ?! I am trying to get something done, hello!... It's easy of course to blame Generics implementation, but if we learn something from the folks whom I kindly asked to leave in the beginning, they'll tell you that finding correct static type for every element in your program is hard. They of course think that hard is good, they are noble men with ideals, they like overcoming challenges. But you and I, we're just trying to make a living. So we curse Sun and back off to an untyped collection. Hm, maybe we're just doing the right thing? Maybe sometimes we just know that our program is correct, but the compiler demands more and more typing and wastes our time?

Java 5 was all about improving type-checking. If pre-defined types were not enough, annotations came handy. Define your own and test it at compile-time or at run-time. Did it ever happen to you that there were so many annotations, that you couldn't see the code?

See, more types is not always a good thing. Unless you're very keen on intellectual challenges. James Gosling said this about Scala - functional programs will make your brain hurt, they are for calculus lovers. He's right. It's the kind of pain you feel in your muscles when you start working out, you know, that indicates they are still alive... So working out is good, but we can't afford doing it all day, ha?

Maybe the appeal of plain old Java was that it's a combination of static and dynamic checks? So it's not that all dynamic is evil, maybe it's a matter of how much and where?

Give dynamic types a break. Who knows, you may find eventually that they're good for some things.

Peace.

P.S. I've done some role playing here, just for the record. I do love Generics, even though they were hard to master. Annotations are overall very useful. Right now I don't do as much Java as I used to, and I do other fascinating languages (static and dynamic) as I, for long time, wanted to.

Two Days In A Life

2008-10-01T01:43:00.008+03:00

The ban that Israel's government has put on The Beatles performance in the 60s disappointed not only Israelis, but many Jewish people who were part of The Beatles phenomena, like their legendary manager Brian Epstein. Over four decades later, Paul McCartney represented the Fab Four on stage in Park HaYarkon, Tel Aviv, and rocked the audience with a great show.

Negotiations for the concert have been tough, but after a series of rumors and denials the deal was finally set. Ticket offices were stormed on opening, 50,000 tickets were sold eventually, filling the park with fans and music lovers.

Read more in this illustrated account of Paul McCartney's visit to the Holy Land...

Beware of the subs

2008-09-30T23:21:00.006+03:00

No, I don't mean these cute yellow things beneath the waves. I am going to talk about List#subList and String#substring methods in Java.

Apparently many people are unaware of what exactly these methods do. Unfortunately their ignorance may lead to unpleasant consequences. So getting straight to the point: both methods do not copy a portion of the original data, instead they create a view, or, in other words, a proxy to it. The important thing is that the new wrapper object holds a strong reference to the original object.

The Javadoc of subList at least admits that it's a view, as for substring, the only way to find out is by looking at the source - the method redirects to this constructor:

//Package private constructor which shares value array for speed.
String(int offset, int count, char value[]) {
  this.value = value;
  this.offset = offset;
  this.count = count;
}

So what's the problem? Let's look at the following snippet from a real code-base:

List leaky = ...; //long list of big & hairy objects
leaky = leaky.subList(from, to);

Assuming that leaky wasn't referenced anywhere else in the code, there is no way for the programmer to access the elements that lie beyond the (from,to) range. But these bytes aint going to rehab, no, no, no - as far as JVM is concerned they are still strongly referenced. So if you really mean to extract a portion of a list (or string), and throw the rest away - copy it manually to a new list (or string). For example:

List sneaky = ...;
sneaky = new ArrayList(sneaky.subList(from, to));

Is there any better way a "sub" could be implemented? Well, maybe the reference to the original data could be kept weak, and only when (if) the original object is enqueued for garbage collection then the data could be copied into the "view". This would require backwards references from "original" object to "views", which would also need to be weak, so... overall this doesn't seem worth the effort, and hence avoiding leaky lists and strings shall remain the responsibility of the programmer.

Speaking of subList, another nasty thing about it is that sub-list is not Serializable, nor Cloneable or anything like that, even if the original list was. And speaking of leaky things that are caused by undercover strong references - never forget non-static inner classes that refer to their enclosing instance.

Take care, and keep your head above the water :-)

Software: Live and Let Die

2008-07-04T23:27:00.016+03:00

This week I was lucky to attend Gilad Bracha's guest lecture on Networked Serviced Programming at the Hebrew University. He has been talking about Service Objects for some time now, but nothing compares to hearing it live - Gilad's presentation was witty and fun!

So here is my interpretation and some take-aways.

What's the problem with software? It is too damn complex. Projects crumble under their own weight. It happens to successful projects - our dearly loved Java, for example. Also look at Vista, if you dare. And it's certainly true for the monster-size projects I used to work on. Once upon a time our team took the corporate "quality improvement" policy seriously and decided to investigate what causes bugs in our multi-million lines of code project. We collected all sorts of statistics and ran all possible metrics (which was tricky 'cause some of the tools would choke on such a huge code-base) but long story short our finding was this: the only metric that correlated clearly with defectiveness was LoC. It is hardly news, but a cure to the disease has yet to be found.

There are certainly several things to be done, but what this talk focused on is getting rid of code which shouldn't be there, or in other words - dumping unused code and backwards compatibility. The way we work today - we are bound to not just specs and APIs, but to all the accidental behaviors and bugs in the previous version of our code. It seems that Gilad views code as if it was a live organism. Staying alive means being connected (and network plays a central role in his vision), but we should make way for evolution and some code should die - Gilad calls it "bit rot".

So how do we turn software into a healthy living organism? According to Gilad, there are several things to be done on the technical front first.

Take advantage of the network: maintain a bi-directional connection with the control center - let programs pull upgrades from the net, but also send back operation statistics. This means that programming platform has to be aware of the network, and aware of the fallacies of distributed computing. This is cloud computing utopia: Internet as a platform, browser as an OS, and Javascript as the low-level programming language into which other languages can compile (in a GWT kind a way).
Modularity: it should be possible to extend and replace individual objects without interference to the whole organism. Gilad has a well developed theory about how modularity should be done in a programming language, based on principles of object-orientation using mix-ins, nested classes and inheritance hierarchies; his new language, Newspeak, is going to implement it.
Explicit Dependencies between modules - no static, no imports, modules are truely independent and dependency management (wiring) is performed by passing other module instances as parameters to module constructors. This allows to maintain clear boundaries between modules and flexibility in module composition.
Frequent Updates: to allow the "clients" of the object APIs to deal with changes, in addition to maintaining modularity, the changes should be made small and frequent. Call it agility, if you like. That means that we can't afford reboots, and we need to find out when the system is quiescent so that upgrade can be performed, which brings us to the next point...
Reflection and Hot-swapping: objects should allow other objects to find out both static and run-time information about them without breaking the encapsulation. Objects should also allow other objects to modify them "live". Gilad and his team at Cadence are building the support for these features in Newspeak using Mirrors, a concept that originates in Self programming language. It's worth noting that there exist dynamically typed languages that implement hot-swapping today - Erlang being one of them.
Security is important in any distributed system, and even more so if we allow remote objects to mess with the program. So in addition to dynamic typing and pointer safety, Gilad proposes mirrors to be guarded by capability-based security, similar to the one in E programming language.
Synchronization: many programs need to work with persistent data and it is important to keep the program and the data in-sync. Gilad proposes orthogonal synchronization, based on Smalltalk orthogonal persistence idea, where objects are split into transient and persistent ones by marking object tree roots accordingly. Persistent objects are upgraded whenever the corresponding part of the program is upgraded, and transient objects are lazily recomputed. If the data is ever to outlive the service, it would be exported into some generic format, such as XML.
The most extreme and bold part of this vision is probably No Versions and No Releases - there would be only one version for every program out there. Gilad sees software becoming more of a service than a product, but in order for this to realize we'll have to overcome not just technological, but also psychological and economical barriers - we will have to change the way we develop software and the way we make money of it.

Bottom line, all this may sound too futuristic, but "software as a service" and "platform as a service" are making their way in the industry and this wave, if strong and successful enough, may bring the significant change Gilad is predicting.

P.S. As for the rest of us, living in the JVM world, some of the ideas ring a bell. We hope that modularity JSRs and OSGi will improve Java. Those of us who survived Ant and Maven, and felt the weight of a DI framework, will probably appreciate the amount attention Gilad is putting into software composition. It's worth noting the attempts to address hot-swapping on JVM, such as JavaRebel and Jonas Boner's experiments with Scala Actors and Terracotta. Terracotta server also utilizes some ideas which (in my mind at least) look quite similar to the orthogonal synchronization scheme.

JunkedIn

2008-06-25T15:25:00.002+03:00

Indeed LinkedIn runs on Java, no doubt about it:

Noble cause

2008-06-17T14:37:00.004+03:00

Duchess is an on-line community of female Java developers with members from all over the world, but currently active mainly in the Netherlands. I figured they could use some promotion, besides it's a perfect excuse to put this cute mascot on my blog:

Typesy Turvy

2008-06-05T23:30:00.007+03:00

Lambda The Ultimate feed just notified me that types are considered harmful. Ah yes, I heard this before, so what's the news? The news is that it's not Stevey vs. Cedric or anything like that, this is Benjamin C. Pierce in his own write. Benjamin C. Pierce, from "Types and Programming Languages" and "Advanced Types and Programming Languages", King of ML, Lord of the Functional Programming Commonwealth, Defender of Type Systems Faith!

Nice presentation, BTW. I think it's one of these situations when a big shot computer scientist is confronted with a real life problem. So you say the language is perfect for writing a compiler - good for you. How about a database application with a web front-end?

(click to see full-size)

Mr. Language Designer, where are you in this picture? Yes you, who designed the mousetrap which "the bug" has safely escaped from. Now it is here, so are you at least by the developer side, handing him something heavy to throw at "the bug", or are you on the bed with the rest of the crowd going "ah ah ah, what do they teach computer science graduates these days..."?

One thing that impressed me during otherwise boring (let me just read you aloud the tutorial) JRuby on Rails preso I recently attended - here is a system that tries to serve the needs of the developer. Not server vendor, not language designer, not JSR politician, not some brandthirsty marketing person or buzz-oriented architect, THE DEVELOPER. I am not used to that. So all I have to say - programmers of the world, unite! Stand for your rights! We deserve better tools, because we are the ones getting the job done.

Thank you for reading.

In whining there is truth?

2008-05-15T19:17:00.004+03:00

What do Java developers want? Hard to say. But here is what Java bloggers don't want Sun to do:
No evolving Java syntax - no properties, no closures, etc. - fix what is already there first.
No extending Java capabilities via annotations.
No investing in a new JVM language (JavaFX, or JRuby in the past)
Ok, maybe add Groovy, but don't change JVM spec.
Hey, wait, why are key people leaving Sun? We didn't want that either!

Girl Power

2008-05-13T15:20:00.016+03:00

I have been reading recently that women are abandoning computer science, and that percentage of women in our profession is not just low, but getting lower. I'm not sure actually that the situation here is as bad as in North America, but it's certainly true that software engineering and computer science are not very popular among women. Why? To be honest, I don't have the answer.

Here is what comes to mind:

Geek-ness is viewed in the society as the opposite of being attractive; this is much more important for young women, than for men. For a guy - a nice high-tech salary will provide the attraction instead.
Sitting in a cube by the computer all day and doing one thing, coding, is probably not very attractive for most people, but for women especially, since most of us are better at performing a variety of tasks and interacting with people - I mean there has to be an explanation why "secretary" or "teacher" are such typical women professions.
Hi-tech jobs are very demanding, too demanding. For most women family life is at least as important as professional life, and usually family comes first. But there just aren't many positions you can find after getting computer science degree that allow you easily balance work and family life.
Hi-tech is for young people - look around, how many programmers you know are over 45? Why? For the same reason you don't see many women - it's hard to compete with the smart kids when you are pregnant, or haven't slept for a week, or worried about some family matter.
But can't we just switch roles with the husband? well, go back to first bullet, double standards of the society certainly don't make it easier on us or our partners.
People we are surrounded with (nerdy young men mostly) are pretty anti-social creatures in the first place, even more so with species of the other sex, even more so with the ones that don't fit social stereotype.
Now suppose you survived all the obstacles, because you passionately love science and engineering. Did you watch the knack? There's a grain of truth there... society became less tolerant of weirdness, and people who 100 years ago may have been referred to as crazy geniuses nowadays live "normal life" on prescribed medications. And for a girl it's even stranger to be a crazy genius then for a boy.
Male domination in the field - yes, it's chicken and egg problem. The field will change only if there will be enough women in it to drive the change from within and help other women. So I will devote the rest of the post to the ones who made it.

Lady Ada (Byron) Lovelace

She was more of a technical writer really, but she was a visionary.

Admiral Grace Hopper

The inventor of COBOL and debugging.

Professor Barbara Liskov

The one from the substitution principle. I chose her among several prominent computer scientist women because she was the first female computer science Ph.D in the US.

SVP Jayshree Ullal

I originally thought to put the Google princess here, but having personally met Jayshree (she was my manager's manager's manager at one point) and impressed by her personality and professionalism (she is so clever, and yet such nice and humble person), I decided she's a better candidate to represent successful women in computer industry.

Alice, Dilbert's workaholic colleague

What did she achieve exactly? Surviving in the office should not be underestimated! So I am going to honor another ex-colleague, although we never met, for providing inspiration for a character I can identify with :-)

Keep coding girls!

Net, Goddess of Wisdom

2008-05-13T11:56:00.006+03:00

No, it's not a joke or some kind of wordplay.

Net (also Neith) was an ancient Egyptian goddess, not very well known perhaps, unlike her Hellenistic reincarnation - Athena. Net was a warrior goddess, just like Athena, and the goddess of weaving - "weaver" is actually the translation of her name.

Athena was a weaver too, according to the Arachne myth, and of course the goddess of wisdom.

So why weaving, or net, and wisdom? Well, for several reasons. What our ancient ancestors have probably recognized is that wisdom is acquired via communication. Today we know in addition that human thinking is powered by neural net in our brains. Net, yes net. We even use the same word. Internet, collective intelligence... "there's nothing new under the sun" said another wise ancient man.

Synthetic bridge method annotations

2008-04-30T16:29:00.003+03:00

Update regarding "walls and bridges": Sun has kindly opened an enhancement request on my behalf for the issue. Please vote if you are affected by it too.

Java properties links

2008-04-28T10:50:00.003+03:00

Since I am trying to keep up to date with what's going on in "Java Properties" land, I started with Alex Miller's collection and have been adding things that I find either relevant or influential. Another idea I nicked from Alex was to put them on Tumblog.

A disclaimer: this is a personal collection, so if something looks irrelevant, it means that I think it is influential :-)

I'll be more than glad to hear suggestions for more links or feeds.

My OO My

2008-04-27T22:19:00.003+03:00

No, I don't mean the song, but it's to do with Scandinavia. Mads Torgersen mentioned this during interview with Joe Armstrong (highly recommended, there is also a part II), and later I saw another reference in a paper: apparently there is "Scandinavian school of object orientation", sometimes compared to "American school".

Here are some insights on the difference between the two. The point is that the former is about concepts and philosophy, and the latter is about pragmatic aspects like software reuse and organization. I am not gonna pretend that I'm smart - all I know about it comes from searching Google, so you can go there and find out for yourself.

* On a side note, since I am linking to MSDN here, I want to say that even though my last post wasn't very favorable to Microsoft, to be fair - the company plays an important role in renaissance and democratization of functional programming (LINQ, F#, etc.), with guys like Mads Torgersen and Erik Meijer largely responsible.

Open the source, sesame

2008-04-27T16:56:00.010+03:00

About a month ago a big shot from Microsoft came to speak at the university - looks like he's on some kind of a tour, and his mission is to convince universities to teach Windows operating system aside Unix/Linux. He waved at the audience with a free license and tried very hard to prove that some aspects of Windows design are better than Unix, mainly because it is "newer" design and better suited for nowadays computers.

Long story short, one of the things he said, was that open source is bad, because engineer who looks at the source of a library will design his application assuming particular implementation, which is now tightly coupled to library internals - bad. Obviously a discussion erupted - design by contract and Eiffel were thrown in the air, relation between programming language and operating systems..., but frankly all I could think of is my distant past as a VB developer and those 1000 pages "Windows *** Unleashed" books. They were written by hackers who used trial-and-error against Windows DLLs trying to make some sense of the APIs - the formal documentation was either sloppy or intentionally incorrect to misguide us, Windows application creators outside Microsoft, since we were potential competitors. Things were so different when I moved to Java. (Though my first Swing experience made me - believe it or not - miss VB, but that's another story.)

Anyway, statement like this coming out of the mouth of a Microsoft employee, even if he has an impressive Unix record, was easy for me to dismiss. But here is a respected Smalltalker saying

"I think people should have the source ...not to get miss quoted saying I'm against open source, but I think it's important class libraries should be viewed like caves."

(Ah, wait, he is from IBM - he can be ignored too... just kidding)

I mean, seriously look at this puzzler for example - it's pretty cool, but the solution heavily relies on a particular implementation, I would even say this is actually a hack. And sometimes a hack is necessary... or is it? Is there or will there ever be a perfect environment where we can program without hacking?

P.S. The reference to Ali Baba tale in the title is intentional. Open source is good, very good, it's a treasure and in the world dominated by large corporations - almost a miracle. But IMO we should resist the temptation to exploit it to the last bit, because we might find ourselves locked in the cave.

Static methods as function objects

2008-04-27T12:34:00.009+03:00

In one of the earlier posts I described a policy object created by "functional style programming" in Java. The basic idea is to make methods into Function and Predicate objects and then composite the logic from these building blocks. The problem though was that the policy implementation looked alien to Java. To re-cap: I wanted to invoke a different function based on the class of the object - classical multi-dispatch with a single argument.

So inspired by extension methods idea I added a new mechanism to define properties (and functions in general) - via static methods. You define a bunch of static methods with the same name (or appropriately annotated) like this:

static int size(Collection c) {
 return c.size();
}
static int size(Map map) {
 return map.size();
}
static int size(Object obj) {
 return (obj.getClass().isArray() ?
   Array.getLength(obj) : 1);
}

Now assuming I have a property declaration

Property<Object,Integer> size =
 new Property<Object,Integer>(0) {};

The underlying implementation of the size.of(object) would be a "policy" (composite function) dispatching to one of the static methods according to run-time type of the object. So that size.of("foo") is 1, size.of(Arrays.asList("a","b","c")) is 3 and size.of(null) is 0.

There are couple of issues though.

Issue #1: automatically ordering the rules. Now when rules are not added manually, I should be careful to test for more specific class first. The solution is to calculate a distance between two classes and always look for a best match - basically same algorithm javac uses when choosing methods at compile-time. This isn't fully unambiguous though: if class C implements 2 interfaces A and B, having different implementations of size(A) and size(B) would put me into a dilemma which one to choose for size(C). Javac in such a case reports an error - thanks to Java being statically typed. But once I make it a run-time decision the only logical solution is to report an error at run-time. To make the problem less acute, I could add an annotation that controls ordering of the methods.

Issue #2: this is less generic than a general predicate-based policy - it is limited to "instanceof" predicate. One of the policies I use internally in the properties project is reflectively invoke getFoo() if the class has a "getFoo" method. This isn't possible with static methods.

Issue #3: reflection. It's not really a problem, but rather an enhancement I want to make: instead of using reflection I could generate the functions at compile-time with APT, like Bruce Chapman does here. This is one of my next adventures, which I will hopefully describe in a future post.

Digg a pony

2008-04-26T15:43:00.003+03:00

Here is another post with no helpful information, sorry, I meant to write a meaningful piece but I am on vacation and in fun spirits :-)

Anyway, I just got reminded of something that crossed my mind before and is now more relevant than ever - the term John Lennon invented almost 40 years ago: "bagism". It's amazing that at the time everybody laughed - a person giving an interview in a bag. You can look this up, or even better watch the video (fast-forward about 2 and a half minutes). Here's a short re-cap:

"if everybody had to go in a bag for a job (interview) there would be no prejudice, you see, you'd have to judge people on their quality within, you know, we call it total communication..."

I think this is really the web: blogging, mash-ups, syndications, and social networks, and why all this is a lot of fun - you really know nothing about the one you are talking to, except what he (she? it?) tells you.

Dig a pony lyrics mention

"penetrate any place you go" (...internet?...)
"radiate everything you are" (...wireless?...)
"imitate everyone you know" (...blogging? syndication?...)
"indicate everything you see" (...search? google?...)
"syndicate any boat you row" (...social networks?...)

Cheers to total communication!

P.S. A tribute to Passover holiday (the reason for my vacation): there were once four boys (ארבעה בנים) - the wise and wicked one, the cute one, the quite one, and the one that can't sing... and they were fab.

Java and closures

2008-04-24T00:37:00.004+03:00

My reaction to the likely postponing of closures till Java 8.

Mercurial adventure on Widows

2008-02-06T22:15:00.000+02:00

With a friend at work we decided to check out Mercurial as source control system. The repository was set up on a Linux box and zillion lines of code and libraries were sucked in from ClearCase, now I just had to clone it to my lap-top running Windows XP.

Installed Mercurial 0.9.5 via TortoiseHg (I am familiar with Tortoise with SVN so it was a natural choice) - very easy and straightforward so far. Issued clone command - hm, filename too long. Switching to cygwin - doesn't help.

So Ok, we have pretty deep directories structure and Mercurial makes it worse because it translates everything to lower case by "escaping" capital letters - adding underscore before the letter, so "A" becomes "_a" and a single underscore becomes a double one. Why? This is to avoid problems when working with case-insensitive operating systems like Windows. Fair enough, but what do I do with these long filenames now... Python bug? Oh, great. In the meantime the error became "file not found", but I (correctly) suspected the cause is the same.

I found this bug report (and this one) - at least it happenned to other people, it says I can use a 'usestore' workaround. Cool, found this - says change one of the scripts, but hey, they are compiled?! Ok, downloaded ActivePython (I admit, I never touched Python in my life) also downloaded the sources. But apparently I need a C compiler too for the full build from source - that's too much, I decided to just run Mercurial with the interpreter.

Found the right script, changed it. Boom, of course extra space in indentation - until now I only heard jokes about this (besides similar experience with Fortran half a life ago). Nevermind, got it now. Still not working. I take another look at the script - the if statement I just added was already there before! Oops, what's going on? Trying to understand the Python code - it is looking for some 'usestore' configuration option, which BTW is supposed to disable the "escaping". All I need now is to set it. Hm, how? Eventually I find out that I need to edit <TortoiseHg-install-dir>/Mercurial.ini adding

[format]
usestore=False

(If you use plain Mercurial, not Tortoise, create Mercurial.ini under <mercurial-install-dir>)

I do it and... voilà, the repository cloned!!! So bottom line - all I needed was ini file change. It is somewhat dangerous 'coz I still have the mixed case, but at least I have the code on my laptop. Why didn't I find it in an FAQ or tips'n'tricks or something? No idea. Hope this post helps the next person who googles for it.

Blonde Moment

2008-01-25T09:07:00.000+02:00

Kirsten Dunst wearing Red and Sensual Java.

The taste of Java Generics

2008-01-22T16:17:00.000+02:00

As the closure debate flames are rising, people are attacking Generics in order to attack closures. Some use it as a negative example against particular closure proposal, some try to discourage Java language changes in general and some to promote other languages. When I read articles that start with "Generics are hard" and lead into "Generics are bad", I can't help thinking of the fox and the grapes. Yeah, they are hard. So is concurrency, NIO and many other things. So? That's why people create frameworks on top of Java.

Don't get me wrong - Java Generics have faults, and they are long known. Here's a hit-parade of my biggest complaints, but I will also mention some solutions I have used:
1. Erasure
The way to work around it is by using type token or super-type token. As any workaround, it is limited, but it can solve part of the problem - Class#cast and Class#newInstance don't issue the annoying warnings.
2. Generic parameters not covariant
It is usually a good idea to use bound wildcards extensively.
3. No lower bounds support (except wildcards)
We can use static methods instead (ala extensions) with <T extends Super> rather than <S super Sub>
4. Verbosity, instead of more aggressive type inference
Not much to do, hm, learn to type faster? :-)
5. The angle brackets
Yeah, I hate them most when typing on blogger, luckily I am old enough to have been coding HTML in plain text editor a long time ago, so... back to GT and LT escapes.

BTW Java didn't die, it has simply grown beyond its hipness age. And I am glad that it spawned even better things, such as Scala. But there is a difference between "good" and "popular". Java is popular and not that bad, other languages may be better, but not as popular. Will Scala be willing to sacrifice some of its expressive power and shortcuts (gee, stuff like that reminds me why I don't code in Perl) to become simpler to learn/read and potentially more popular? Should it? (Ruby consciously didn't and ... didn't.) We shall see.

Listen To What The Man Said

2008-01-10T12:55:00.000+02:00

Just couple of nice quotes for today: Alan Kay on Innovation and the not so new but still relevant Edsger W. Dijkstra transcriptions - look at this Q&A for a quick feel.

"Machine capacities now give us room galore for making a mess of it. Opportunities unlimited for fouling things up! Developing the austere intellectual discipline of keeping things sufficiently simple is in this environment a formidable challenge, both technically and educationally."

Wildcards observation

2008-01-01T13:09:00.000+02:00

"A bug in poker is a limited form of wild card."
- Wikipedia

While looking for academic references for my properties project, I bumped into the expression problem - the write-up by Phil Wadler and follow-ups by Mads Torgersen and Matthias Zenger/Martin Odersky are really fun and educative readings.

This stuff caused me to re-read the the wildcards paper and I suddenly understood why I had this strange gut feeling about wildcard usage. Almost all learning material about Generics, including the wildcard paper itself, emphasizes the distinction between read and write methods - having a wildcard in a collection parameter prevents you from adding elements to the collection, but allows reading. This is true, but somewhat misleading. Actually, wildcard in the parameter allows us to call a method that returns the thing behind the wildcard and treat it as if the returned object is of the type of wildcard bound (or an Object, if the wildcard is unbound); the wildcard prevents us, however, from calling any method that takes "the thing behind the wildcard" as a parameter. Now obviously, to add something to a collection you need to call a method that takes that something as parameter, and it is indeed prevented, but other modifications are not disallowed, e.g. clear() .

The consequence of wildcard usage applies to any method, not just "write method", for example - containment test. So looking at Kevin's example, it now seems to me that doSomeReading method shouldn't have used a wildcard if it wished to perform a safe containment test - that's the catch (or should I say, hm, capture...)

Happy New Year!

Past forward

2007-12-31T16:18:00.000+02:00

Since my previous post was devoted to Java history and this interview with Martin Odersky recently came out, I decided to do one more short transcription related to Generics history.

"I am an academic, actually, I'm working at the university in Switzerland. ... A long time ago I was a Ph.D. student of Nicholas Wirth, so I am in the Modula 2 operand camp... But then I drifted more and more into functional programming community, I have a good friend named Phil Wadler who is very active in the community, so he sometimes told me ... "there is this new thing, it's gonna bury functional programming, it is called Java". I wanted to find out more and he said it has garbage collection, and it runs everywhere on the Web ..., so what are you gonna do against that?!

So ... we said we'll try to do a functional language on the Java platform, because the Java platform looked very exciting to us then (it was in 1995, so a long time ago). ... To find out more about what it is (I always learn best when I write a compiler) let's write a Java compiler. We had this second ever Java compiler out there after the old JavaC.

Then we said - if we want to do a language it should be Java compatible, and we wrote a language called Pizza, which added stuff from functional programming to Java. We experimented a little bit with that and then Sun came and said - we like what you did, in particular the Generics, we wanna do that. Then we had a project with Gilad Bracha at Sun and we did this GJ thing, which was the Generics thing.

The compiler for GJ eventually became the current JavaC compiler, Sun took that over because they decided that this compiler was more maintainable than the original. (Interviewer: "from JavaC 5 on...") From Java 1.3 already.

So the compiler was in 1.3, but the Generics were disabled. In fact, the Generics were at first not that much disabled, there was a secret switch that people found out about, so there was this stealthy movement of people using Generics already in 1998 when it first came out. And then Sun pulled the plug, they said - you can't do that, so you have to re-write your compiler so that it doesn't make any Generics available. So there was a special mutilation script in the compiler which would rip it off, so that people couldn't use it for a while, until it came back in Java 5."

Back to the future

2007-12-13T11:51:00.000+02:00

If you look at the programming language history poster there's a bunch or arrows leading to Java genesis, one of which comes from Smalltalk. Ok, I heard about Smalltalk and saw some code in books. I also knew that Gilad Bracha was one of the authors of Strongtalk (Smalltalk with static types) and later immensely contributed to Java evolution while working at Sun. I also saw references to this supposedly great language called Self in some programming language articles. But only now I see how it all linked together thanks to Avi Bryant's Keynote at RailsConf (yeah, of all places to learn about Java...) so I decided to transcribe here a small portion of this fascinating talk.

"There's a legend that there's something inherently about Ruby that makes it hard to implement a fast VM for it, because of open classes, because of dynamic typing - there's something about meta-programming, something about Ruby that makes it hard in this day to make it run fast. And it's just not true! It was true 20 years ago - it was a hard problem, but people solved it.

This is actually from Sun site, this is a bunch of papers that collectively explain how do you make Ruby go fast. But you'll note that they're written in 1989-91, you can go and look at the URL here, they come from the Self project. And the history is kind of interesting here, so since this is about the future and about the past, I'll give you a really brief history.

Self was a project at Sun to do Smalltalk, but even more so. Even purer object-oriented. And it meant that they had to find ways, because they wanted to make it so object-oriented, so pure, so turtles all the way down, they had to find more and more ways to make it go fast. And the technology they came up with was so interesting that they actually spawn off a start-up to try to implement new super-fast Smalltalk that was gonna be used on Wall Street. But before they got to release it, this was gonna be called Strongtalk, before they got to release it, Sun bought them back.

And Sun used the technology, used the HotSpot profiling and Just-In-Time compilation technology that have been developed here to build the Java VM. Which is sort of one of the great tragedies of technology, but it's true, that Java HotSpot VM in fact is what came of all of this great work of making dynamic languages go fast - of course they crippled it so that it doesn't make dynamic languages go fast anymore, but maybe that'll change. One interesting footnote to this is that a lot of the engineers that were involved in that have since left Sun and are back to doing Smalltalk."

Extension methods proposal

2007-12-09T22:11:00.000+02:00

I'd like to talk here about another Java 7 feature proposal lobbied by Google that generated lots of waves in the blog-sphere after being revealed by Neal Gafter. A lot have been said (Peter, Alex, Stephen, Rémi and Stefan) so I'll add just 2 humble observations.

Adding capabilities to the language vs. maintaining a safe and clear programming environment is a tough balance to keep. I had my doubts in this particular case, and I think I tend to favor it, with one modification. I think it would be better if extension method import was different from normal static import, so the developer knows exactly what he is doing. I propose following syntax (that BTW does not require any additional keywords in Java)

import static java.util.Collections.sort
  extends java.util.List;

Compiler would then check that the first argument of sort is a List (or its super-class or super-interface) and compile

List list = ...;
list.sort();

into

Collections.sort(list);

The main advantage of this proposal is that extension methods are imported consciously, rather than implicitly.

The "importer" is also given some control over the overloading rules for the extension method - what if Collection has a sort method? what if ArrayList does? well, with the syntax I propose here the developer targets the method to a specific class, so overloading rules can be applied just as if the method was in that class, in this example it would overload Collection one, but not ArrayList one. It reminds me of the Ruby feature that Neal Ford had mentioned - calling a method, and if it does not exist, executing some code piece instead. So with proper overloading rules, could imported extension method be Java alternative for method missing? At last, there's this dark corner question "What if List had a sort method?" Personally, I think that if the class explicitly imports an extension method, it should be treated as local definition, while the method defined in List as inherited definition, and therefore (following the principles laid down by Gilad Bracha in one of his recent papers) the extension method should override the one in List. This is somewhat similar to the infamous "Mother May I" rule, but I better leave this for greater minds to solve.

Another interesting aspect of extension methods in general is their behavior in case the first parameter of the method is generic. Take Collections.max method for example. It accepts collections which element type extends <T extends Object & Comparable<? super T>>. AFAIK there is no definition of extension method behavior with generics. Maybe they are not allowed, but if they were, I would expect that list.max() compiles only if elements of my list are compatible with the static method definition, which in this particular case means that they should be Comparable to each other. Now this is an enhancement of Java generics system - being able to add methods to classes based on their generic parameters! Wow, it's somewhat spooky, but really powerful, I can imagine a few neat things I could do with this :-) and since I am now going to spend some time dreaming, I am going to leave you to think about this...

P.S. (12/12/07) Just came across a vivid discussion of similar feature in .NET. First reaction - "ah, oh, I get it now, it's another one of those .NET has it so we gotta have it too features!". But seriously now - let's learn from their experience, the conclusions are pretty similar to what is discussed here.

Faster than C

2007-12-05T12:47:00.000+02:00

Not everyday you come across an example of Java program that executes faster than its C equivalent. Although the original code is in Scala, the speed is because of the JVM and HotSpot, and not because of the language which the program was written in. This is a good illustration of JVM strength which IMO can and should be reused for other languages. (To be completely fair, gcc is not the strongest competitor among C compilers, there are optimizers that do the sort of trick that JVM did. Nevertheless, it's impressive.)

Speaking of Java versus C/C++, there's an interesting debate on LtU regarding performance of GC vs. explicit memory management. The conclusion is not very surprising - there is a cost, but it is reasonable to pay given the other benefits of GC. It reminds me of a discussion I had with a guy who used to work with Java RTS (Real-Time Java System). I admit, when he first mentioned it, I didn't know what it was and it surely sounded weird, so I went and read a few web-pages. Java with explicit memory management.... hm? Poor Java, why would you do such a thing to it? So I asked him "Why don't you just use C or C++?" The answer was - "It's proven that programmers prefer Java and they are more productive writing Java code than C++". "All well, but shouldn't it be at least partially attributed to the fact that in normal Java we (programmers) don't need to manage the memory?" "I see your point" he smiled. That said, I admit that I know practically nothing about JME platform and how it solves real-time problems with Java. So I'm not shutting the door here - maybe I'm wrong.

But I just can't help and generalize this problem, looking at where things go with JSE. I mean it's great that Java evolves, but as wonderful as Java is, it's not this one universal tool that can solve every problem in the world. If there is a need for Java-like language for real-time applications, then why can't some smart guy (at Sun or Google or wherever) create a language that does explicit memory management, but incorporates other Java advantages. Just don't call it Java, please! (Yeah, I know that marketing guys really want you to, but let's try to keep some dignity, fellow enginerds).

JRuby pod-cast

2007-11-30T10:20:00.000+02:00

I almost accidentally got to hear Neal Ford's interview about JRuby this morning.

Love it or hate it, Ruby cannot be ignored. Yet the articles and blog-posts you find on Ruby vs. Java are many times so emotional and single-minded that it became a really good joke. But not this one.

Neal starts with a bit of history, goes through Ruby comparison to Java (expressive power, meta-programming capabilities), the rising importance of JVM as a multi-language platform, comparison between JRuby and other JVM languages (Groovy, Jython, Jaskell, Scala) when to use them and how to get JRuby into your corporate development environment (the tips include - "don't say the word Ruby as long as it runs on the JVM" and "use it in test or build environment first - Ruby has great advantages there and it's easier to justify a new tool for build or test than for production") and finally about his work on first commercial product developed entirely in JRuby - the agile team management suite called Mingle.

I also liked his general comments regarding how G0F design patterns illustrate language deficiencies, his objection to XML programming and why the concept of protective programming environment allowing to scale software development through hiring zillions of bad programmers is bankrupt.

P.S. Martin Fowler, Neal's colleague at ThoughtWorks, recently blogged about JRuby too.

Reflection and auto-boxing

2007-11-27T19:26:00.000+02:00

In an earlier post I discovered that the way reflection handles synthetic bridge methods is somewhat lacking. Here is another reflection issue, this time a known problem: 6176992 - sort of paradox in a box.

Apparently Class#isAssignableFrom does not handle auto-boxing so if you have a method that returns a boolean, Boolean.class.isAssignableFrom(method.getReturnType()) will evaluate to false. I played with it a bit further, and if you have a method that takes an int, MyClass.class.getMethod(methodName, Integer.class) yields NoSuchMethodException.

I expected these API's to return a non-primitive class because Method#invoke accepts only Objects and translates them internally, and it worked long before auto-boxing feature was added to Java. However, after giving this more thought, I decided that it's unfair to complain that the API's return exact types, also a primitive type does not extend or inherit its boxed counterpart and Javadoc describes the behavior of isAssignableFrom impeccably - the method supports widening reference conversion only (chapter 5.1.5 in the 3rd edition of Java Language Specification).

Nevertheless, Java Language Specification also defines widening primitive conversion (chapter 5.1.2) and (un)boxing (in 5.1.7 and 5.1.8) and I think it is fair to ask for a utility method that tests the compatibility of 2 classes that represent primitive types and a utility for boxing and un-boxing of classes. So what I'm really asking for is a java.lang.reflect.Primitive class, that provides some static utilities just like java.lang.reflect.Array does.

Until then, I am going to create my own utility for boxing/un-boxing - a Function<Class,Class> of course :-)

Properties, static and yet dynamic

2007-11-22T20:57:00.000+02:00

I think it's time to share some more of my Java properties ideas.

Property is a super-type-token and the definition looks like this:

public static final Property<Person,String> name =
            new Property<Person,String> ("name") {};

So I've taken the property out of its usual habitat - the object it applies to, and let it be defined in any class! One prominent flexibility is the ability to add properties to an existing class by loading a new class.

How is the value associated with a particular object instance? A property should be assigned a "definition", which is in essence a Function that retrieves the value from a given object, by using reflection for example. The definition can be changed on the fly, and user of the property needs not be concerned.

That's all very nice, but reflection is not enough if we add properties on the fly. There are 2 ways of dealing with it. The more obvious one is having a Map or an array within the object where the values can be stored. This of course means that the object class is in advance "prepared" to have dynamic properties.

Another way is rather shocking at first glance... to me it was, anyway. Traditionally the object stores a property->value map. But why can't we have the property store an object->value map!? When I read about column-oriented databases, I realized it's not as crazy as it first looks. All I needed was WeakIdentityHashMap, unfortunately JDK does not have one, but google-collections do - the ReferenceMap. Also cloning and serialization need special hooks, but it's not too much hassle.

I have 4 types of properties for now, Property that only allows read access, UpdatableProperty that supports read and write, and their multi-value counterparts: IterableProperty and CollectionProperty. This list can of course be extended. Any property can have a default value, and updatable property can have a list of value constraints. I have a library of definitions - plain function, reflected, mapped, ones using internal and external map like I described above and of course more can be added.

Last but not least - all properties that apply to a certain class of objects are tracked, so there's an API which I can query for applicable properties for a given class. It would collect properties of all super-classes and super-interfaces - note that property can be defined for an interface and in an interface (being a final static).

That's it for tonight. To be continued...

P.S. JavaEdge was really great, thanks Alpha folks!

Software proletariat

2007-11-19T08:33:00.000+02:00

Don't miss out today's Dilbert on the lower depths of our profession.

Predicate + Function = Rule

2007-11-18T19:41:00.000+02:00

Another enhancement on top of google-collections that I want to share with you.

In essence, rule is condition and action. So I will use Predicate as a condition and Function as an action. The composite function then consists of list of rules, for the first predicate that evaluates to true, it will invoke the corresponding function. If none of the predicates are matched - default function will be invoked. Here is the source code, if you want more details.

For example, here is how I define a function "size" of an arbitrary object:

Function<Object, Integer> size = new Rules<Object,Integer>().
addRule(isNull(), constant(0)).
addRule(
or(
 instanceOf(Collection.class),
 instanceOf(Map.class)),
self().reflect("size").cast(Integer.class)).
addRule(
asPredicate(self().reflect("getClass").
           reflect("isArray").cast(Boolean.class)),
self().reflect(Array.class, "getLength").cast(Integer.class)).
setDefault(constant(1));

If you read my earlier post, you already know about the FunctionChain, this is where I statically import self() from. The reflect() function transfers Object to Object by invoking a parameterless method with given name on the current object, or passes current object to a single-parameter method of another object (or class for static methods). The cast() method performs a cast to given class. The instanceOf() function is a shortcut to ClassPredicate which returns true if the class it holds is assignable from a given class. I also assume statically imported or and isNull Predicates and Functions.constant. (I could replace reflect function calls with a special size and length functions, but I think it's enough to illustrate the idea as it is.)

The same segment in "normal" coding would look something like that:

if (object == null) {
 return 0;
} else if (object instanceof Collection) {
 return ((Collection)object).size(); {
} else if (object instanceof Map) {
 return ((Map)object).size(); {
} else if (object.getClass().isArray()) {
 return Array.getLength(object);
} else {
 return 1;
}

I don't pretend one is better than the other, but it's interesting enough, I think, and sometimes useful.

Modularity - outside in

2007-11-18T19:24:00.000+02:00

It is probably a pretty obvious thing, and yet some developers and software architects manage to overlook this.

When refactoring a project under the flag of "better modularity" one needs to start with northbound, southbound and other external interfaces - remove the coupling between the "big boxes" in the system, and only then go inside each box and improve its architecture. Otherwise we'll find ourselves redoing huge amounts of code over and over. Just moving your middle tier code base to JEE does not solve the problem... it is like adding meat balls to spaghetti, while what you really want is lasagna.

Just a thought.

Walls and bridges

2007-11-14T16:25:00.000+02:00

Last time I told you about the FunctionChain utility I wrote on top of google-collections. But I discovered a very strange problem, so I have posted a fixed version and I'd like to tell you about the problem, since I learned a lot while dealing with it.

I got a Function instance and I want to find out whether its apply method accepts nulls. In order to do that, I need to check the existence of @Nullable annotation on the first parameter of the apply method. The annotation's retention policy is run-time, so the task sounds pretty trivial, right? Wrong!

How do I get hold of the apply method? At first I thought function.getClass().getMethod("apply", Object.class) should do the job, you know, because of erasure. And indeed a method is always returned, only it was sometimes missing the annotations. This was driving me crazy, until I looked at all the methods of function using reflection. Guess what - there are sometimes multiple apply methods. How could it be - it's an anonymous inner class implementing the Function interface alone...? And then the flashback hit me - right, synthetic bridge methods! If I defined my function to transform Integer to String, there would be a String apply(Integer) method in addition to Object apply(Object). And if I defined my function to transform an Object to Class (equivalent of Object.getClass()) then I got 2 apply methods taking Object, one returns an Object and the other one a Class. Unbelievable, ha?!

Another thing worth noting, is that the annotation is present only on the original method, not on the bridge method. I have browsed the web and found 2 references to similar problem here and here. Both Michael Ernst and Rob Harrop think that not copying annotations to bridge method is a javac bug, but Sun Bug Database has no trace, so I'm going to submit one.

Ok, at least I knew now what's going on, but how do I find out the right method? I decided that the only way of doing it is figuring out the first parameter in the Function generic definition. I used Neal Gafter's super-type-tokens trick with some minor enhancements - array support and ability to process arbitrary number of parameters. It allows me to find out the classes that substitute generic parameter definitions, and once I get the first one - it's the apply method parameter class, so I'll use it to look up the method. The apply method return type should not be a problem, because according to getMethod JavaDoc the method with a more specific return type will be found (and I can't imagine how there can be more than two methods with same parameter in our case).

I had to change Neal's code a bit, because while he's using getClass().getGenericSuperclass() I had to use getClass().getGenericInterfaces() and to find the one representing Function. I noticed that interfaces need to be manually collected from all super-classes, because each class would only return the ones it explicitly implements, and (usually) not the ones that the super-class implements. Bummer... To add insult to injury - there may be several implemented Types that represent a Function - for example if the parent implements Function<T,String> and child implements Function<Integer,String>, those would be 2 different types, and I, of course, am looking for the most specific one. LinkedHashSet did the job. Phew.

You can look at the source code here, let me know what you think. All in all, it turned out to be much harder than I originally expected, and I am wondering if google-collection folks would be willing to add similar functionality to their package... after all run-time retention of @Nullable is useless without it.

Null-safe access to properties

2007-11-07T16:46:00.000+02:00

First I am going to show a nice side-effect of using properties with google-collections interface Function. Like many others I am annoyed to write code of the style


Bar getBarOfThing(Thing thing) {
  Bar bar = null;
  if (thing != null) {
    Foo foo = thing.getFoo();
    if (foo != null) {
      bar = foo.getBar()
    }
  }
  return bar;
}

All the null checks make it unreadable. And heaven forbid I forget one null test... BOOM! NPE! It's much nicer in SQL, for example, where everything's a bag, no results - no problem, join it with whatever you want, you'll just get an empty bag.

Now imagine that

foo = new Function<Thing,Foo>() {
  public Foo apply(Thing from) {
    return from.getFoo();
  }
}

and

bar = Function<Foo,Bar>() {
  public Bar apply(Foo from) {
    return from.getBar();
  }
}

Luckily google-collections have defined a @Nullable annotation and I am going to use it to differentiate between functions that can gracefully handle null (e.g. return some default value) and the ones that don't. Now I'm going to define a FunctionChain class, that provides me a fluent interface to chain the functions:

  FunctionChain<Thing,Bar> chain =
FunctionChain.<Thing>self().function(foo).function(bar);

Which I can safely apply to any Thing, including null

  assertTrue(chain.applyTo(null) == null);

Properties for Java

2007-11-06T16:07:00.000+02:00

The need for better support for properties in Java has been raised many times and in several contexts, and I am definitely not going to repeat the reasoning here. It used to be on the list of features proposed for Java 7, and here's the last time I read about it on Rémi Forax's blog (quite a while ago). Fred recently blogged about another way of addressing the problem via abstract Enums. There were several other interesting proposals, and the main trends are summarized very nicely by Stephen Colebourne here. Stephen is also the one who first blogged about bean-independent property object and wrote a reference implementation - Joda Beans project. I had started developing similar ideas in parallel, and since I have now reached a stage of a nice working prototype, this is what I hope to describe in the upcoming posts.

First of all, I chose the bean-independent property approach (although it was before Stephen coined it that), because I think the real added value in properties is being able to refer to the property regardless of any particular object instance. Another big design decision is to avoid changing Java compiler, unlike Fred and Rémi. I mean it's a very interesting exercise to play with javac and I am really grateful that Sun made it possible, but... IMO that's the last resort to solve the problem. I mean, even in the best case where the enhancement is accepted by Sun, the earliest time when it will work in an official Java version is Java 7, which is still very far away. And using a home-made javac in production environment is just not very realistic, I mean imagine justifying it to your boss (or customer). I also tried to stay away from bytecode or source generation, and managed well so far, although I might need to use it in the future in order to improve verbosity and safety of my solution.

So here is what I managed to do: no compiler changes, fully type-safe properties, some null-safety built in (ideas similar to what Stephen blogged here), the properties can have annotations and can be referred to in annotations, they can be applied to existing Java Beans via reflection and they expose their own meta-data via reflection. In addition to all that some Ruby-esque adding/modifying properties on the fly is supported. The last, but not least, is that I have been using google-collections quite a lot in this project so there's very nice inter-operation, for example Property implements com.google.common.base.Function and writing pseudo-functional code in Java using google-collections and fluent interfaces has been discussed quite a lot recently.

So this is the blog-post number 0, sort of preface, that hopefully will keep you interested enough to come back and read the blog. To be continued...

Functional programming for the JVM

2007-11-03T21:34:00.000+02:00

Dear Java, my good old friend, there is something I need to tell you. You've been my only one for over 10 years, but now I have trouble staying faithful. It's not you, it's me. And no, I am not moving out, not just yet, because I can't afford to leave all your stuff behind.

But you don't satisfy all my needs anymore!

I know you are trying, dear, and good people try to help you - here Neal Gafter is working hard to give you closure support, Ricky Clarkson, a true alpha-geek who played with the first prototype of Neal's work, has blogged the results. But, sorry love, I can't wait until the big makeover in Java 7. What do you expect me to do with all these JVM languages around? These languages inter-operate (well, to certain extent) with your libraries, so I can enjoy all the usual stuff but I can also be more expressive. Here is an example of hooking up JRuby with Velocity and another one of writing servlets in Scala under Tomcat.

You wanna know who they are? Among the young ones, Scala deserves to be mentioned first, alongside Ruby, JavaScript and Python to name just a few. Classic ones include several Lisp dialects (e.g. Clojure and Common Lisp), Scheme and OCaml implemented over the JVM as well. Either there is a compiler that produces bytecode, or there's a JVM scripting engine, and sometimes both. I hope one day to make a deeper study and produce a proper comparison table, maybe then you'll understand.

You think that closures alone aren't good enough reason to have a little flirt on the side with another language? Hmm, consider the fact that "the other language" may offer me hot-swapping of classes, mix-ins, better modularity, pattern matching, better concurrency model and dynamic typing support ... who can blame me for being tempted.

Still loving you, yours truly but... not entirely.

Beware: Overflows in numeric operations

2007-11-01T13:59:00.000+02:00

Well, isn't it amazing how huge can be the problems caused by a simple little code piece? Have a look at Java Puzzlers - there are plenty of examples. What is important to understand is that it's no fiction nor Neal and Josh perverted minds. It's out there in the software we use and the one we produce. Want proof? Nearly All Binary Searches and Mergesorts are Broken.

Here is the story of what happened to me. The code was written by a colleague, and at some point became my responsibility. I didn't notice anything wrong when first reviewing it, but later on I had to fix the problems. Now have a look:


public class Node implements Comparable {
  private long _id;
  
  public Node() {
    _id = IdGenerator.getNewId();
  }
  public Node(long id) {
    _id = id;
  }

  public boolean equals(Object obj) {
    return compareTo(obj) == 0;
  }

  public int hashCode() {
    return (int)(_id ^ (_id >> 32));
  }

  public int compareTo(Object obj) {
    if (obj == null) {
      return -1;
    }
    if (obj instanceof Node) {
      Node tmp = (Node) obj;
      return (int)(_id - tmp._id);
    }
    return -1;
  }
}

Looks innocent, right? Why would then new Node(12884902889L).equals(new Node(4294968297L)) be true?! Ah, because 12884902889-4294968297=8589934592=2^33 and when cast to int, 2^33 is of course 0. Bellisimo!

But hey, here's one even worse. IdGenerator generates long Ids from 2 integers - persistent high and in-memory low, low being incremented all the time and high only when MAX_VALUE of ids have been given out or when the process restarts. So far so good. Look at this function then:


public synchronized long getNewAndReserve(int interval) 
                    throws IdGeneratorException 
{
  long value;
  if (low + interval >= MAX_VALUE) {
    high = getNewHighValue();
    low = 0;
  }
  value = getNewId(high, low);
  low += interval;
  return value;
}

MAX_VALUE is BTW Integer.MAX_VALUE. Now think puzzlers... YES! a sum of 2 integers is an integer, it can NEVER exceed a MAX_VALUE, it overflows to a negative number! The fix for this is to change the condition to (low >= MAX_VALUE - interval) and to add an assertion that (interval > 0).

All this makes you think whether high-level programming language like Java should help you avoid this kind of problems...? But that's worth of another post sometime. In the meantime, look after your numbers, folks.

Why is this blog named this way?

2007-10-12T12:16:00.000+02:00

Because of my main occupation and lipstick colour.