Software: Live and Let Die

This week I was lucky to attend Gilad Bracha's guest lecture on Networked Serviced Programming at the Hebrew University. He has been talking about Service Objects for some time now, but nothing compares to hearing it live - Gilad's presentation was witty and fun!

So here is my interpretation and some take-aways.

What's the problem with software? It is too damn complex. Projects crumble under their own weight. It happens to successful projects - our dearly loved Java, for example. Also look at Vista, if you dare. And it's certainly true for the monster-size projects I used to work on. Once upon a time our team took the corporate "quality improvement" policy seriously and decided to investigate what causes bugs in our multi-million lines of code project. We collected all sorts of statistics and ran all possible metrics (which was tricky 'cause some of the tools would choke on such a huge code-base) but long story short our finding was this: the only metric that correlated clearly with defectiveness was LoC. It is hardly news, but a cure to the disease has yet to be found.

There are certainly several things to be done, but what this talk focused on is getting rid of code which shouldn't be there, or in other words - dumping unused code and backwards compatibility. The way we work today - we are bound to not just specs and APIs, but to all the accidental behaviors and bugs in the previous version of our code. It seems that Gilad views code as if it was a live organism. Staying alive means being connected (and network plays a central role in his vision), but we should make way for evolution and some code should die - Gilad calls it "bit rot".

So how do we turn software into a healthy living organism? According to Gilad, there are several things to be done on the technical front first.

• Take advantage of the network: maintain a bi-directional connection with the control center - let programs pull upgrades from the net, but also send back operation statistics. This means that programming platform has to be aware of the network, and aware of the fallacies of distributed computing. This is cloud computing utopia: Internet as a platform, browser as an OS, and Javascript as the low-level programming language into which other languages can compile (in a GWT kind a way).
  
• Modularity: it should be possible to extend and replace individual objects without interference to the whole organism. Gilad has a well developed theory about how modularity should be done in a programming language, based on principles of object-orientation using mix-ins, nested classes and inheritance hierarchies; his new language, Newspeak, is going to implement it.
• Explicit Dependencies between modules - no static, no imports, modules are truely independent and dependency management (wiring) is performed by passing other module instances as parameters to module constructors. This allows to maintain clear boundaries between modules and flexibility in module composition.
  
• Frequent Updates: to allow the "clients" of the object APIs to deal with changes, in addition to maintaining modularity, the changes should be made small and frequent. Call it agility, if you like. That means that we can't afford reboots, and we need to find out when the system is quiescent so that upgrade can be performed, which brings us to the next point...
  
• Reflection and Hot-swapping: objects should allow other objects to find out both static and run-time information about them without breaking the encapsulation. Objects should also allow other objects to modify them "live". Gilad and his team at Cadence are building the support for these features in Newspeak using Mirrors, a concept that originates in Self programming language. It's worth noting that there exist dynamically typed languages that implement hot-swapping today - Erlang being one of them.
  
• Security is important in any distributed system, and even more so if we allow remote objects to mess with the program. So in addition to dynamic typing and pointer safety, Gilad proposes mirrors to be guarded by capability-based security, similar to the one in E programming language.
• Synchronization: many programs need to work with persistent data and it is important to keep the program and the data in-sync. Gilad proposes orthogonal synchronization, based on Smalltalk orthogonal persistence idea, where objects are split into transient and persistent ones by marking object tree roots accordingly. Persistent objects are upgraded whenever the corresponding part of the program is upgraded, and transient objects are lazily recomputed. If the data is ever to outlive the service, it would be exported into some generic format, such as XML.
  
• The most extreme and bold part of this vision is probably No Versions and No Releases - there would be only one version for every program out there. Gilad sees software becoming more of a service than a product, but in order for this to realize we'll have to overcome not just technological, but also psychological and economical barriers - we will have to change the way we develop software and the way we make money of it.
  

Bottom line, all this may sound too futuristic, but "software as a service" and "platform as a service" are making their way in the industry and this wave, if strong and successful enough, may bring the significant change Gilad is predicting.

P.S. As for the rest of us, living in the JVM world, some of the ideas ring a bell. We hope that modularity JSRs and OSGi will improve Java. Those of us who survived Ant and Maven, and felt the weight of a DI framework, will probably appreciate the amount attention Gilad is putting into software composition. It's worth noting the attempts to address hot-swapping on JVM, such as JavaRebel and Jonas Boner's experiments with Scala Actors and Terracotta. Terracotta server also utilizes some ideas which (in my mind at least) look quite similar to the orthogonal synchronization scheme.