The next programming frontier

June 2005

Comparing the general state of programming today to what most people practiced twenty, fifteen or even ten years ago, no one can deny that things have improved. Even for those who don't program in Eiffel, the level of abstraction has risen a bit. We reason less operationally and more statically; as an example that will appear obvious only to those who did not live through the pre-structured-programming days, we shed the goto style of reasoning -- "When we reach that label, we are coming from that place, so it must be the case that... wait, we could also be coming from that other instruction, but only if in the previous iteration x was positive, or maybe also when ...", and so on -- in favor of more reasonable control structures and a more peaceful program understanding process. (As soon as I write this, of course, I realize that I am still in an Eiffel mindset: just about everywhere else the goto continues to rear its ugly head as "exit", "break" or "return". Oh well. The progress of civilization isn't synchronous.)

Like an entire province forgotten by history, one area of programming is still living in the nineteen sixties, to the point that you can almost see the flowers in the hairdo and hear the Beatles on the radio. We do concurrent programming with the techniques of that time, in particular semaphores (Dijkstra, 1965), a technique that is even more of an insult to concurrent programming than the goto is to its sequential forms.

What makes this situation ever less bearable is that programming is becoming increasingly concurrent. Multithreading, in particular, is everywhere, handled through incredibly low-level techniques, usually through libraries relying on semaphores and similar techniques and affixed to object-oriented languages like a wooden wing to a race car.

We all suffer from the results, because so many applications today need multithreading or some other form of concurrency. A joke of the early nineties, at the time object technology was capturing the imagination of the masses if not yet their practice, was that OOP is like teenage sex: everyone says they're doing it, very few are actually doing it, and those who are doing it are not doing it very well. Not being a teenager, either now or in the nineties, I can't testify about the first term of the metaphor, but much of its conclusion applies today to concurrent programming, with the worrying difference that many are doing it, or trying to. Hence endless problems of "data races" (two operations wrongly handling the same value at the same time), deadlocks and the like. Debugging such applications is a nightmare; the huge literature on the topic (trying in particular to fix the "Java memory model") is a sign of the difficulty of the problems involved. The growing need for concurrency in many new areas -- whether through multithreading, Web services, distributed programming, multi-core CPUs, grid computing ... -- makes the need for better techniques ever more urgent.

The solution goes beyond patching this or that particular case; instead it involves applying to concurrent programming the same leap of abstraction that structured programming, and object technology after it, brought to sequential programming. We must stop reasoning so operationally, if only because this is an order of magnitude harder than with gotos in sequential programs; instead we must look for the proper abstract concepts to reason statically about concurrent programs. This is the most critical challenge in programming today.

For a number of years the SCOOP model under its successive refinements has attempted to provide such a solution. I won't describe it here in any detail since this is taken care of by chapter 30 of "Object-Oriented Software Construction, 2nd edition" and also by a number of papers reachable from the URL given below. Suffice it to say that SCOOP takes advantage of the abstraction concepts of Eiffel and particularly of Design by Contract to provide a clear and simple mechanism of concurrent programming. Just one example: in the famous "Dining philosophers" problem, the basic operation in SCOOP, executed by a philosopher, is just

eat (left, right)

which grabs the left and right fork, at once. Both are "separate" objects. This is typical of how SCOOP transfers to the programming model, and the language implementation, treacherous tasks which traditionally are handled by the programmer -- in this case reserving two objects, with the difficulty of what to do if only one is available (you might already have reserved the other, and hence must relinquish it, but in the right way -- this is all tricky and error-prone).

The ideas have been around for a while -- although they continue to be polished and improved -- but remained largely a paper design, in spite of the appearance of a few prototypes over the years (one from Eiffel Software as early as 1995). What's really new is the full-fledged effort at implementation being pursued at ETH in Zurich. You can see the current results, including the latest publicly downloadable releases, at

http://se.ethz.ch/research/scoop.html

The implementation, currently built on top of .NET, uses a library, SCOOPLI. A preprocessor translates from SCOOP text (Eiffel with the "separate" keyword extension and associated semantic rules) into standard Eiffel with calls to SCOOPLI, so you can use all the standard Eiffel tools, EiffelStudio in particular. All the examples in the OOSC-2 chapter mentioned above, plus several others, most with a GUI interface, have been implemented and are available for download.

SCOOP and SCOOPLI as currently implemented are only a first step. Many issues remain to be addressed. But we firmly believe that they are the root of a solution to the most urgent and far-reaching problem in programming today. Take a look at SCOOP, and see if you agree.

--Bertrand Meyer
(Let me use the opportunity for a little pitch for what promises to be a memorable event on the topic of concurrent and real-time programming, with some of the best experts in the history of the field: there are still seats available for the LASER summer school, Elba, 11-17 September: http://se.inf.ethz.ch/laser/.)