What will remain of Extreme Programming?

February 2006

It is difficult after reading "Extreme Programming Refactored: The Case against XP" by Matt Stephens and Doug Rosenberg (Apress, 2003) to suppress a smile the next time you hear extreme praise for XP. Placed under the patronage of Groucho Marx, the book takes delight in constrasting the hype of those it calls "the Extremos" with the more prosaic results they produce. The funniest chapter is the autopsy of the "C3" project at Chrysler. C3 catapulted XP to world prominence, its wave reverberating as far as The Economist whose adoring article, one year after the termination of C3, missed the minor detail that this mother of all eXtreme projects had after four years of effort failed to deliver, and was discarded. The chapter includes a detailed flowchart on how to put a good face on such a regrettable turn of events, with boxes and arrows labeled "Hype and Fanfare!", "Generate a Quick Illusion of Success" ("We're the King of the World!"), "Refactor endlessly" (with a self-pointing arrow causing a loop, of course), "Abandon Ship!", "Cancellation", "Claims of Victory and Success", "Puzzlement in the Newsgroups", leading to the apotheosis: "Claims That It Wasn't Really Important".

If, as seems likely, XP joins the rapidly growing cemetery of software buzzwords, will that mean it hasn't have made any contributions? A fair answer has to be more nuanced.

Let's deal with the surplus baggage first. We can ignore the folklore, like pair programming (the kind of nice idea that may be useful once in a while and worth a footnote somewhere, but hardly a software engineering principle), or CRC cards. We should firmly reject the truly wrong ideas, such as the rejection of specification and design and the refusal to consider future change and reuse in the original plan. This is just bad software engineering and best forgotten. The same applies to the mantra that you can promise customers deadlines, or you can promise them functionality, but not both. Sure, this is a very comfortable principle if you are a consultant, and it sounds very much late twentieth century, the time (remember?) when companies were so desperate for developers, any developers, that consultants could impose their every condition. The world doesn't work like this any more, if only because offshore development has changed the rules of the game (see "The Unspoken Revolution in Software Engineering" in IEEE Computer, January 2006, http://se.ethz.ch/~meyer/publications/computer/outsourcing.pdf, an expansion of the August 2004 EiffelWorld column at http://tinyurl.com/d9ruo. In 2006, and in fact in 1996 and always, the successful software developers are those who can indeed consistently guarantee both functionality and delivery times. It is rewarding to know that some have relied on Eiffel to build consulting businesses that have succeeded through this recipe, over many years and away from the successive waves of short-lived hype.

At a basic level of analysis, Extreme Programming is not a technical discipline but a sociological phenomenon: the revolt of the cubicles. Dilbert vs Dilbert's boss. Picture a speech by a pointy-haired manager or fancy consultant who's chanting process, UML, RUP, CMMI, ISO; and picture the street-smart developers' reaction: it's not plans and standards and boxes and arrows and milestones and meetings and policies that run, or don't, on the computer. It's code, and we are the ones who understand code. This rehabilitation of the program as artefact is fundamentally healthy and I think it is what will remain of Extreme Programming. Never again will we let ourselves be brainwashed into forgetting that programs are the stuff of our trade and that everything else is scaffolding.

This idea has always been at the core of the Eiffel method, in particular through the principles of seamless development expounded (on the basis of ideas of Brian Henderson-Sellers) in a paper of 1990, then in "Object Success" (Prentice Hall, 1995) and later publications. But where Eiffel differs fundamentally from XP and, I think, gets things right, is that the primacy of program text in no way implies the rejection of analysis and design. It's exactly the reverse: Eiffel raises the level of abstraction of the programming language so that the same techniques can be applied to analysis, specification, design, implementation and further. Instead of despising the program, like Dilbert's boss probably does, we make everything -- all the products of the software lifecycle -- be programs. A design, for example, is no longer some vague combination of boxes and arrows with fuzzy semantics, but just an abstract program: it concentrates on the essential, leaving implementation details to be filled in later on, while already enjoying the precision and clarity that characterize texts expressed in a good programming language; and it can be processed by the same tools, such as EiffelStudio, that handle fully implemented programs.

Contracts, deferred classes, inheritance and other Eiffel techniques make it possible to work seamlessly throughout the process, with a constant concern for reuse and extension, those critical concerns of software engineering that, notwithstanding their dismissal by XP, we cannot ignore if we are going to serve our customers right.

Other good XP ideas follow from this restoration of the role of code. The most significant one is the insistence on fast delivery cycles. XP did not invent this (the "daily build" was practiced by Microsoft before), but made it popular. "Object Success" includes an "Integration Principle" that states: "The time between successive integrations of all of a project's cluster should never be more than four weeks", followed by a recommendation to shoot for two weeks. These guidelines sound rather timid today (four weeks or even two is a long time!), but at least the general idea was there: the "periodic big bang" approach of letting everyone develop in all directions for several months, then trying to put everything together and see what happens, just doesn't work. XP removed the timidity.

We should integrate all the time; but that's not an excuse to forego specification, design, reuse, extendibility, and all that distinguishes professional software engineering from delightful hacking. XP, on the other hand, got the basic intuition right: to paraphrase Stephen Hawking (http://tinyurl.com/99kt4), it's code all the way down.

 -- Bertrand Meyer

Bookmark and Share