Test or spec? Test and spec? Test from spec!
Which came first? As an intermediate result towards a more general research
goal, it has recently been demonstrated that the egg precedes the omelet. A full
proof falls beyond the scope of this modest EiffelWorld column, but here is the
idea: you can construct the omelet from the egg, but not the egg from the
omelet. (The chicken is covered by a separate lemma.) The reader will already
have jumped mentally to an important special case; for omelet read test, and for
egg read specification, particularly in the form of Eiffel contracts.
We are being told from some quarters that you can't specify anyway, and that
development should be "test-driven". Eiffel programmers know from daily practice
and from observation of the libraries that the first proposition is false:
precise specifications in the form of contracts, even if they cover only part of
the functionality, radically change the software development process. In the new
scheme, tests play a critical role (see another little article of a few years
back, "Qualityfirst" at:
which I believe already suggested a few of the good ideas of agile methods), but
tests are not a replacement for specification: they follow from these
specifications. It is indeed remarkable to see how the presence of contracts can
drive the testing process. If a slogan is needed and two can do, I will venture
"Contract-driven testing" and "Test-obsessed development". That works very well
-- you should test all the time, with the intent of finding bugs -- but it's not
a reason to drop specification and design. Specification and design are what
propels both the testing process and the test cases themselves.
Going from specifications to test is one-way: you lose the abstractions. A
specification describes the general properties of a computation, for all
possible inputs; a test addresses one particular result for one particular
input. From the general you can deduce the particular, but not the other way
around: even a billion tests don't reveal the insight -- the abstraction -- of a
specification. Omelets beget no eggs.
For an Eiffel programmer, testing consists of turning on contract monitoring,
exercising components, and waiting for a contract to fail. Since any test oracle
can be expressed by a postcondition, but no collection of test oracles can
replace a specification, we lose nothing, but we gain a great advantage over
na´ve (contract-less) test-driven development: retaining the fundamental
abstractions that underlie our programs and their individual modules.
In some later installment of this column focused more on research, I'll discuss
how it is becoming possible to automate the component testing process
completely, test case generation included. Suffice it for the moment to note
that while we should be grateful to our extreme friends for helping to
rehabilitate the much maligned process of software testing, we -- Eiffel
programmers -- know something they don't seem to have realized yet: you can test
from specs, but you can't spec from tests.
-- Bertrand Meyer