Writing Test Suites in LISP

Context

Two issues with sharing code are:

• Explaining your code to other people;
• Making sure that your new changes does not break existing code.

Unit tests are a useful way to address these issues. A unit test is a canned example that runs some functions. The outputs are automatically compared to some expectation in order to see if a test has passed or failed the tests.

Unit tests are also very useful when writing your own code. Authors like Kent Beck argue that software development should be structured around building and running unit tests. Test-driven development (TDD) requires the programmer to first fail the test cases. The idea is to ensure that the test really works and can catch an error. Once this is shown, the normal cycle will commence. This has been coined the "Test-Driven Development Mantra", known as red/green/refac- tor where red means fail and green is pass. In a , work proceeds as follows :

• Planning rules:
  • User stories are written. (a.k.a. lots of tests);
  • Make frequent small releases (a.k.a. shows a string of tests);
  • Fix the program only when it breaks (a.k.a. "code-run-break" is the usual cycle).
• Coding rules:
  • Code the unit test first.
• Testing rules:
  • All code must have unit tests.
  • All code must pass all unit tests before it can be released.
  • When a bug is found tests are created.
  • Acceptance tests are run often and the score is published.

We offer here an adaption of Peter Seibel's unit testing code. The deftest function defined a new test and the tests macro runs a bunch of tests, the prints the number of passes and fails. For example, the following code defines a function test-bell that checks that the function (bell!) returns a distribution structure containing a bell shaped curve:

(deftest test-bell! ()
  (check
    (samep "#S(DIST
               :BUCKETS ((1 . 0.0)     (2 . 0.0)     (3 . 0.0625) (4 . 0.15625) ( 5 . 0.28125)
                         (6 . 0.28125) (7 . 0.15625) (8 . 0.0625) (9 . 0.0)     (10 . 0.0))
               :CARDINALITY 10
               :MIN 0
               :MAX 100)"
           (bell!))))

(The samep function compares two strings after removing all white space- so the test result can be laid out pretty without mucking up the test.)

At the time of this writing, the code has six deftests. We can run them all as follows:

CL-USER> (tests)
......
PASSES: 6 (100 %)
FAILS : 0
NIL

Which is to say that we ran six tests and none of them failed.

Hint: it is useful to write unit tests are written in a hierarchy. If some nested test is called within a super-test, then the code recurses into sub-tests. In this way, so centralized test facility can be defined in terms of multiple layers of tests spread through out in multiple files across the entire code.