10.1 Running the tests

Running the test cases is quite simple. Typically, you might do it after compiling but before installing CSSC. After running “configure”, you might compile CSSC with

make

and test it with

make check

The full test suite takes just over five minutes to run on a 486 running Linux. If everything works correctly, you will see messages like:-

cd tests && make all-tests
make[1]: Entering directory `..../CSSC/compile-here/tests'
cd ../lndir && make
make[2]: Entering directory `..../CSSC/compile-here/lndir'
make[2]: `lndir' is up to date.
make[2]: Leaving directory `..../CSSC/compile-here/lndir'
../lndir/lndir ../../Master-Source/tests
../../Master-Source/tests/get:
command-names: .././common/command-names
test-common: .././common/test-common

...more messages from lndir...

/bin/sh -ec 'cd admin && for i in *.sh ; \
do \
	/bin/sh $i || break; \
done'
C1...passed
C2...passed

.... more output ....

C12...passed
C13...passed
PASS comment.sh:
v1...passed

.... more output ....

b11...passed
b12...passed
PASS flags.sh:

.... more output ....

Tests passed.
make[1]: Leaving directory `..../CSSC/compile-here/tests'

If something goes wrong you will see a “FAIL” message, which looks something like this:-

C1...FAIL comment.sh C1: ../../admin -ifoo s.new.txt:
Expected return value 0, got return value 1
make[1]: *** [test-admin] Error 1
make[1]: Leaving directory `..../CSSC/compile-here/tests'
Tests failed.
make: *** [all-tests] Error 2

The thing to remember is that when you run make check, the make program will print on the last line a message saying “Error” only if the tests have failed.

If the test suite does indicate that the tests have failed, please submit a bug report (see Reporting Bugs). Please include in your bug report

• The output of the test suite (you may find the Unix script program invaluable for this)
• The contents of the directory containing the test that failed (if you compiled with separate source and object directories, I just want the one in the “object” directory).
• As much information about your system as you think is useful, for example the names and versions of the operating system and compiler that you are using.

If you want to run just some of the tests, there are rules in the makefile for just running some of them. For example, the tests in the directory tests/admin can be run with make test-admin. Each test directory is named after one of the CSSC programs. This indicates which program the tests concentrate on verifying. Inevitably these tests will use more than just one CSSC program; for example, most of the tests involve using admin to create a SCCS file in the first place. However, the directory indicates which tool those tests concentrate on.

It is possible for a test to neither pass or fail, but just go wrong. This can happen when the test script comes upon something that prevents the test itself working correctly, for example, because it can’t remove a temporary file or uudecode a built-in sample SCCS file. When this happens you get output much like this:-

$ sh flags.sh
rm: foo: Permission denied
flags.sh: Test could not be completed

The part before the colon (flags.sh) indicates which script could not be completed. No further tests will be attempted. Diagnosing the problem may or may not be simple. In this case, it’s not hard; the problem is that the test suite is trying to clear away any temporary files but it can’t remove the file “foo” (because the current directory was made read-only to force the test to miscarry, in this contrived case). When the test suite miscarries like this and you can’t find the problem, please follow the bug-reporting procedure (see Reporting Bugs), but please indicate that it is a miscarriage in the test suite rather than a concrete test failure.