How 'make check' works

Extensive work has been put into testing the accuracy of atlc, so I have a high-degree of confidence in the code. However, over the years that atlc has been developed, a few problems have been discovered, often only affecting one hardware system, or one operating system, or one compiler or one set of optimisation flags. For this reason, extenive self-checking is performed to try to discover any problems.

1. First, the basic accuracy of the code has been checked on a Sun Ultra 80, where all development is done. That checks the results are correct, but only on one platform.
   
   
2. A number of self tests call the programs create_bmp_for_circ_in_circ, create_bmp_for_circ_in_rect, create_bmp_for_microstrip_coupler, create_bmp_for_rect_cen_in_rect, create_bmp_for_rect_cen_in_rect_coupler, create_bmp_for_rect_in_circ, create_bmp_for_stripline_coupler and create_bmp_for_symmetrical_stripline. These programs create bitmap files. The validity of the bitmaps files is checked with Gimp, to meansure the width and height of the generated bitmaps, the diameters or sizes of conductors, etc. Note, it would take an error both in Gimp and one of
   the create_bmp_for_* files for an error to not be detected when viewed in the Gimp.
   
   
3. The MD5 checksums of these bitmaps are computed with the program mymd5sum in the tools/src subdirectory. The MD5 checksums of the generated should be identical to those expected. For example, test 2a-create_bmp_for_rect_in_rect.bmp.test executes
   
   create_bmp_for_rect_in_rect -b 4 500 100 20 240 20 460 50 40 1 4.8 test_rect_in_rect.bmp
   
   The MD5 checksum of test_rect_in_rect.bmp should be c761645b57a08057064eb284a44859e9 and if not, test 2a-create_bmp_for_rect_in_rect.bmp.test will fail.
   
   
4. The program atlc, is run, using the bitmaps generated above. For example, test 2b-create_bmp_for_rect_in_rect.txt.test executes:
   
   atlc -S test_rect_in_rect.bmp > test_rect_in_rect.txt
   
   This creates the text file test_rect_in_rect.txt as well as the bitmaps test_rect_in_rect.Ex.bmp, test_rect_in_rect.Ey.bmp, test_rect_in_rect.E.bmp, test_rect_in_rect.V.bmp, test_rect_in_rect.U.bmp and test_rect_in_rect.Er.bmp

.
5. The results of the numerical computations are checked against those expected, which should be close to the theoretical values. For example, in test create_bmp_for_rect_in_rect.txt.test the capacitance computation is expected to be 121.0 pF/m and the impedance 44.61 Ohms. If not, test 2b-create_bmp_for_rect_in_rect.txt.test will fail.
   
   
6. The MD5 checksums of the bitmap files created by atlc are computed and checked against what is expected. For example, test 2c-create_bmp_for_rect_in_rect.E.bmp.test compares the MD5 checksum of test_rect_in_rect.E.bmp to that was expected (49ce7df03fcd0ee4e0120269e136f506), test 2d-create_bmp_for_rect_in_rect.Er.bmp.test that of test_rect_in_rect.Er.bmp to that expected (83c85493d8cfc248b65b72fc6bb1935d) etc. In a few rare incidences, a single pixel of the bitmaps created in step 4 depend on the platform. The differences are due to rounding errors. For example, in test 1g-create_bmp_for_symmetrical_stripline.U.bmp.test, the MD5 checksum is computed of the file test_sym_strip.U.bmp, but depending on the platform, one pixel will vary in gray level by one shade of gray, so two MD5 checksums are considered acceptable - 353a368663a112d7e8f0af6a63df75f3 and a983da63a2f7b5d56d263d93988c1f8d. Such instances are rare.
   
   
7. atlc will, unless the -S option is given, write binary files, with the .bin extension. Only the lengths of these are currently checked - not their contents, as that would be too dependant on rounding errors. Perhaps at a later date their binary data will be convered into an integer format (say 256 levels) and the data compared. Currently no use is made of the binary data files, so not much effort has been put into checking them.


To be continued.