[rt25901_atf] tests/HOWTO-unit-test mostly moved to Devel Guide

- moved major parts of HOWTO to Developer's Guide, leaving only
  brief notes and pointers to new location

- described Valgrind

3 files changed, 148 insertions, 53 deletions

diff --git a/doc/devel/atf.dox b/doc/devel/atf.dox
index 29615c7e..ff4e182b 100644
--- a/doc/devel/atf.dox
+++ b/doc/devel/atf.dox
@@ -3,6 +3,29 @@
 
 @section testsOverview Testing Overview
 
+In DHCP, a unit test exercises a particular piece of code in
+isolation. There is a separate unit test per module or API. Each unit
+test lives in a directory beneath the code it is designed to exercise.
+So, we (will eventually) have:
+
+@verbatim
+server/tests/
+client/tests/
+common/tests/
+dhcpctl/tests/
+...
+@endverbatim
+
+And so on.
+
+Ideally each function would be invoked with every possible type of input, and
+each branch of every function would be checked. In practice we try to be a bit
+more pragmatic, and target the most basic operations, as well tricky code, and
+areas we have seen bugs in the past.
+
+We are using <a href="http://code.google.com/p/kyua/wiki/ATF">ATF (Automated
+Test Framework)</a> as a framework to run our unittests.
+
 @section testsAtf ATF unit-tests
 
 ATF stands for Automated Test Framework, and is the framework used for unit
@@ -21,17 +44,17 @@ tested version of ATF that DHCP's unittests were run against is 0.15.
 To build the unit-tests, use the following:
 
 @verbatim
-./configure --with-atf
-make
-make check
+$ ./configure --with-atf
+$ make
+$ make check
 @endverbatim
 
 The following syntax is supported as well:
 @verbatim
-./configure --with-atf=/path/to/your/atf/install
+$ ./configure --with-atf=/path/to/your/atf/install
 @endverbatim
 
-but it seems to have troubles detecting ATF installation, at least
+but it seems to have troubles sometimes detecting ATF installation, at least
 with ATF 0.14 and Mac OS X 10.6.8.
 
 Each code directory (e.g. server/) that has unit-tests has a sub-directory
@@ -42,69 +65,131 @@ Unit-tests are grouped into suites, each suite being a separate
 executable. The typical way to run tests is:
 
 @verbatim
-atf-run | atf-report
+$ atf-run | atf-report
 @endverbatim
 
-atf-run will read the Atffile in the current directory and execute
-all the tests specified in it. Using atf-run - rather than calling the
-test binary directly - has several major benefits. The main one is that
-atf-run is able to recover from test segfault and continue execution
-from the next case onwards.
+atf-run will read the Atffile in the current directory and execute all the tests
+specified in it. Using atf-run - rather than calling the test binary directly -
+has several major benefits. The main one is that atf-run is able to recover from
+test segfault and continue execution from the next case onwards. Another is that
+it is possible to specify a timeout for a test. atf-run will kill the test in
+case of any infinite loops and will continue running next tests.
 
-It is possible to run atf-run without passing its output to atf-report,
-but its output is somewhat convoluted. That is useful in some situations,
-e.g. when one wants to see test output.
+It is possible to run atf-run without passing its output to atf-report, but its
+output is somewhat convoluted. That is useful in some situations, e.g. when one
+wants to see test output.
 
-It is possible to run test binary directly. The only required parameter
-is the test case name. The binary will print out a warning that direct
-binary execution is not recommended as it won't be able to recover
-from crash.  However, such an approach is convenient for running the
-test under the debugger.
+It is possible to run test binary directly. The only required parameter is the
+test case name. The binary will print out a warning that direct binary execution
+is not recommended as it won't be able to recover from crash.  However, such an
+approach is convenient for running the test under the debugger.
 
 @section testsAtfAdding Adding new unit-tests
 
-There are a small number of unit-tests that are not ATF based. They will
-be converted to ATF soon. Please do not use any other frameworks.
+There are a small number of unit-tests that are not ATF based. They will be
+converted to ATF soon. Please do not use any other frameworks.
 
 Sadly, the DHCP code was not written with unit-testing in mind: often a
-non-standard approach is required for writing unit-tests. The existing
-code often has many dependencies that make testing a single piece of code
-awkward to unit test.  For example, to test hash tables, one needs to
-also include the OMAPI code. Rather than significantly refactoring the
-code (a huge task that could take months), we decided to link whatever
-is needed in the tests. If developing new test suite, it is recommended
-that you take a look at existing tests and just copy them.  In particular,
-the following things should be done for adding new tests:
-
-1. Create new file that will hold test code. It is recommended you
-name it (tested_feature_name)_unittest.c and put the file in specified
-tests directory.  For example tests related to hash tables used on the
-server side should be named server/tests/hash_unittest.c. If in doubt,
-it is convenient to name the test code after the file that holds tested
-code, e.g. server/mdb6.c is tested in server/tests/mdb6_unittest.c.
-
-2. Implement the test. The file server/tests/simple_unittest.c holds a
-template explaining the basic layout of the ATF tests.  There may be many
-test cases in a single *_unittest.c file. Make sure that you register
-all your test cases using ATF_TP_ADD_TC() macro, and try to minimize
-modifications to the tested code if possible. Keep in mind that we are
-using modernized \ref codingGuidelines for test development. You are
-advised to also look at atf-c-api(3).
-
-3. Extend Makefile.am to build your test. In particular, add your binary
+non-standard approach is required for writing unit-tests. The existing code
+often has many dependencies that make testing a single piece of code awkward to
+unit test.  For example, to test hash tables, one needs to also include the
+OMAPI code. Rather than significantly refactoring the code (a huge task that
+could take months), we decided to link whatever is needed in the tests. If
+developing new test suite, it is recommended that you take a look at existing
+tests and just copy them as a starting point.
+
+
+In particular, the following
+things should be done for adding new tests:
+
+<b>1. Tests directory.</b> For each code component (server, client, common,
+etc.) there should be a tests subdirectory. If it isn't there yet, then it must
+be created. This can be done by:
+
+a). Creating the directory:
+
+@verbatim
+    $ mkdir $subdir/tests
+    $ cvs add tests
+@endverbatim
+
+b). Adding the subdirectory to the build system:
+
+    Add to $subdir/Makefile.am:
+
+@verbatim
+    SUBDIRS = tests
+@endverbatim
+
+    Add to the AC_OUTPUT macro in configure.ac:
+
+@verbatim
+    subdir/tests/Makefile
+@endverbatim
+
+c. Create a Makefile.am in the new directory, something similar to this:
+
+@verbatim
+    AM_CPPFLAGS = -I../..
+
+    check_PROGRAMS = test_foo
+
+    TESTS = test_foo
+
+    test_foo_SOURCES = test_foo.c
+    test_foo_LDADD = ../../tests/libt_api.a     # plus others...
+@endverbatim
+
+See existing Makefile.am for examples, and the Automake documentation:
+
+    http://www.gnu.org/software/automake/manual/html_node/Tests.html
+
+<b>2. Implement the test.</b> That typically means that you create a new file that will
+hold test code. It is recommended you name it (tested_feature_name)_unittest.c
+and put the file in specified tests directory.  For example tests related to
+hash tables used on the server side should be named
+server/tests/hash_unittest.c. If in doubt, it is convenient to name the test
+code after the file that holds tested code, e.g. server/mdb6.c is tested in
+server/tests/mdb6_unittest.c.
+
+The file server/tests/simple_unittest.c holds a template explaining the basic
+layout of the ATF tests.  There may be many test cases in a single *_unittest.c
+file. Make sure that you register all your test cases using ATF_TP_ADD_TC()
+macro, and try to minimize modifications to the tested code if possible. Keep in
+mind that we are using modernized \ref codingGuidelines for test
+development. You are advised to also look at atf-c-api(3) man page.
+
+To add a new test, such as when a new module is added or when you want to start
+testing existing code, you can copy the server/tests/simple_unittest.c as a new
+new file, add the new file as a target in Makefile.am, and begin adding
+tests. Reviewing that file is a good idea, even if you decide to write your test
+from scratch, as it give you quick overview of the essential capabilities of the
+ATF framework (how to write test, how to make checks, pass or fail test
+etc.). Do not forget to add your new file to git via "git add
+yourtest_unittest.c".
+
+<b>3. Extend Makefile.am</b> to build your test. In particular, add your binary
 name to ATF_TESTS. The tests directory will be built only in case where
-ATF is enabled, using --enable-atf during configure phase.
+ATF is enabled, using --with-atf during configure phase.
 
-4. Modify Atffile to include your new test binary, if needed. If you
-followed naming convention proposed in step 2, your test will be included
-and will be included automatically.
+<b>4. Modify Atffile to include your new test</b>, if needed. Tests in specified
+directory must be registered in Atffile. See server/tests/Atffile for
+example. Currently every executable with name that falls into *_unittest pattern
+will be executed automatically. If you followed naming convention proposed in a
+previous step, your test will be included and will be included automatically.
 
-5. Enjoy your improved confidence in the code, as you can run the tests after
+<b>5. Enjoy your improved confidence in the code</b>, as you can run the tests after
 any change you may want to do:
 
 @verbatim
-make check
-atf-run | atf-report
+$ make check
+@endverbatim
+
+or run them manually
+
+@verbatim
+$ cd server/tests
+$ atf-run | atf-report
 @endverbatim
 
 @section testsAtfCoding ATF Coding Guidelines
diff --git a/doc/devel/mainpage.dox b/doc/devel/mainpage.dox
index d96e4d2e..e67f290b 100644
--- a/doc/devel/mainpage.dox
+++ b/doc/devel/mainpage.dox
@@ -19,6 +19,7 @@
     - @subpage qaTests
     - @subpage cppcheck
     - @subpage doxygen
+    - @subpage valgrind
   - @subpage debug
   - @subpage omapi
     - @subpage omapiIntro
diff --git a/doc/devel/qa.dox b/doc/devel/qa.dox
index 32404957..7505330e 100644
--- a/doc/devel/qa.dox
+++ b/doc/devel/qa.dox
@@ -81,4 +81,13 @@ See tests/tools/perfdhcp directory in BIND10 source code.
 href="http://tahi.org/logo/dhcpv6/">DHCPv6 conformance tests</a>. ISC plans to
 deploy and run them periodically in a near future.
 
+ @section valgrind Memory correctness using valgrind
+
+<a href="http://valgrind.org/">Valgrind</a> is a powerful tool for dynamic code
+analysis. It allows running existing code (often even without recompiling) in a
+special environment that tracks memory operations. In particular, it is able to
+detect: memory leaks, buffer overflows, usage of uninitialized memory, double
+frees and similar errors. We currently do not use valgrind in ISC DHCP testing,
+but there are plans for start using it.
+
 */