summaryrefslogtreecommitdiff
path: root/README.testing
blob: 238e01f0a3205cd1cf605768cc064ca372b4a6f1 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
The testing framework basically aims to run the create and deploy tools in a
controlled environment to check for any regressions during code changes and
refactoring.

This are the rules of the test club:

- For each new feature there should be a unit test
- For each bug fixed there should be a unit test

Running the test suite
----------------------
You need to change to the tests directory and execute run_tests.sh:

$ ./run_tests.sh

To run each test individually:

$ ./00_my_individual_test.sh ../path/to/test-create ../path/to/test-deploy

To run the cross platform test:

$ fakeroot -- ./cross_plat.sh

Anatomy of a unit test
----------------------

Each test needs to copy the following boilerplate:

		#!/bin/sh

		TEST_ID="01"
		TEST_NAME="My unit test"

		CREATE=`pwd`/$1
		DEPLOY=`pwd`/$2
		TEST_TOOLS=$3
		. ./test_lib.sh

		############# Test specific code ############
		setup () {
		}
		
		check_results () {
		}
		#############################################
		main $@

Test lib is a shell function library that sets up the environment for the test.
To create a test, we have two main variables $ORIGIN and $TARGET which are the
origin and target directories of the diff respectively. The setup function
should populate the origin and the target directories for the diff test.

This is an example of a simple file difference test. It takes into account both
contents and metadata (mtime, ownership and permissions):

		ORG_FILE=$ORIGIN/b.txt
		TGT_FILE=$TARGET/b.txt

		setup () {
			echo 1 > $ORG_FILE     && \
			echo 2 > $TGT_FILE     && \
			chown :cdrom $TGT_FILE    && \
			chmod 707 $TGT_FILE
		}

		check_results () {
			check_content    $ORG_FILE "2" && \
			check_perm       $ORG_FILE 707 && \
			check_group      $ORG_FILE cdrom && \
			check_same_mtime $ORG_FILE $TGT_FILE
		}

If either create or deploy should fail then define a test_return function that
compares the exit code of the program. If it is supposed to fail with a specific
exit code then check that the codes are equal, if it is just supposed to fail
then check that it is non-zero

		# create must succeed
		create_test_return () { test $1 = 0; }
		# create must fail
		create_test_return () { test $1 != 0; }
		# create must fail with error code 1
		create_test_return () {	test $1 = 1; }


Cross Platform Test
-------------------

The cross-platform test is designed to make sure that patches are endian and 
architecture independent. This is done with PATCHES generated and commited to
begin with, and then deployed and re-created and compared to check that the
patches are identical. This test should be run after every modification to
ensure that tbdiff remains endian and architecture independant after any
modifications are made.