diff options
Diffstat (limited to 'README.TESTING')
-rw-r--r-- | README.TESTING | 170 |
1 files changed, 0 insertions, 170 deletions
diff --git a/README.TESTING b/README.TESTING deleted file mode 100644 index 44065bc7dc..0000000000 --- a/README.TESTING +++ /dev/null @@ -1,170 +0,0 @@ -[IMPORTANT NOTICE] ------------------- - Please do not ask developers why some or all tests are failing in -your environment! Let us know if you find why it fails. Thank you. - - -[Testing Basics] ----------------- - To execute test scripts, you must build PHP with some SAPI, then you -type "make test" to execute all or some test scripts saved under -"tests" directory under source root directory. - -Usage: -make test - - "make test" basically executes "run-tests.php" script -under the source root (parallel builds will not work). Therefore you -can execute the script as follows: - -TEST_PHP_EXECUTABLE=sapi/cli/php \ -sapi/cli/php [-c /path/to/php.ini] run-tests.php [ext/foo/tests/GLOB] - - -[Which "php" executable "make test" look for] ---------------------------------------------- - You must use TEST_PHP_EXECUTABLE environment variable to explicitly -select the php executable to be used to run the tests. That can either -be the CLI or CGI executable. - - "make test" executes "run-tests.php" script with "php" binary. Some -test scripts such as session must be executed by CGI SAPI. Therefore, -you must build PHP with CGI SAPI to perform all tests. - -NOTE: PHP binary executing "run-tests.php" and php binary used for -executing test scripts may differ. If you use different PHP binary for -executing "run-tests.php" script, you may get errors. - - -[Which php.ini is used] ------------------------ - "make test" uses the same php.ini file as it would once installed. -The tests have been written to be independent of that php.ini file, -so if you find a test that is affected by a setting, please report -this, so we can address the issue. - - -[Which test scripts are executed] ---------------------------------- - "run-tests.php" ("make test"), without any arguments executes all -test scripts by extracting all directories named "tests" -from the source root and any subdirectories below. If there are files, -which have a "phpt" extension, "run-tests.php" looks at the sections -in these files, determines whether it should run it, by evaluating -the 'SKIP' section. If the test is eligible for execution, the 'FILE' -section is extracted into a ".php" file (with the same name besides -the extension) and gets executed. -When an argument is given or TESTS environment variable is set, the -GLOB is expanded by the shell and any file with extension "*.phpt" is -regarded as a test file. - - Tester can easily execute tests selectively with as follows. - -Examples: -./sapi/cli/php run-tests.php ext/mbstring/* -./sapi/cli/php run-tests.php ext/mbstring/020.phpt - - -[Test results] --------------- - Test results are printed to standard output. If there is a failed test, -the "run-tests.php" script saves the result, the expected result and the -code executed to the test script directory. For example, if -ext/myext/tests/myext.phpt fails to pass, the following files are created: - -ext/myext/tests/myext.php - actual test file executed -ext/myext/tests/myext.log - log of test execution (L) -ext/myext/tests/myext.exp - expected output (E) -ext/myext/tests/myext.out - output from test script (O) -ext/myext/tests/myext.diff - diff of .out and .exp (D) - - Failed tests are always bugs. Either the test is bugged or not considering -factors applying to the tester's environment, or there is a bug in PHP. -If this is a known bug, we strive to provide bug numbers, in either the -test name or the file name. You can check the status of such a bug, by -going to: http://bugs.php.net/12345 where 12345 is the bug number. -For clarity and automated processing, bug numbers are prefixed by a hash -sign '#' in test names and/or test cases are named bug12345.phpt. - -NOTE: The files generated by tests can be selected by setting the -environment variable TEST_PHP_LOG_FORMAT. For each file you want to be -generated use the character in brackets as shown above (default is LEOD). -The php file will be generated always. - -NOTE: You can set environment variable TEST_PHP_DETAILED to enable -detailed test information. - -[Automated testing] - If you like to keep up to speed, with latest developments and quality -assurance, setting the environment variable NO_INTERACTION to 1, will not -prompt the tester for any user input. - -Normally, the exit status of "make test" is zero, regardless of the results -of independent tests. Set the environment variable REPORT_EXIT_STATUS to 1, -and "make test" will set the exit status ("$?") to non-zero, when an -individual test has failed. - -Example script to be run by cron(1): -========== qa-test.sh ============= -#!/bin/sh - -CO_DIR=$HOME/cvs/php4 -MYMAIL=qa-test@domain.com -TMPDIR=/var/tmp -TODAY=`date +"%Y%m%d"` - -# Make sure compilation enviroment is correct -CONFIGURE_OPTS='--disable-all --enable-cli --with-pcre' -export MAKE=gmake -export CC=gcc - -# Set test environment -export NO_INTERACTIVITY=1 -export REPORT_EXIT_STATUS=1 - -cd $CO_DIR -cvs update . >>$TMPDIR/phpqatest.$TODAY -./cvsclean ; ./buildconf ; ./configure $CONFIGURE_OPTS ; $MAKE -$MAKE test >>$TMPDIR/phpqatest.$TODAY 2>&1 -if test $? -gt 0 -then - cat $TMPDIR/phpqatest.$TODAY | mail -s"PHP-QA Test Failed for $TODAY" $MYMAIL -fi -========== end of qa-test.sh ============= - -NOTE: the exit status of run-tests.php will be 1 when -REPORT_EXIT_STATUS is set. The result of "make test" may be higher -than that. At present, gmake 3.79.1 returns 2, so it is -advised to test for non-zero, rather then a specific value. - - -[Creating new test files] -------------------------- - Writing test file is very easy if you are used to PHP. -See the HOWTO at http://qa.php.net/write-test.php - - -[How to help us] ----------------- - If you find bug in PHP, you can submit bug report AND test script -for us. You don't have to write complete script, just give us test -script with following format. Please test the script and make sure -you write the correct ACTUAL OUTPUT and EXPECTED OUTPUT before you -submit. - -<?php -/* -Bug #12345 -substr() bug. Do not return expected string. - -ACTUAL OUTPUT -XYXA - -EXPECTED OUTPUT -ABCD -*/ - -$str = "XYZABCD"; -echo substr($str,3,7); - -?> |