diff options
Diffstat (limited to 'lib/common_test/doc/src/common_test_app.xml')
| -rw-r--r-- | lib/common_test/doc/src/common_test_app.xml | 557 |
1 files changed, 0 insertions, 557 deletions
diff --git a/lib/common_test/doc/src/common_test_app.xml b/lib/common_test/doc/src/common_test_app.xml index 3fcbda538a..07a2a3e2cd 100644 --- a/lib/common_test/doc/src/common_test_app.xml +++ b/lib/common_test/doc/src/common_test_app.xml @@ -52,564 +52,7 @@ <item>Step-by-step execution of test cases</item> </list> - <p>The following section describes the mandatory and optional test suite - functions that <c>Common Test</c> calls during test execution. - For more details, see section - <seeguide marker="write_test_chapter">Writing Test Suites</seeguide> - in the User's Guide.</p> - </description> - - - <funcs> - <fsdescription> - <title>Test Case Callback Functions</title> - <p>The following functions define the callback interface - for a test suite.</p> - </fsdescription> - <func> - <name since="">Module:all() -> Tests | {skip,Reason} </name> - <fsummary>Returns the list of all test case groups and test cases - in the module.</fsummary> - <type> - <v>Tests = [TestCase | {testcase,TestCase,TCRepeatProps} | {group,GroupName} | {group,GroupName,Properties} | {group,GroupName,Properties,SubGroups}]</v> - <v>TestCase = atom()</v> - <v>TCRepeatProps = [{repeat,N} | {repeat_until_ok,N} | {repeat_until_fail,N}]</v> - <v>GroupName = atom()</v> - <v>Properties = [parallel | sequence | Shuffle | {GroupRepeatType,N}] | default</v> - <v>SubGroups = [{GroupName,Properties} | {GroupName,Properties,SubGroups}]</v> - <v>Shuffle = shuffle | {shuffle,Seed}</v> - <v>Seed = {integer(),integer(),integer()}</v> - <v>GroupRepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | repeat_until_any_ok | repeat_until_any_fail</v> - <v>N = integer() | forever</v> - <v>Reason = term()</v> - </type> - - <desc> - <p>MANDATORY</p> - - <p>Returns the list of all test cases and test case groups in the - test suite module to be executed. This list also specifies the - order the cases and groups are executed by <c>Common Test</c>. - A test case is represented by an atom, - the name of the test case function, or a <c>testcase</c> tuple - indicating that the test case shall be repeated. A test case group is - represented by a <c>group</c> tuple, where <c>GroupName</c>, - an atom, is the name of the group (defined in - <seemfa marker="#Module:groups/0"><c>groups/0</c></seemfa>). - Execution properties for groups can also be specified, both - for a top-level group and for any of its subgroups. - Group execution properties specified here override - properties in the group definition (see - <seemfa marker="#Module:groups/0"><c>groups/0</c></seemfa>). - (With value <c>default</c>, the group definition properties - are used).</p> - - <p>If <c>{skip,Reason}</c> is returned, all test cases - in the module are skipped and <c>Reason</c> - is printed on the HTML result page.</p> - - <p>For details on groups, see section - <seeguide marker="write_test_chapter#test_case_groups">Test Case - Groups</seeguide> in the User's Guide.</p> - - </desc> - </func> - - <func> - <name since="">Module:groups() -> GroupDefs</name> - <fsummary>Returns a list of test case group definitions.</fsummary> - <type> - <v>GroupDefs = [Group]</v> - <v>Group = {GroupName,Properties,GroupsAndTestCases}</v> - <v>GroupName = atom()</v> - <v>Properties = [parallel | sequence | Shuffle | {GroupRepeatType,N}]</v> - <v>GroupsAndTestCases = [Group | {group,GroupName} | TestCase | {testcase,TestCase,TCRepeatProps}]</v> - <v>TestCase = atom()</v> - <v>TCRepeatProps = [{repeat,N} | {repeat_until_ok,N} | {repeat_until_fail,N}]</v> - <v>Shuffle = shuffle | {shuffle,Seed}</v> - <v>Seed = {integer(),integer(),integer()}</v> - <v>GroupRepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | repeat_until_any_ok | repeat_until_any_fail</v> - <v>N = integer() | forever</v> - </type> - - <desc> - <p>OPTIONAL</p> - - <p>Defines test case groups. For details, see section - <seeguide marker="write_test_chapter#test_case_groups">Test Case - Groups</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="">Module:suite() -> [Info] </name> - <fsummary>Test suite info function (providing default data - for the suite).</fsummary> - <type> - <v>Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs}</v> - <v>Time = TimeVal | TimeFunc</v> - <v>TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | {hours,integer()}</v> - <v>TimeFunc = {Mod,Func,Args} | Fun</v> - <v>MilliSec = integer()</v> - <v>Mod = atom()</v> - <v>Func = atom()</v> - <v>Args = list()</v> - <v>Fun = fun()</v> - <v>Required = Key | {Key,SubKeys} | {Key,SubKey} | {Key,SubKey,SubKeys}</v> - <v>Key = atom()</v> - <v>SubKeys = SubKey | [SubKey]</v> - <v>SubKey = atom()</v> - <v>Name = atom()</v> - <v>UserData = term()</v> - <v>Conns = [atom()]</v> - <v>CSSFile = string()</v> - <v>CTHs = [CTHModule |</v> - <v> {CTHModule, CTHInitArgs} |</v> - <v> {CTHModule, CTHInitArgs, CTHPriority}]</v> - <v>CTHModule = atom()</v> - <v>CTHInitArgs = term()</v> - </type> - <desc> - - <p>OPTIONAL</p> - - <p>The test suite information function. Returns a list of tagged - tuples specifying various properties related to the execution of - this test suite (common for all test cases in the suite).</p> - - <p>Tag <c>timetrap</c> sets the maximum time that each - test case is allowed to execute (including - <seemfa marker="#Module:init_per_testcase/2"><c>init_per_testcase/2</c></seemfa> - and - <seemfa marker="#Module:end_per_testcase/2"><c>end_per_testcase/2</c></seemfa>). - If the timetrap time is exceeded, the test case fails with reason - <c>timetrap_timeout</c>. A <c>TimeFunc</c> function can be used to - set a new timetrap by returning a <c>TimeVal</c>. It can also be - used to trigger a timetrap time-out by, at some point, returning a - value other than a <c>TimeVal</c>. For details, see section - <seeguide marker="write_test_chapter#timetraps">Timetrap Time-Outs</seeguide> - in the User's Guide.</p> - - <p>Tag <c>require</c> specifies configuration variables - required by test cases (or configuration functions) - in the suite. If the required configuration variables are not found - in any of the configuration files, all test cases are skipped. - For details about the <c>require</c> functionality, see funtion - <seemfa marker="ct#require/1"><c>ct:require/1,2</c></seemfa>.</p> - - <p>With <c>userdata</c>, the user can - specify any test suite-related information, which can be - read by calling - <seemfa marker="ct#userdata/2"><c>ct:userdata/2</c></seemfa>.</p> - - <p>Tag <c>ct_hooks</c> specifies the - <seeguide marker="ct_hooks_chapter">Common Test Hooks</seeguide> - to be run with this suite.</p> - - <p>Other tuples than the ones defined are ignored.</p> - - <p>For details about the test suite information function, see section - <seeguide marker="write_test_chapter#suite">Test - Suite Information Function</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="">Module:init_per_suite(Config) -> NewConfig | {skip,Reason} | - {skip_and_save,Reason,SaveConfig}</name> - <fsummary>Test suite initializations.</fsummary> - <type> - <v>Config = NewConfig = SaveConfig = [{Key,Value}]</v> - <v>Key = atom()</v> - <v>Value = term()</v> - <v>Reason = term()</v> - </type> - <desc> - - <p>OPTIONAL; if this function is defined, then <seemfa - marker="#Module:end_per_suite/1"><c>end_per_suite/1</c></seemfa> - must also be defined.</p> - - <p>This configuration function is called as the first function in the - suite. It typically contains initializations that are common for - all test cases in the suite, and that must only be done - once. Parameter <c>Config</c> is the configuration data - that can be modified. Whatever is returned from this - function is specified as <c>Config</c> to all configuration functions - and test cases in the suite.</p> - - <p>If <c>{skip,Reason}</c> - is returned, all test cases in the suite are skipped - and <c>Reason</c> is printed in the overview log for the suite.</p> - - <p>For information on <c>save_config</c> and <c>skip_and_save</c>, - see section - <seeguide marker="dependencies_chapter#save_config">Saving - Configuration Data</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="">Module:end_per_suite(Config) -> term() | - {save_config,SaveConfig}</name> - <fsummary>Test suite finalization.</fsummary> - <type> - <v>Config = SaveConfig = [{Key,Value}]</v> - <v>Key = atom()</v> - <v>Value = term()</v> - </type> - - <desc> - <p>OPTIONAL; if this function is defined, then <seemfa - marker="#Module:init_per_suite/1"><c>init_per_suite/1</c></seemfa> - must also be defined.</p> - - <p>This function is called as the last test case in the - suite. It is meant to be used for cleaning up after - <seemfa marker="#Module:init_per_suite/1"><c>init_per_suite/1</c></seemfa>.</p> - <p>For information on <c>save_config</c>, see section - <seeguide marker="dependencies_chapter#save_config">Saving - Configuration Data</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="OTP R15B">Module:group(GroupName) -> [Info] </name> - <fsummary>Test case group information function (providing default data - for a test case group, that is, its test cases and - subgroups).</fsummary> - <type> - <v>Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs}</v> - <v>Time = TimeVal | TimeFunc</v> - <v>TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | {hours,integer()}</v> - <v>TimeFunc = {Mod,Func,Args} | Fun</v> - <v>MilliSec = integer()</v> - <v>Mod = atom()</v> - <v>Func = atom()</v> - <v>Args = list()</v> - <v>Fun = fun()</v> - <v>Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}</v> - <v>Key = atom()</v> - <v>SubKeys = SubKey | [SubKey]</v> - <v>SubKey = atom()</v> - <v>Name = atom()</v> - <v>UserData = term()</v> - <v>Conns = [atom()]</v> - <v>CSSFile = string()</v> - <v>CTHs = [CTHModule |</v> - <v> {CTHModule, CTHInitArgs} |</v> - <v> {CTHModule, CTHInitArgs, CTHPriority}]</v> - <v>CTHModule = atom()</v> - <v>CTHInitArgs = term()</v> - </type> - <desc> - - <p>OPTIONAL</p> - - <p>The test case group information function. It is supposed to - return a list of tagged tuples that specify various properties - related to the execution of a test case group (that is, its test - cases and subgroups). Properties set by - <seemfa marker="#Module:group/1"><c>group/1</c></seemfa> override - properties with the same key that have been set previously by - <seemfa marker="#Module:suite/0"><c>suite/0</c></seemfa>.</p> - - <p>Tag <c>timetrap</c> sets the maximum time that each - test case is allowed to execute (including - <seemfa marker="#Module:init_per_testcase/2"><c>init_per_testcase/2</c></seemfa> - and - <seemfa marker="#Module:end_per_testcase/2"><c>end_per_testcase/2</c></seemfa>). - If the timetrap time is - exceeded, the test case fails with reason - <c>timetrap_timeout</c>. A <c>TimeFunc</c> function can be used to - set a new timetrap by returning a <c>TimeVal</c>. It can also be - used to trigger a timetrap time-out by, at some point, returning a - value other than a <c>TimeVal</c>. For details, see section - <seeguide marker="write_test_chapter#timetraps">Timetrap - Time-Outs</seeguide> in the User's Guide.</p> - - <p>Tag <c>require</c> specifies configuration variables - required by test cases (or configuration functions) - in the suite. If the required configuration variables are not found - in any of the configuration files, all test cases in this group are - skipped. For details about the <c>require</c> functionality, see - function - <seemfa marker="ct#require/1"><c>ct:require/1,2</c></seemfa>.</p> - - <p>With <c>userdata</c>, the user can - specify any test case group related information that can be - read by calling - <seemfa marker="ct#userdata/2"><c>ct:userdata/2</c></seemfa>.</p> - - <p>Tag <c>ct_hooks</c> specifies the - <seeguide marker="ct_hooks_chapter">Common Test Hooks</seeguide> - to be run with this suite.</p> - - <p>Other tuples than the ones defined are ignored.</p> - - <p>For details about the test case group information function, - see section <seeguide marker="write_test_chapter#group_info">Group - Information Function</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="">Module:init_per_group(GroupName, Config) -> NewConfig | - {skip,Reason}</name> - <fsummary>Test case group initializations.</fsummary> - <type> - <v>GroupName = atom()</v> - <v>Config = NewConfig = [{Key,Value}]</v> - <v>Key = atom()</v> - <v>Value = term()</v> - <v>Reason = term()</v> - </type> - <desc> - - <p>OPTIONAL; if this function is defined, then <seemfa - marker="#Module:end_per_group/2"><c>end_per_group/2</c></seemfa> - must also be defined.</p> - - <p>This configuration function is called before execution of a - test case group. It typically contains initializations that are - common for all test cases and subgroups in the group, and that - must only be performed once. <c>GroupName</c> is the name of the - group, as specified in the group definition (see - <seemfa marker="#Module:groups/0"><c>groups/0</c></seemfa>). - Parameter <c>Config</c> is the configuration data that can be - modified. - The return value of this function is given as <c>Config</c> - to all test cases and subgroups in the group.</p> - - <p>If <c>{skip,Reason}</c> - is returned, all test cases in the group are skipped and - <c>Reason</c> is printed in the overview log for the group.</p> - - <p>For information about test case groups, see section - <seeguide marker="write_test_chapter#test_case_groups">Test Case - Groups</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="">Module:end_per_group(GroupName, Config) -> term() | - {return_group_result,Status}</name> - <fsummary>Test case group finalization.</fsummary> - <type> - <v>GroupName = atom()</v> - <v>Config = [{Key,Value}]</v> - <v>Key = atom()</v> - <v>Value = term()</v> - <v>Status = ok | skipped | failed</v> - </type> - - <desc> - <p>OPTIONAL; if this function is defined, then <seemfa - marker="#Module:init_per_group/2"><c>init_per_group/2</c></seemfa> - must also be defined.</p> - - <p>This function is called after the execution of a test case group - is finished. It is meant to be used for cleaning up after - <seemfa marker="#Module:init_per_group/2"><c>init_per_group/2</c></seemfa>. - A status value for a nested subgroup can be returned with - <c>{return_group_result,Status}</c>. The status can be retrieved in - <seemfa marker="#Module:end_per_group/2"><c>end_per_group/2</c></seemfa> - for the group on the level above. The status is also used by - <c>Common Test</c> for deciding if execution of a group is to - proceed if property <c>sequence</c> or <c>repeat_until_*</c> - is set.</p> - - <p>For details about test case groups, see section - <seeguide marker="write_test_chapter#test_case_groups">Test Case - Groups</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="">Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail,Reason} | {skip,Reason}</name> - <fsummary>Test case initializations.</fsummary> - <type> - <v> TestCase = atom()</v> - <v> Config = NewConfig = [{Key,Value}]</v> - <v> Key = atom()</v> - <v> Value = term()</v> - <v> Reason = term()</v> - </type> - <desc> - - <p>OPTIONAL; if this function is defined, - then <seemfa marker="#Module:end_per_testcase/2"> - <c>end_per_testcase/2</c></seemfa> must also be - defined.</p> - - <p>This function is called before each test case. Argument - <c>TestCase</c> is the test case name, and - <c>Config</c> (list of key-value tuples) is the configuration - data that can be modified. The <c>NewConfig</c> list returned - from this function is given as <c>Config</c> to the test case. - If <c>{fail,Reason}</c> is returned, the test case is - marked as failed without being executed.</p> - - <p>If <c>{skip,Reason}</c> is returned, the test case is skipped - and <c>Reason</c> is printed in the overview log for the suite.</p> - </desc> - </func> - - <func> - <name since="">Module:end_per_testcase(TestCase, Config) -> term() | {fail,Reason} | {save_config,SaveConfig}</name> - <fsummary>Test case finalization.</fsummary> - <type> - <v>TestCase = atom()</v> - <v>Config = SaveConfig = [{Key,Value}]</v> - <v>Key = atom()</v> - <v>Value = term()</v> - <v>Reason = term()</v> - </type> - <desc> - - <p>OPTIONAL; if this function is defined, - then <seemfa marker="#Module:init_per_testcase/2"> - <c>init_per_testcase/2</c></seemfa> must also be - defined.</p> - - <p>This function is called after each test case, and can be used - to clean up after - <seemfa marker="#Module:init_per_testcase/2"><c>init_per_testcase/2</c></seemfa> - and the test case. Any return value (besides <c>{fail,Reason}</c> - and <c>{save_config,SaveConfig}</c>) is ignored. By returning - <c>{fail,Reason}</c>, <c>TestCase</c> is marked as faulty (even - though it was successful in the sense that it returned - a value instead of terminating).</p> - - <p>For information on <c>save_config</c>, see section - <seeguide marker="dependencies_chapter#save_config">Saving - Configuration Data</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="OTP R14B">Module:Testcase() -> [Info] </name> - <fsummary>Test case information function.</fsummary> - <type> - <v>Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns}</v> - <v>Time = TimeVal | TimeFunc</v> - <v>TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | {hours,integer()}</v> - <v>TimeFunc = {Mod,Func,Args} | Fun</v> - <v>MilliSec = integer()</v> - <v>Mod = atom()</v> - <v>Func = atom()</v> - <v>Args = list()</v> - <v>Fun = fun()</v> - <v>Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}</v> - <v>Key = atom()</v> - <v>SubKeys = SubKey | [SubKey]</v> - <v>SubKey = atom()</v> - <v>Name = atom()</v> - <v>UserData = term()</v> - <v>Conns = [atom()]</v> - </type> - - <desc> - - <p>OPTIONAL</p> - - <p>The test case information function. It is supposed to - return a list of tagged tuples that specify various properties - related to the execution of this particular test case. - Properties set by - <seemfa marker="#Module:Testcase/0"><c>Testcase/0</c></seemfa> - override properties set previously for the test case by - <seemfa marker="#Module:group/1"><c>group/1</c></seemfa> or - <seemfa marker="#Module:suite/0"><c>suite/0</c></seemfa>.</p> - - <p>Tag <c>timetrap</c> sets the maximum time that the - test case is allowed to execute. If the timetrap time is - exceeded, the test case fails with reason <c>timetrap_timeout</c>. - <seemfa marker="#Module:init_per_testcase/2"><c>init_per_testcase/2</c></seemfa> - and - <seemfa marker="#Module:end_per_testcase/2"><c>end_per_testcase/2</c></seemfa> - are included in the timetrap time. - A <c>TimeFunc</c> function can be used to - set a new timetrap by returning a <c>TimeVal</c>. It can also be - used to trigger a timetrap time-out by, at some point, returning a - value other than a <c>TimeVal</c>. For details, see section - <seeguide marker="write_test_chapter#timetraps">Timetrap - Time-Outs</seeguide> in the User's Guide.</p> - - <p>Tag <c>require</c> specifies configuration variables - that are required by the test case (or <c>init_per_testcase/2</c> - or <c>end_per_testcase/2</c>). - If the required configuration variables are not found in any of the - configuration files, the test case is skipped. For details about - the <c>require</c> functionality, see function - <seemfa marker="ct#require/1"><c>ct:require/1,2</c></seemfa>.</p> - - <p>If <c>timetrap</c> or <c>require</c> is not set, the - default values specified by - <seemfa marker="#Module:suite/0"><c>suite/0</c></seemfa> (or - <seemfa marker="#Module:group/1"><c>group/1</c></seemfa>) are used.</p> - - <p>With <c>userdata</c>, the user can specify any test case-related - information that can be read by calling - <seemfa marker="ct#userdata/3"><c>ct:userdata/3</c></seemfa>.</p> - - <p>Other tuples than the ones defined are ignored.</p> - - <p>For details about the test case information function, see section - <seeguide marker="write_test_chapter#info_function">Test - Case Information Function</seeguide> in the User's Guide.</p> - </desc> - </func> - - <func> - <name since="OTP R14B">Module:Testcase(Config) -> term() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() </name> - <fsummary>A test case.</fsummary> - <type> - <v>Config = SaveConfig = [{Key,Value}]</v> - <v>Key = atom()</v> - <v>Value = term()</v> - <v>Reason = term()</v> - <v>Comment = string()</v> - </type> - - <desc> - <p>MANDATORY</p> - - <p>The implementation of a test case. Call the functions to test and - check the result. If something fails, ensure the - function causes a runtime error or call - <seemfa marker="ct#fail/1"><c>ct:fail/1,2</c></seemfa> - (which also causes the test case process to terminate).</p> - - <p>Elements from the <c>Config</c> list can, for example, be read - with <c>proplists:get_value/2</c> in STDLIB - (or the macro <c>?config</c> defined in <c>ct.hrl</c>).</p> - - <p>If you decide not to run the test case after all, return - <c>{skip,Reason}</c>. <c>Reason</c> is then - printed in field <c>Comment</c> on the HTML result page.</p> - - <p>To print some information in field <c>Comment</c> on the HTML - result page, return <c>{comment,Comment}</c>.</p> - - <p>If the function returns anything else, the test case is - considered successful. The return value always gets printed - in the test case log file.</p> - - <p>For details about test case implementation, see section - <seeguide marker="write_test_chapter#test_cases">Test Cases</seeguide> - in the User's Guide.</p> - - <p>For information on <c>save_config</c> and <c>skip_and_save</c>, - see section - <seeguide marker="dependencies_chapter#save_config">Saving - Configuration Data</seeguide> in the User's Guide.</p> - </desc> - </func> - - </funcs> - </erlref> |