From 4d9c76e9e44119ee2578c94b2b98bb72e80e874b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9my=20Coutable?= Date: Thu, 18 Apr 2019 22:08:04 +0200 Subject: Reorganize test jobs by level MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Rémy Coutable --- doc/development/testing_guide/testing_levels.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) (limited to 'doc/development/testing_guide/testing_levels.md') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 1fa6e38ea5a..b5155b6b7fa 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -4,12 +4,14 @@ _This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._ -As of 2019-04-16, we have the following distribution of tests per level: +As of 2019-05-01, we have the following distribution of tests per level: -- 67 black-box tests at the system level (aka end-to-end or QA tests) in CE, 98 in EE. This represents 0.3% of all the CE tests (0.3% in EE). -- 5,457 white-box tests at the system level (aka system or feature tests) in CE, 6,585 in EE. This represents 24.6% of all the CE tests (20.3% in EE). -- 8,298 integration tests in CE, 10,633 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.2% of all the CE tests (32.8% in EE). -- 8,403 unit tests in CE, 15,090 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.8% of all the CE tests (46.6% in EE). +| Test level | Community Edition | Enterprise Edition | Community + Enterprise Edition | +| --------- | ---------- | -------------- | ----- | +| Black-box tests at the system level (aka end-to-end or QA tests) | 68 (0.14%) | 31 (0.2%) | 99 (0.17%) | +| White-box tests at the system level (aka system or feature tests) | 5,471 (11.9%) | 969 (7.4%) | 6440 (10.9%) | +| Integration tests | 8,333 (18.2%) | 2,244 (17.2%) | 10,577 (17.9%) | +| Unit tests | 32,031 (69.7%) | 9,778 (75.1%) | 41,809 (71%) | ## Unit tests -- cgit v1.2.1 From 4f20c9de6f8e09131d751174243214d9bd3b6419 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Kamil=20Trzci=C5=84ski?= Date: Wed, 22 May 2019 11:45:46 +0000 Subject: Revert "Merge branch 'reorganize-tests-jobs-by-level' into 'master'" This reverts merge request !27514 --- doc/development/testing_guide/testing_levels.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) (limited to 'doc/development/testing_guide/testing_levels.md') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index b5155b6b7fa..1fa6e38ea5a 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -4,14 +4,12 @@ _This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._ -As of 2019-05-01, we have the following distribution of tests per level: +As of 2019-04-16, we have the following distribution of tests per level: -| Test level | Community Edition | Enterprise Edition | Community + Enterprise Edition | -| --------- | ---------- | -------------- | ----- | -| Black-box tests at the system level (aka end-to-end or QA tests) | 68 (0.14%) | 31 (0.2%) | 99 (0.17%) | -| White-box tests at the system level (aka system or feature tests) | 5,471 (11.9%) | 969 (7.4%) | 6440 (10.9%) | -| Integration tests | 8,333 (18.2%) | 2,244 (17.2%) | 10,577 (17.9%) | -| Unit tests | 32,031 (69.7%) | 9,778 (75.1%) | 41,809 (71%) | +- 67 black-box tests at the system level (aka end-to-end or QA tests) in CE, 98 in EE. This represents 0.3% of all the CE tests (0.3% in EE). +- 5,457 white-box tests at the system level (aka system or feature tests) in CE, 6,585 in EE. This represents 24.6% of all the CE tests (20.3% in EE). +- 8,298 integration tests in CE, 10,633 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.2% of all the CE tests (32.8% in EE). +- 8,403 unit tests in CE, 15,090 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.8% of all the CE tests (46.6% in EE). ## Unit tests -- cgit v1.2.1 From 6145ddf515f93fcc7ed873b911b6369cc2bc0462 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9my=20Coutable?= Date: Wed, 22 May 2019 14:16:49 +0200 Subject: Revert "Merge branch 'revert-04c3c6dd' into 'master'" This reverts commit 744f1f2e7037f5c70c3168d9e2e89b1c327465d2, reversing changes made to c4d930e5f54e7da07c80cc028dfc0f5c08719146. --- doc/development/testing_guide/testing_levels.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) (limited to 'doc/development/testing_guide/testing_levels.md') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index 1fa6e38ea5a..b5155b6b7fa 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -4,12 +4,14 @@ _This diagram demonstrates the relative priority of each test type we use. `e2e` stands for end-to-end._ -As of 2019-04-16, we have the following distribution of tests per level: +As of 2019-05-01, we have the following distribution of tests per level: -- 67 black-box tests at the system level (aka end-to-end or QA tests) in CE, 98 in EE. This represents 0.3% of all the CE tests (0.3% in EE). -- 5,457 white-box tests at the system level (aka system or feature tests) in CE, 6,585 in EE. This represents 24.6% of all the CE tests (20.3% in EE). -- 8,298 integration tests in CE, 10,633 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.2% of all the CE tests (32.8% in EE). -- 8,403 unit tests in CE, 15,090 in EE: 0.3% of all the CE tests (0.3% in EE). This represents 37.8% of all the CE tests (46.6% in EE). +| Test level | Community Edition | Enterprise Edition | Community + Enterprise Edition | +| --------- | ---------- | -------------- | ----- | +| Black-box tests at the system level (aka end-to-end or QA tests) | 68 (0.14%) | 31 (0.2%) | 99 (0.17%) | +| White-box tests at the system level (aka system or feature tests) | 5,471 (11.9%) | 969 (7.4%) | 6440 (10.9%) | +| Integration tests | 8,333 (18.2%) | 2,244 (17.2%) | 10,577 (17.9%) | +| Unit tests | 32,031 (69.7%) | 9,778 (75.1%) | 41,809 (71%) | ## Unit tests -- cgit v1.2.1 From eb066f649b197634def3a5edac1fe3feaff8e1a9 Mon Sep 17 00:00:00 2001 From: ddavison Date: Tue, 21 May 2019 18:34:12 -0700 Subject: Consolidate documentation written for e2e tests under dev docs Previously the documentation was separated. If we want to bake quality into the product, how better than to include everything we use directly in the development documentation Signed-off-by: ddavison Fix broken internal doc link Move documentation for page objects to the e2e section --- doc/development/testing_guide/testing_levels.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc/development/testing_guide/testing_levels.md') diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md index b5155b6b7fa..e1ce4d3b7d1 100644 --- a/doc/development/testing_guide/testing_levels.md +++ b/doc/development/testing_guide/testing_levels.md @@ -178,7 +178,7 @@ Every new feature should come with a [test plan]. | ---------- | -------------- | ----- | | `qa/qa/specs/features/` | [Capybara] + [RSpec] + Custom QA framework | Tests should be placed under their corresponding [Product category] | -> See [end-to-end tests](end_to_end_tests.md) for more information. +> See [end-to-end tests](end_to_end/index.md) for more information. Note that `qa/spec` contains unit tests of the QA framework itself, not to be confused with the application's [unit tests](#unit-tests) or -- cgit v1.2.1