summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorDan Davison <ddavison@gitlab.com>2018-08-20 18:13:54 +0000
committerRémy Coutable <remy@rymai.me>2018-08-20 18:13:54 +0000
commitd67936b68f2ba872f5335be1f672f862410c03e5 (patch)
tree3b53966225837a5001d21cc9ef00263d8a10c335
parentc5bf09e7a40cbd04ce4c6733f866e7a00c15c642 (diff)
downloadgitlab-ce-d67936b68f2ba872f5335be1f672f862410c03e5.tar.gz
add initial smoke tests and documentation
-rw-r--r--doc/development/testing_guide/smoke.md16
-rw-r--r--doc/development/testing_guide/testing_levels.md8
-rw-r--r--qa/README.md12
-rw-r--r--qa/qa.rb6
-rw-r--r--qa/qa/scenario/taggable.rb17
-rw-r--r--qa/qa/scenario/template.rb33
-rw-r--r--qa/qa/scenario/test/instance.rb36
-rw-r--r--qa/qa/scenario/test/instance/all.rb15
-rw-r--r--qa/qa/scenario/test/instance/smoke.rb17
-rw-r--r--qa/qa/specs/features/api/basics_spec.rb2
-rw-r--r--qa/qa/specs/features/api/users_spec.rb2
-rw-r--r--qa/qa/specs/features/login/basic_spec.rb15
-rw-r--r--qa/qa/specs/features/login/ldap_spec.rb2
-rw-r--r--qa/qa/specs/features/mattermost/group_create_spec.rb2
-rw-r--r--qa/qa/specs/features/mattermost/login_spec.rb2
-rw-r--r--qa/qa/specs/features/merge_request/create_spec.rb23
-rw-r--r--qa/qa/specs/features/merge_request/rebase_spec.rb2
-rw-r--r--qa/qa/specs/features/merge_request/squash_spec.rb2
-rw-r--r--qa/qa/specs/features/project/activity_spec.rb2
-rw-r--r--qa/qa/specs/features/project/add_deploy_key_spec.rb2
-rw-r--r--qa/qa/specs/features/project/add_secret_variable_spec.rb2
-rw-r--r--qa/qa/specs/features/project/auto_devops_spec.rb2
-rw-r--r--qa/qa/specs/features/project/create_issue_spec.rb2
-rw-r--r--qa/qa/specs/features/project/create_spec.rb2
-rw-r--r--qa/qa/specs/features/project/deploy_key_clone_spec.rb2
-rw-r--r--qa/qa/specs/features/project/fork_project_spec.rb2
-rw-r--r--qa/qa/specs/features/project/import_from_github_spec.rb2
-rw-r--r--qa/qa/specs/features/project/pipelines_spec.rb2
-rw-r--r--qa/qa/specs/features/project/wikis_spec.rb2
-rw-r--r--qa/qa/specs/features/repository/clone_spec.rb2
-rw-r--r--qa/qa/specs/features/repository/protected_branches_spec.rb2
-rw-r--r--qa/qa/specs/features/repository/push_spec.rb2
-rw-r--r--qa/qa/specs/runner.rb8
-rw-r--r--qa/spec/scenario/test/instance/all_spec.rb (renamed from qa/spec/scenario/test/instance_spec.rb)14
-rw-r--r--qa/spec/scenario/test/instance/smoke_spec.rb45
35 files changed, 207 insertions, 100 deletions
diff --git a/doc/development/testing_guide/smoke.md b/doc/development/testing_guide/smoke.md
new file mode 100644
index 00000000000..3f2843cba6e
--- /dev/null
+++ b/doc/development/testing_guide/smoke.md
@@ -0,0 +1,16 @@
+# Smoke Tests
+
+It is imperative in any testing suite that we have Smoke Tests. In short, smoke tests are will run quick sanity
+end-to-end functional tests from GitLab QA and are designed to run against the specified environment to ensure that
+basic functionality is working.
+
+Currently, our suite consists of this basic functionality coverage:
+
+- User Login (Standard Auth)
+- Project Creation
+- Issue Creation
+- Merge Request Creation
+
+---
+
+[Return to Testing documentation](index.md)
diff --git a/doc/development/testing_guide/testing_levels.md b/doc/development/testing_guide/testing_levels.md
index 07ced36f0c1..4403072e96f 100644
--- a/doc/development/testing_guide/testing_levels.md
+++ b/doc/development/testing_guide/testing_levels.md
@@ -120,6 +120,14 @@ running feature tests (i.e. using Capybara) against it.
The actual test scenarios and steps are [part of GitLab Rails] so that they're
always in-sync with the codebase.
+### Smoke tests
+
+Smoke tests are quick tests that may be run at any time (especially after the pre-deployment migrations).
+
+Much like feature tests - these tests run against the UI and ensure that basic functionality is working.
+
+> See [Smoke Tests](smoke.md) for more information.
+
Read a separate document about [end-to-end tests](end_to_end_tests.md) to
learn more.
diff --git a/qa/README.md b/qa/README.md
index be4cf89ebbc..f8a5c00efd4 100644
--- a/qa/README.md
+++ b/qa/README.md
@@ -35,7 +35,7 @@ following call would login to a local [GDK] instance and run all specs in
`qa/specs/features`:
```
-bin/qa Test::Instance http://localhost:3000
+bin/qa Test::Instance::All http://localhost:3000
```
### Writing tests
@@ -48,14 +48,14 @@ You can also supply specific tests to run as another parameter. For example, to
run the repository-related specs, you can execute:
```
-bin/qa Test::Instance http://localhost qa/specs/features/repository/
+bin/qa Test::Instance::All http://localhost qa/specs/features/repository/
```
Since the arguments would be passed to `rspec`, you could use all `rspec`
options there. For example, passing `--backtrace` and also line number:
```
-bin/qa Test::Instance http://localhost qa/specs/features/project/create_spec.rb:3 --backtrace
+bin/qa Test::Instance::All http://localhost qa/specs/features/project/create_spec.rb:3 --backtrace
```
### Overriding the authenticated user
@@ -67,7 +67,7 @@ If you need to authenticate as a different user, you can provide the
`GITLAB_USERNAME` and `GITLAB_PASSWORD` environment variables:
```
-GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password bin/qa Test::Instance https://gitlab.example.com
+GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password bin/qa Test::Instance::All https://gitlab.example.com
```
If your user doesn't have permission to default sandbox group
@@ -75,13 +75,13 @@ If your user doesn't have permission to default sandbox group
`GITLAB_SANDBOX_NAME`:
```
-GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password GITLAB_SANDBOX_NAME=jsmith-qa-sandbox bin/qa Test::Instance https://gitlab.example.com
+GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password GITLAB_SANDBOX_NAME=jsmith-qa-sandbox bin/qa Test::Instance::All https://gitlab.example.com
```
In addition, the `GITLAB_USER_TYPE` can be set to "ldap" to sign in as an LDAP user:
```
-GITLAB_USER_TYPE=ldap GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password GITLAB_SANDBOX_NAME=jsmith-qa-sandbox bin/qa Test::Instance https://gitlab.example.com
+GITLAB_USER_TYPE=ldap GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password GITLAB_SANDBOX_NAME=jsmith-qa-sandbox bin/qa Test::Instance::All https://gitlab.example.com
```
All [supported environment variables are here](https://gitlab.com/gitlab-org/gitlab-qa#supported-environment-variables).
diff --git a/qa/qa.rb b/qa/qa.rb
index 0b48cf58766..4b927067449 100644
--- a/qa/qa.rb
+++ b/qa/qa.rb
@@ -77,14 +77,16 @@ module QA
#
autoload :Bootable, 'qa/scenario/bootable'
autoload :Actable, 'qa/scenario/actable'
- autoload :Taggable, 'qa/scenario/taggable'
autoload :Template, 'qa/scenario/template'
##
# Test scenario entrypoints.
#
module Test
- autoload :Instance, 'qa/scenario/test/instance'
+ module Instance
+ autoload :All, 'qa/scenario/test/instance/all'
+ autoload :Smoke, 'qa/scenario/test/instance/smoke'
+ end
module Integration
autoload :Github, 'qa/scenario/test/integration/github'
diff --git a/qa/qa/scenario/taggable.rb b/qa/qa/scenario/taggable.rb
deleted file mode 100644
index b1f24d742e0..00000000000
--- a/qa/qa/scenario/taggable.rb
+++ /dev/null
@@ -1,17 +0,0 @@
-module QA
- module Scenario
- module Taggable
- # rubocop:disable Gitlab/ModuleWithInstanceVariables
-
- def tags(*tags)
- @tags = tags
- end
-
- def focus
- @tags.to_a
- end
-
- # rubocop:enable Gitlab/ModuleWithInstanceVariables
- end
- end
-end
diff --git a/qa/qa/scenario/template.rb b/qa/qa/scenario/template.rb
index d21a9d52997..765b7d317e0 100644
--- a/qa/qa/scenario/template.rb
+++ b/qa/qa/scenario/template.rb
@@ -1,15 +1,36 @@
module QA
module Scenario
class Template
- def self.perform(*args)
- new.tap do |scenario|
- yield scenario if block_given?
- break scenario.perform(*args)
+ class << self
+ def perform(*args)
+ new.tap do |scenario|
+ yield scenario if block_given?
+ break scenario.perform(*args)
+ end
+ end
+
+ def tags(*tags)
+ @tags = tags
+ end
+
+ def focus
+ @tags.to_a
end
end
- def perform(*_args)
- raise NotImplementedError
+ def perform(address, *rspec_options)
+ Runtime::Scenario.define(:gitlab_address, address)
+
+ Specs::Runner.perform do |specs|
+ specs.tty = true
+ specs.tags = self.class.focus
+ specs.options =
+ if rspec_options.any?
+ rspec_options
+ else
+ File.expand_path('../../specs/features', __dir__)
+ end
+ end
end
end
end
diff --git a/qa/qa/scenario/test/instance.rb b/qa/qa/scenario/test/instance.rb
deleted file mode 100644
index 46eb2dabb11..00000000000
--- a/qa/qa/scenario/test/instance.rb
+++ /dev/null
@@ -1,36 +0,0 @@
-module QA
- module Scenario
- module Test
- ##
- # Base class for running the suite against any GitLab instance,
- # including staging and on-premises installation.
- #
- class Instance < Template
- include Bootable
- extend Taggable
-
- tags :core
-
- def perform(address, *rspec_options)
- Runtime::Scenario.define(:gitlab_address, address)
-
- ##
- # Perform before hooks, which are different for CE and EE
- #
- Runtime::Release.perform_before_hooks
-
- Specs::Runner.perform do |specs|
- specs.tty = true
- specs.tags = self.class.focus
- specs.options =
- if rspec_options.any?
- rspec_options
- else
- ::File.expand_path('../../specs/features', __dir__)
- end
- end
- end
- end
- end
- end
-end
diff --git a/qa/qa/scenario/test/instance/all.rb b/qa/qa/scenario/test/instance/all.rb
new file mode 100644
index 00000000000..a07c26431bd
--- /dev/null
+++ b/qa/qa/scenario/test/instance/all.rb
@@ -0,0 +1,15 @@
+module QA
+ module Scenario
+ module Test
+ ##
+ # Base class for running the suite against any GitLab instance,
+ # including staging and on-premises installation.
+ #
+ module Instance
+ class All < Template
+ include Bootable
+ end
+ end
+ end
+ end
+end
diff --git a/qa/qa/scenario/test/instance/smoke.rb b/qa/qa/scenario/test/instance/smoke.rb
new file mode 100644
index 00000000000..a7d2cb27f27
--- /dev/null
+++ b/qa/qa/scenario/test/instance/smoke.rb
@@ -0,0 +1,17 @@
+module QA
+ module Scenario
+ module Test
+ module Instance
+ ##
+ # Base class for running the suite against any GitLab instance,
+ # including staging and on-premises installation.
+ #
+ class Smoke < Template
+ include Bootable
+
+ tags :smoke
+ end
+ end
+ end
+ end
+end
diff --git a/qa/qa/specs/features/api/basics_spec.rb b/qa/qa/specs/features/api/basics_spec.rb
index 6563b56d1b4..7defb2ea93e 100644
--- a/qa/qa/specs/features/api/basics_spec.rb
+++ b/qa/qa/specs/features/api/basics_spec.rb
@@ -1,7 +1,7 @@
require 'securerandom'
module QA
- describe 'API basics', :core do
+ describe 'API basics' do
before(:context) do
@api_client = Runtime::API::Client.new(:gitlab)
end
diff --git a/qa/qa/specs/features/api/users_spec.rb b/qa/qa/specs/features/api/users_spec.rb
index 8a63d8095c9..2a9360734bb 100644
--- a/qa/qa/specs/features/api/users_spec.rb
+++ b/qa/qa/specs/features/api/users_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'API users', :core do
+ describe 'API users' do
before(:context) do
@api_client = Runtime::API::Client.new(:gitlab)
end
diff --git a/qa/qa/specs/features/login/basic_spec.rb b/qa/qa/specs/features/login/basic_spec.rb
new file mode 100644
index 00000000000..f866466c7bf
--- /dev/null
+++ b/qa/qa/specs/features/login/basic_spec.rb
@@ -0,0 +1,15 @@
+module QA
+ describe 'basic user login', :smoke do
+ it 'user logs in using basic credentials' do
+ Runtime::Browser.visit(:gitlab, Page::Main::Login)
+ Page::Main::Login.act { sign_in_using_credentials }
+
+ # TODO, since `Signed in successfully` message was removed
+ # this is the only way to tell if user is signed in correctly.
+ #
+ Page::Menu::Main.perform do |menu|
+ expect(menu).to have_personal_area
+ end
+ end
+ end
+end
diff --git a/qa/qa/specs/features/login/ldap_spec.rb b/qa/qa/specs/features/login/ldap_spec.rb
index b7a284c584b..de6111eea64 100644
--- a/qa/qa/specs/features/login/ldap_spec.rb
+++ b/qa/qa/specs/features/login/ldap_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'LDAP user login', :ldap do
+ describe 'LDAP user login', :orchestrated, :ldap do
before do
Runtime::Env.user_type = 'ldap'
end
diff --git a/qa/qa/specs/features/mattermost/group_create_spec.rb b/qa/qa/specs/features/mattermost/group_create_spec.rb
index a59761da341..097e1713aef 100644
--- a/qa/qa/specs/features/mattermost/group_create_spec.rb
+++ b/qa/qa/specs/features/mattermost/group_create_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'create a new group', :mattermost do
+ describe 'create a new group', :orchestrated, :mattermost do
it 'creating a group with a mattermost team' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/mattermost/login_spec.rb b/qa/qa/specs/features/mattermost/login_spec.rb
index b140191e160..27f7d4c245f 100644
--- a/qa/qa/specs/features/mattermost/login_spec.rb
+++ b/qa/qa/specs/features/mattermost/login_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'logging in to Mattermost', :mattermost do
+ describe 'logging in to Mattermost', :orchestrated, :mattermost do
it 'can use gitlab oauth' do
Runtime::Browser.visit(:gitlab, Page::Main::Login) do
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/merge_request/create_spec.rb b/qa/qa/specs/features/merge_request/create_spec.rb
index 36d7efb02e1..71e79956b85 100644
--- a/qa/qa/specs/features/merge_request/create_spec.rb
+++ b/qa/qa/specs/features/merge_request/create_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'creates a merge request', :core do
+ describe 'creates a merge request with milestone' do
it 'user creates a new merge request' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
@@ -29,4 +29,25 @@ module QA
end
end
end
+
+ describe 'creates a merge request', :smoke do
+ it 'user creates a new merge request' do
+ Runtime::Browser.visit(:gitlab, Page::Main::Login)
+ Page::Main::Login.act { sign_in_using_credentials }
+
+ current_project = Factory::Resource::Project.fabricate! do |project|
+ project.name = 'project-with-merge-request'
+ end
+
+ Factory::Resource::MergeRequest.fabricate! do |merge_request|
+ merge_request.title = 'This is a merge request'
+ merge_request.description = 'Great feature'
+ merge_request.project = current_project
+ end
+
+ expect(page).to have_content('This is a merge request')
+ expect(page).to have_content('Great feature')
+ expect(page).to have_content(/Opened [\w\s]+ ago/)
+ end
+ end
end
diff --git a/qa/qa/specs/features/merge_request/rebase_spec.rb b/qa/qa/specs/features/merge_request/rebase_spec.rb
index 163dcbe7963..c36d28e4237 100644
--- a/qa/qa/specs/features/merge_request/rebase_spec.rb
+++ b/qa/qa/specs/features/merge_request/rebase_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'merge request rebase', :core do
+ describe 'merge request rebase' do
it 'rebases source branch of merge request' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/merge_request/squash_spec.rb b/qa/qa/specs/features/merge_request/squash_spec.rb
index 4856bbe1a69..3ecc36a5ae1 100644
--- a/qa/qa/specs/features/merge_request/squash_spec.rb
+++ b/qa/qa/specs/features/merge_request/squash_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'merge request squash commits', :core do
+ describe 'merge request squash commits' do
it 'when squash commits is marked before merge' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/project/activity_spec.rb b/qa/qa/specs/features/project/activity_spec.rb
index 02074e386b6..c7ce8dfdcc6 100644
--- a/qa/qa/specs/features/project/activity_spec.rb
+++ b/qa/qa/specs/features/project/activity_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'activity page', :core do
+ describe 'activity page' do
it 'push creates an event in the activity page' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/project/add_deploy_key_spec.rb b/qa/qa/specs/features/project/add_deploy_key_spec.rb
index 14642af97ad..24f9f4c77f8 100644
--- a/qa/qa/specs/features/project/add_deploy_key_spec.rb
+++ b/qa/qa/specs/features/project/add_deploy_key_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'deploy keys support', :core do
+ describe 'deploy keys support' do
it 'user adds a deploy key' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/project/add_secret_variable_spec.rb b/qa/qa/specs/features/project/add_secret_variable_spec.rb
index 32c91dd9d4d..04d9fe488e2 100644
--- a/qa/qa/specs/features/project/add_secret_variable_spec.rb
+++ b/qa/qa/specs/features/project/add_secret_variable_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'secret variables support', :core do
+ describe 'secret variables support' do
it 'user adds a secret variable' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/project/auto_devops_spec.rb b/qa/qa/specs/features/project/auto_devops_spec.rb
index b3e4889abc0..248669b6046 100644
--- a/qa/qa/specs/features/project/auto_devops_spec.rb
+++ b/qa/qa/specs/features/project/auto_devops_spec.rb
@@ -1,7 +1,7 @@
require 'pathname'
module QA
- describe 'Auto Devops', :kubernetes do
+ describe 'Auto Devops', :orchestrated, :kubernetes do
after do
@cluster&.remove!
end
diff --git a/qa/qa/specs/features/project/create_issue_spec.rb b/qa/qa/specs/features/project/create_issue_spec.rb
index ac2ed2ca2a1..793e7db87cb 100644
--- a/qa/qa/specs/features/project/create_issue_spec.rb
+++ b/qa/qa/specs/features/project/create_issue_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'creates issue', :core do
+ describe 'creates issue', :smoke do
let(:issue_title) { 'issue title' }
it 'user creates issue' do
diff --git a/qa/qa/specs/features/project/create_spec.rb b/qa/qa/specs/features/project/create_spec.rb
index 14ecd464685..5e19e490778 100644
--- a/qa/qa/specs/features/project/create_spec.rb
+++ b/qa/qa/specs/features/project/create_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'create a new project', :core do
+ describe 'create a new project', :smoke do
it 'user creates a new project' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/project/deploy_key_clone_spec.rb b/qa/qa/specs/features/project/deploy_key_clone_spec.rb
index 054f49b408a..1d099508c24 100644
--- a/qa/qa/specs/features/project/deploy_key_clone_spec.rb
+++ b/qa/qa/specs/features/project/deploy_key_clone_spec.rb
@@ -1,7 +1,7 @@
require 'digest/sha1'
module QA
- describe 'cloning code using a deploy key', :core, :docker do
+ describe 'cloning code using a deploy key', :docker do
def login
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/project/fork_project_spec.rb b/qa/qa/specs/features/project/fork_project_spec.rb
index 8ad0120305a..d3534d736e4 100644
--- a/qa/qa/specs/features/project/fork_project_spec.rb
+++ b/qa/qa/specs/features/project/fork_project_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'Project fork', :core do
+ describe 'Project fork' do
it 'can submit merge requests to upstream master' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/project/import_from_github_spec.rb b/qa/qa/specs/features/project/import_from_github_spec.rb
index 221b5c27fba..57695d2c726 100644
--- a/qa/qa/specs/features/project/import_from_github_spec.rb
+++ b/qa/qa/specs/features/project/import_from_github_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'user imports a GitHub repo', :core, :github do
+ describe 'user imports a GitHub repo', :orchestrated, :github do
let(:imported_project) do
Factory::Resource::ProjectImportedFromGithub.fabricate! do |project|
project.name = 'imported-project'
diff --git a/qa/qa/specs/features/project/pipelines_spec.rb b/qa/qa/specs/features/project/pipelines_spec.rb
index ddedde7a8bc..6c6b4e80626 100644
--- a/qa/qa/specs/features/project/pipelines_spec.rb
+++ b/qa/qa/specs/features/project/pipelines_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'CI/CD Pipelines', :core, :docker do
+ describe 'CI/CD Pipelines', :orchestrated, :docker do
let(:executor) { "qa-runner-#{Time.now.to_i}" }
after do
diff --git a/qa/qa/specs/features/project/wikis_spec.rb b/qa/qa/specs/features/project/wikis_spec.rb
index 59cc455fffc..9af2dbd1264 100644
--- a/qa/qa/specs/features/project/wikis_spec.rb
+++ b/qa/qa/specs/features/project/wikis_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'Wiki Functionality', :core do
+ describe 'Wiki Functionality' do
def login
Runtime::Browser.visit(:gitlab, Page::Main::Login)
Page::Main::Login.act { sign_in_using_credentials }
diff --git a/qa/qa/specs/features/repository/clone_spec.rb b/qa/qa/specs/features/repository/clone_spec.rb
index a04ce4e44d9..8b0613c5f78 100644
--- a/qa/qa/specs/features/repository/clone_spec.rb
+++ b/qa/qa/specs/features/repository/clone_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'clone code from the repository', :core do
+ describe 'clone code from the repository' do
context 'with regular account over http' do
let(:location) do
Page::Project::Show.act do
diff --git a/qa/qa/specs/features/repository/protected_branches_spec.rb b/qa/qa/specs/features/repository/protected_branches_spec.rb
index c2de94516d9..aa23145478d 100644
--- a/qa/qa/specs/features/repository/protected_branches_spec.rb
+++ b/qa/qa/specs/features/repository/protected_branches_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'branch protection support', :core do
+ describe 'branch protection support' do
let(:branch_name) { 'protected-branch' }
let(:commit_message) { 'Protected push commit message' }
let(:project) do
diff --git a/qa/qa/specs/features/repository/push_spec.rb b/qa/qa/specs/features/repository/push_spec.rb
index fc40b60d915..1e89942e932 100644
--- a/qa/qa/specs/features/repository/push_spec.rb
+++ b/qa/qa/specs/features/repository/push_spec.rb
@@ -1,5 +1,5 @@
module QA
- describe 'push code to repository', :core do
+ describe 'push code to repository' do
context 'with regular account over http' do
it 'user pushes code to the repository' do
Runtime::Browser.visit(:gitlab, Page::Main::Login)
diff --git a/qa/qa/specs/runner.rb b/qa/qa/specs/runner.rb
index f8f6fe65599..ccb9d5591de 100644
--- a/qa/qa/specs/runner.rb
+++ b/qa/qa/specs/runner.rb
@@ -14,7 +14,13 @@ module QA
def perform
args = []
args.push('--tty') if tty
- tags.to_a.each { |tag| args.push(['-t', tag.to_s]) }
+
+ if tags.any?
+ tags.each { |tag| args.push(['-t', tag.to_s]) }
+ else
+ args.push(%w[-t ~orchestrated])
+ end
+
args.push(options)
Runtime::Browser.configure!
diff --git a/qa/spec/scenario/test/instance_spec.rb b/qa/spec/scenario/test/instance/all_spec.rb
index 0d0b534911f..bc0b21c6494 100644
--- a/qa/spec/scenario/test/instance_spec.rb
+++ b/qa/spec/scenario/test/instance/all_spec.rb
@@ -1,10 +1,4 @@
-describe QA::Scenario::Test::Instance do
- subject do
- Class.new(described_class) do
- tags :rspec
- end
- end
-
+describe QA::Scenario::Test::Instance::All do
context '#perform' do
let(:arguments) { spy('Runtime::Scenario') }
let(:release) { spy('Runtime::Release') }
@@ -26,16 +20,16 @@ describe QA::Scenario::Test::Instance do
end
context 'no paths' do
- it 'should call runner with default arguments' do
+ it 'calls runner with default arguments' do
subject.perform("test")
expect(runner).to have_received(:options=)
- .with(::File.expand_path('../../../qa/specs/features', __dir__))
+ .with(::File.expand_path('../../../../../qa/specs/features', __dir__))
end
end
context 'specifying paths' do
- it 'should call runner with paths' do
+ it 'calls runner with paths' do
subject.perform('test', 'path1', 'path2')
expect(runner).to have_received(:options=).with(%w[path1 path2])
diff --git a/qa/spec/scenario/test/instance/smoke_spec.rb b/qa/spec/scenario/test/instance/smoke_spec.rb
new file mode 100644
index 00000000000..66d71610341
--- /dev/null
+++ b/qa/spec/scenario/test/instance/smoke_spec.rb
@@ -0,0 +1,45 @@
+describe QA::Scenario::Test::Instance::Smoke do
+ subject { Class.new(described_class) { tags :smoke } }
+
+ context '#perform' do
+ let(:arguments) { spy('Runtime::Scenario') }
+ let(:release) { spy('Runtime::Release') }
+ let(:runner) { spy('Specs::Runner') }
+
+ before do
+ stub_const('QA::Runtime::Release', release)
+ stub_const('QA::Runtime::Scenario', arguments)
+ stub_const('QA::Specs::Runner', runner)
+
+ allow(runner).to receive(:perform).and_yield(runner)
+ end
+
+ it 'sets an address of the subject' do
+ subject.perform("hello")
+
+ expect(arguments).to have_received(:define)
+ .with(:gitlab_address, "hello")
+ end
+
+ it 'has a smoke tag' do
+ expect(subject.focus).to eq([:smoke]) # rubocop:disable Focus
+ end
+
+ context 'no paths' do
+ it 'calls runner with default arguments' do
+ subject.perform("test")
+
+ expect(runner).to have_received(:options=)
+ .with(File.expand_path('../../../../../qa/specs/features', __dir__))
+ end
+ end
+
+ context 'specifying paths' do
+ it 'calls runner with paths' do
+ subject.perform('test', 'path1', 'path2')
+
+ expect(runner).to have_received(:options=).with(%w[path1 path2])
+ end
+ end
+ end
+end