diff options
86 files changed, 426 insertions, 665 deletions
diff --git a/HACKING.rst b/HACKING.rst index 81a7c2ca5..04b5eb6d5 100644 --- a/HACKING.rst +++ b/HACKING.rst @@ -312,3 +312,57 @@ example of this would be:: * Boot an additional instance from the new snapshot based volume * Check written content in the instance booted from snapshot """ + +Branchless Tempest Considerations +--------------------------------- + +Starting with the OpenStack Icehouse release Tempest no longer has any stable +branches. This is to better ensure API consistency between releases because +the API behavior should not change between releases. This means that the stable +branches are also gated by the Tempest master branch, which also means that +proposed commits to Tempest must work against both the master and all the +currently supported stable branches of the projects. As such there are a few +special considerations that have to be accounted for when pushing new changes +to tempest. + +1. New Tests for new features +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When adding tests for new features that were not in previous releases of the +projects the new test has to be properly skipped with a feature flag. Whether +this is just as simple as using the @test.requires_ext() decorator to check +if the required extension (or discoverable optional API) is enabled or adding +a new config option to the appropriate section. If there isn't a method of +selecting the new **feature** from the config file then there won't be a +mechanism to disable the test with older stable releases and the new test won't +be able to merge. + +2. Bug fix on core project needing Tempest changes +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When trying to land a bug fix which changes a tested API you'll have to use the +following procedure:: + + - Propose change to the project, get a +2 on the change even with failing + - Propose skip on Tempest which will only be approved after the + corresponding change in the project has a +2 on change + - Land project change in master and all open stable branches (if required) + - Land changed test in Tempest + +Otherwise the bug fix won't be able to land in the project. + +3. New Tests for existing features +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If a test is being added for a feature that exists in all the current releases +of the projects then the only concern is that the API behavior is the same +across all the versions of the project being tested. If the behavior is not +consistent the test will not be able to merge. + +API Stability +------------- + +For new tests being added to Tempest the assumption is that the API being +tested is considered stable and adheres to the OpenStack API stability +guidelines. If an API is still considered experimental or in development then +it should not be tested by Tempest until it is considered stable. diff --git a/README.rst b/README.rst index 7af0025da..9aaea240c 100644 --- a/README.rst +++ b/README.rst @@ -59,50 +59,49 @@ and reference data to be used in testing. will have a configuration file already set up to work with your devstack installation. -Tempest is not tied to any single test runner, but testr is the most commonly -used tool. After setting up your configuration file, you can execute -the set of Tempest tests by using ``testr`` :: +Tempest is not tied to any single test runner, but `testr`_ is the most commonly +used tool. Also, the nosetests test runner is **not** recommended to run tempest. + +After setting up your configuration file, you can execute the set of Tempest +tests by using ``testr`` :: $> testr run --parallel -To run one single test :: +.. _testr: http://testrepository.readthedocs.org/en/latest/MANUAL.html + +To run one single test serially :: - $> testr run --parallel tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server + $> testr run tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server Alternatively, you can use the run_tempest.sh script which will create a venv -and run the tests or use tox to do the same. +and run the tests or use tox to do the same. Tox also contains several existing +job configurations. For example:: + + $> tox -efull + +which will run the same set of tests as the OpenStack gate. (it's exactly how +the gate invokes tempest) Or:: + + $> tox -esmoke + +to run the tests tagged as smoke. + Configuration ------------- Detailed configuration of tempest is beyond the scope of this -document. The etc/tempest.conf.sample attempts to be a self -documenting version of the configuration. +document see :ref:`tempest-configuration` for more details on configuring +tempest. The etc/tempest.conf.sample attempts to be a self documenting version +of the configuration. -To generate the sample tempest.conf file, run the following +You can generate a new sample tempest.conf file, run the following command from the top level of the tempest directory: tox -egenconfig The most important pieces that are needed are the user ids, openstack -endpoints, and basic flavors and images needed to run tests. - -Common Issues -------------- - -Tempest was originally designed to primarily run against a full OpenStack -deployment. Due to that focus, some issues may occur when running Tempest -against devstack. - -Running Tempest, especially in parallel, against a devstack instance may -cause requests to be rate limited, which will cause unexpected failures. -Given the number of requests Tempest can make against a cluster, rate limiting -should be disabled for all test accounts. - -Additionally, devstack only provides a single image which Nova can use. -For the moment, the best solution is to provide the same image uuid for -both image_ref and image_ref_alt. Tempest will skip tests as needed if it -detects that both images are the same. +endpoint, and basic flavors and images needed to run tests. Unit Tests ---------- @@ -132,57 +131,3 @@ present in python 2.6 will be used. If you're running you're OpenStack services on an earlier release with python 2.6 you can easily run tempest against it from a remote system running python 2.7. (or deploy a cloud guest in your cloud that has python 2.7) - -Branchless Tempest Considerations ---------------------------------- - -Starting with the OpenStack Icehouse release Tempest no longer has any stable -branches. This is to better ensure API consistency between releases because -the API behavior should not change between releases. This means that the stable -branches are also gated by the Tempest master branch, which also means that -proposed commits to Tempest must work against both the master and all the -currently supported stable branches of the projects. As such there are a few -special considerations that have to be accounted for when pushing new changes -to tempest. - -1. New Tests for new features -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When adding tests for new features that were not in previous releases of the -projects the new test has to be properly skipped with a feature flag. Whether -this is just as simple as using the @test.requires_ext() decorator to check -if the required extension (or discoverable optional API) is enabled or adding -a new config option to the appropriate section. If there isn't a method of -selecting the new **feature** from the config file then there won't be a -mechanism to disable the test with older stable releases and the new test won't -be able to merge. - -2. Bug fix on core project needing Tempest changes -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -When trying to land a bug fix which changes a tested API you'll have to use the -following procedure:: - - - Propose change to the project, get a +2 on the change even with failing - - Propose skip on Tempest which will only be approved after the - corresponding change in the project has a +2 on change - - Land project change in master and all open stable branches (if required) - - Land changed test in Tempest - -Otherwise the bug fix won't be able to land in the project. - -3. New Tests for existing features -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -If a test is being added for a feature that exists in all the current releases -of the projects then the only concern is that the API behavior is the same -across all the versions of the project being tested. If the behavior is not -consistent the test will not be able to merge. - -API Stability -------------- - -For new tests being added to Tempest the assumption is that the API being -tested is considered stable and adheres to the OpenStack API stability -guidelines. If an API is still considered experimental or in development then -it should not be tested by Tempest until it is considered stable. diff --git a/doc/source/configuration.rst b/doc/source/configuration.rst index f772aa3ad..15369dedc 100644 --- a/doc/source/configuration.rst +++ b/doc/source/configuration.rst @@ -1,6 +1,23 @@ +.. _tempest-configuration: + Tempest Configuration Guide =========================== +This guide is a starting point for configuring tempest. It aims to elaborate +on and explain some of the mandatory and common configuration settings and how +they are used in conjunction. The source of truth on each option is the sample +config file which explains the purpose of each individual option. + +Lock Path +--------- + +There are some tests and operations inside of tempest that need to be +externally locked when running in parallel to prevent them from running at +the same time. This is a mandatory step for configuring tempest and is still +needed even when running serially. All that is needed to do this is: + + #. Set the lock_path option in the oslo_concurrency group + Auth/Credentials ---------------- diff --git a/etc/tempest.conf.sample b/etc/tempest.conf.sample index 024648880..80d52a440 100644 --- a/etc/tempest.conf.sample +++ b/etc/tempest.conf.sample @@ -422,6 +422,11 @@ # Does the test environment have the ec2 api running? (boolean value) #ec2_api = true +# Does Nova preserve preexisting ports from Neutron when deleting an +# instance? This should be set to True if testing Kilo+ Nova. (boolean +# value) +#preserve_ports = false + [dashboard] @@ -873,6 +878,9 @@ # Allowed values: public, admin, internal, publicURL, adminURL, internalURL #endpoint_type = publicURL +# Role required for users to be able to manage stacks (string value) +#stack_owner_role = heat_stack_owner + # Time in seconds between build status checks. (integer value) #build_interval = 1 diff --git a/requirements.txt b/requirements.txt index 17c296801..3ab5b3f28 100644 --- a/requirements.txt +++ b/requirements.txt @@ -11,8 +11,6 @@ paramiko>=1.13.0 netaddr>=0.7.12 python-ceilometerclient>=1.0.6 python-glanceclient>=0.15.0 -python-keystoneclient>=1.1.0 -python-neutronclient>=2.3.11,<3 python-cinderclient>=1.1.0 python-heatclient>=0.3.0 python-saharaclient>=0.7.6 diff --git a/tempest/api/baremetal/admin/base.py b/tempest/api/baremetal/admin/base.py index 2834b2bb5..cf2484da0 100644 --- a/tempest/api/baremetal/admin/base.py +++ b/tempest/api/baremetal/admin/base.py @@ -16,6 +16,7 @@ from tempest_lib.common.utils import data_utils from tempest_lib import exceptions as lib_exc from tempest import clients +from tempest.common import credentials from tempest import config from tempest import test @@ -69,7 +70,11 @@ class BaseBaremetalTest(test.BaseTestCase): @classmethod def setup_credentials(cls): super(BaseBaremetalTest, cls).setup_credentials() - cls.mgr = clients.AdminManager() + if (not hasattr(cls, 'isolated_creds') or + not cls.isolated_creds.name == cls.__name__): + cls.isolated_creds = credentials.get_isolated_credentials( + name=cls.__name__, network_resources=cls.network_resources) + cls.mgr = clients.Manager(cls.isolated_creds.get_admin_creds()) @classmethod def setup_clients(cls): diff --git a/tempest/api/compute/admin/test_baremetal_nodes.py b/tempest/api/compute/admin/test_baremetal_nodes.py index 1381f805d..64099c391 100644 --- a/tempest/api/compute/admin/test_baremetal_nodes.py +++ b/tempest/api/compute/admin/test_baremetal_nodes.py @@ -31,14 +31,26 @@ class BaremetalNodesAdminTestJSON(base.BaseV2ComputeAdminTest): skip_msg = ('%s skipped as Ironic is not available' % cls.__name__) raise cls.skipException(skip_msg) cls.client = cls.os_adm.baremetal_nodes_client + cls.ironic_client = cls.os_adm.baremetal_client - @test.attr(type='smoke') + @test.attr(type=['smoke', 'baremetal']) @test.idempotent_id('e475aa6e-416d-4fa4-b3af-28d5e84250fb') - def test_list_baremetal_nodes(self): - # List all baremetal nodes. - baremetal_nodes = self.client.list_baremetal_nodes() - self.assertNotEmpty(baremetal_nodes, "No baremetal nodes found.") - - for node in baremetal_nodes: - baremetal_node = self.client.get_baremetal_node(node['id']) - self.assertEqual(node['id'], baremetal_node['id']) + def test_list_get_baremetal_nodes(self): + # Create some test nodes in Ironic directly + test_nodes = [] + for i in range(0, 3): + _, node = self.ironic_client.create_node() + test_nodes.append(node) + self.addCleanup(self.ironic_client.delete_node, node['uuid']) + + # List all baremetal nodes and ensure our created test nodes are + # listed + bm_node_ids = set([n['id'] for n in + self.client.list_baremetal_nodes()]) + test_node_ids = set([n['uuid'] for n in test_nodes]) + self.assertTrue(test_node_ids.issubset(bm_node_ids)) + + # Test getting each individually + for node in test_nodes: + baremetal_node = self.client.get_baremetal_node(node['uuid']) + self.assertEqual(node['uuid'], baremetal_node['id']) diff --git a/tempest/api/compute/admin/test_flavors_negative.py b/tempest/api/compute/admin/test_flavors_negative.py deleted file mode 100644 index c7eb9ae40..000000000 --- a/tempest/api/compute/admin/test_flavors_negative.py +++ /dev/null @@ -1,120 +0,0 @@ -# Copyright 2012 OpenStack Foundation -# All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. - -import uuid - -from tempest_lib.common.utils import data_utils -from tempest_lib import exceptions as lib_exc - -from tempest.api.compute import base -from tempest.api_schema.request.compute.v2 import flavors -from tempest import config -from tempest import test - - -CONF = config.CONF - -load_tests = test.NegativeAutoTest.load_tests - - -class FlavorsAdminNegativeTestJSON(base.BaseV2ComputeAdminTest): - - """ - Tests Flavors API Create and Delete that require admin privileges - """ - - @classmethod - def skip_checks(cls): - super(FlavorsAdminNegativeTestJSON, cls).skip_checks() - if not test.is_extension_enabled('OS-FLV-EXT-DATA', 'compute'): - msg = "OS-FLV-EXT-DATA extension not enabled." - raise cls.skipException(msg) - - @classmethod - def setup_clients(cls): - super(FlavorsAdminNegativeTestJSON, cls).setup_clients() - cls.client = cls.os_adm.flavors_client - cls.user_client = cls.os.flavors_client - - @classmethod - def resource_setup(cls): - super(FlavorsAdminNegativeTestJSON, cls).resource_setup() - cls.flavor_name_prefix = 'test_flavor_' - cls.ram = 512 - cls.vcpus = 1 - cls.disk = 10 - cls.ephemeral = 10 - cls.swap = 1024 - cls.rxtx = 2 - - @test.attr(type=['negative', 'gate']) - @test.idempotent_id('404451c0-c1ae-4448-8d50-d74f26f93ec8') - def test_get_flavor_details_for_deleted_flavor(self): - # Delete a flavor and ensure it is not listed - # Create a test flavor - flavor_name = data_utils.rand_name(self.flavor_name_prefix) - - # no need to specify flavor_id, we can get the flavor_id from a - # response of create_flavor() call. - flavor = self.client.create_flavor(flavor_name, - self.ram, - self.vcpus, self.disk, - None, - ephemeral=self.ephemeral, - swap=self.swap, - rxtx=self.rxtx) - # Delete the flavor - new_flavor_id = flavor['id'] - self.client.delete_flavor(new_flavor_id) - - # Deleted flavors can be seen via detailed GET - flavor = self.client.get_flavor_details(new_flavor_id) - self.assertEqual(flavor['name'], flavor_name) - - # Deleted flavors should not show up in a list however - flavors = self.client.list_flavors_with_detail() - flag = True - for flavor in flavors: - if flavor['name'] == flavor_name: - flag = False - self.assertTrue(flag) - - @test.attr(type=['negative', 'gate']) - @test.idempotent_id('6f56e7b7-7500-4d0c-9913-880ca1efed87') - def test_create_flavor_as_user(self): - # only admin user can create a flavor - flavor_name = data_utils.rand_name(self.flavor_name_prefix) - new_flavor_id = str(uuid.uuid4()) - - self.assertRaises(lib_exc.Forbidden, - self.user_client.create_flavor, - flavor_name, self.ram, self.vcpus, self.disk, - new_flavor_id, ephemeral=self.ephemeral, - swap=self.swap, rxtx=self.rxtx) - - @test.attr(type=['negative', 'gate']) - @test.idempotent_id('a9a6dc02-8c14-4e05-a1ca-3468d4214882') - def test_delete_flavor_as_user(self): - # only admin user can delete a flavor - self.assertRaises(lib_exc.Forbidden, - self.user_client.delete_flavor, - self.flavor_ref_alt) - - -@test.SimpleNegativeAutoTest -class FlavorCreateNegativeTestJSON(base.BaseV2ComputeAdminTest, - test.NegativeAutoTest): - _service = CONF.compute.catalog_type - _schema = flavors.flavor_create diff --git a/tempest/api/compute/admin/test_floating_ips_bulk.py b/tempest/api/compute/admin/test_floating_ips_bulk.py index 3c5f5071c..3c48d9e3a 100644 --- a/tempest/api/compute/admin/test_floating_ips_bulk.py +++ b/tempest/api/compute/admin/test_floating_ips_bulk.py @@ -17,6 +17,7 @@ import netaddr from tempest.api.compute import base from tempest import config +from tempest import exceptions from tempest import test CONF = config.CONF @@ -51,7 +52,7 @@ class FloatingIPsBulkAdminTestJSON(base.BaseV2ComputeAdminTest): msg = ("Configured unallocated floating IP range is already " "allocated. Configure the correct unallocated range " "as 'floating_ip_range'") - raise cls.skipException(msg) + raise exceptions.InvalidConfiguration(msg) return def _delete_floating_ips_bulk(self, ip_range): diff --git a/tempest/api/compute/admin/test_security_groups.py b/tempest/api/compute/admin/test_security_groups.py index 578f73bff..6d79a7747 100644 --- a/tempest/api/compute/admin/test_security_groups.py +++ b/tempest/api/compute/admin/test_security_groups.py @@ -39,7 +39,7 @@ class SecurityGroupsTestAdminJSON(base.BaseV2ComputeAdminTest): @test.idempotent_id('49667619-5af9-4c63-ab5d-2cfdd1c8f7f1') @testtools.skipIf(CONF.service_available.neutron, - "Skipped because neutron do not support all_tenants" + "Skipped because neutron does not support all_tenants " "search filter.") @test.attr(type='smoke') @test.services('network') diff --git a/tempest/api/data_processing/base.py b/tempest/api/data_processing/base.py index 5992921d7..d91fbaa40 100644 --- a/tempest/api/data_processing/base.py +++ b/tempest/api/data_processing/base.py @@ -66,7 +66,6 @@ class BaseDataProcessingTest(tempest.test.BaseTestCase): cls.client.delete_job_binary_internal) cls.cleanup_resources(getattr(cls, '_data_sources', []), cls.client.delete_data_source) - cls.clear_isolated_creds() super(BaseDataProcessingTest, cls).resource_cleanup() @staticmethod diff --git a/tempest/api/identity/base.py b/tempest/api/identity/base.py index 543dea14b..882ef9827 100644 --- a/tempest/api/identity/base.py +++ b/tempest/api/identity/base.py @@ -31,8 +31,8 @@ class BaseIdentityAdminTest(tempest.test.BaseTestCase): @classmethod def setup_credentials(cls): super(BaseIdentityAdminTest, cls).setup_credentials() - cls.os_adm = clients.AdminManager() - cls.os = clients.Manager() + cls.os = cls.get_client_manager() + cls.os_adm = clients.Manager(cls.isolated_creds.get_admin_creds()) @classmethod def disable_user(cls, user_name): diff --git a/tempest/api/image/base.py b/tempest/api/image/base.py index 728d0772f..d513b0c6e 100644 --- a/tempest/api/image/base.py +++ b/tempest/api/image/base.py @@ -61,7 +61,6 @@ class BaseImageTest(tempest.test.BaseTestCase): for image_id in cls.created_images: cls.client.wait_for_resource_deletion(image_id) - cls.isolated_creds.clear_isolated_creds() super(BaseImageTest, cls).resource_cleanup() @classmethod diff --git a/tempest/api/network/base.py b/tempest/api/network/base.py index e057bb805..cc2d21af2 100644 --- a/tempest/api/network/base.py +++ b/tempest/api/network/base.py @@ -157,7 +157,6 @@ class BaseNetworkTest(tempest.test.BaseTestCase): for network in cls.networks: cls._try_delete_resource(cls.client.delete_network, network['id']) - cls.clear_isolated_creds() super(BaseNetworkTest, cls).resource_cleanup() @classmethod diff --git a/tempest/api/object_storage/base.py b/tempest/api/object_storage/base.py index f75f4c843..c8697e1ad 100644 --- a/tempest/api/object_storage/base.py +++ b/tempest/api/object_storage/base.py @@ -67,11 +67,6 @@ class BaseObjectTest(tempest.test.BaseTestCase): cls.account_client.auth_provider.clear_auth() @classmethod - def resource_cleanup(cls): - cls.isolated_creds.clear_isolated_creds() - super(BaseObjectTest, cls).resource_cleanup() - - @classmethod def delete_containers(cls, containers, container_client=None, object_client=None): """Remove given containers and all objects in them. diff --git a/tempest/api/orchestration/base.py b/tempest/api/orchestration/base.py index 1877bbf5e..59fdec06c 100644 --- a/tempest/api/orchestration/base.py +++ b/tempest/api/orchestration/base.py @@ -18,6 +18,7 @@ from tempest_lib import exceptions as lib_exc import yaml from tempest import clients +from tempest.common import credentials from tempest import config import tempest.test @@ -38,7 +39,19 @@ class BaseOrchestrationTest(tempest.test.BaseTestCase): @classmethod def setup_credentials(cls): super(BaseOrchestrationTest, cls).setup_credentials() - cls.os = clients.Manager() + if (not hasattr(cls, 'isolated_creds') or + not cls.isolated_creds.name == cls.__name__): + cls.isolated_creds = credentials.get_isolated_credentials( + name=cls.__name__, network_resources=cls.network_resources) + stack_owner_role = CONF.orchestration.stack_owner_role + if not cls.isolated_creds.is_role_available(stack_owner_role): + skip_msg = ("%s skipped because the configured credential provider" + " is not able to provide credentials with the %s role " + "assigned." % (cls.__name__, stack_owner_role)) + raise cls.skipException(skip_msg) + else: + cls.os = clients.Manager(cls.isolated_creds.get_creds_by_roles( + [stack_owner_role])) @classmethod def setup_clients(cls): @@ -70,7 +83,7 @@ class BaseOrchestrationTest(tempest.test.BaseTestCase): @classmethod def _get_identity_admin_client(cls): """Returns an instance of the Identity Admin API client.""" - manager = clients.AdminManager() + manager = clients.Manager(cls.isolated_creds.get_admin_creds()) admin_client = manager.identity_client return admin_client diff --git a/tempest/api/telemetry/base.py b/tempest/api/telemetry/base.py index 336e2d41c..ed719c2b9 100644 --- a/tempest/api/telemetry/base.py +++ b/tempest/api/telemetry/base.py @@ -101,7 +101,6 @@ class BaseTelemetryTest(tempest.test.BaseTestCase): cls.cleanup_resources(cls.telemetry_client.delete_alarm, cls.alarm_ids) cls.cleanup_resources(cls.servers_client.delete_server, cls.server_ids) cls.cleanup_resources(cls.image_client.delete_image, cls.image_ids) - cls.clear_isolated_creds() super(BaseTelemetryTest, cls).resource_cleanup() def await_samples(self, metric, query): diff --git a/tempest/api/volume/admin/test_volume_quotas.py b/tempest/api/volume/admin/test_volume_quotas.py index 7a64de3cf..86d90f6fe 100644 --- a/tempest/api/volume/admin/test_volume_quotas.py +++ b/tempest/api/volume/admin/test_volume_quotas.py @@ -95,7 +95,8 @@ class BaseVolumeQuotasAdminV2TestJSON(base.BaseVolumeAdminTest): self.assertEqual(quota_usage['volumes']['in_use'] + 1, new_quota_usage['volumes']['in_use']) - self.assertEqual(quota_usage['gigabytes']['in_use'] + 1, + self.assertEqual(quota_usage['gigabytes']['in_use'] + + volume["size"], new_quota_usage['gigabytes']['in_use']) @test.attr(type='gate') diff --git a/tempest/api/volume/admin/test_volume_quotas_negative.py b/tempest/api/volume/admin/test_volume_quotas_negative.py index 98b7143dd..d7287f036 100644 --- a/tempest/api/volume/admin/test_volume_quotas_negative.py +++ b/tempest/api/volume/admin/test_volume_quotas_negative.py @@ -31,7 +31,9 @@ class BaseVolumeQuotasNegativeV2TestJSON(base.BaseVolumeAdminTest): @classmethod def resource_setup(cls): super(BaseVolumeQuotasNegativeV2TestJSON, cls).resource_setup() - cls.shared_quota_set = {'gigabytes': 3, 'volumes': 1, 'snapshots': 1} + cls.default_volume_size = cls.volumes_client.default_volume_size + cls.shared_quota_set = {'gigabytes': 3 * cls.default_volume_size, + 'volumes': 1, 'snapshots': 1} # NOTE(gfidente): no need to restore original quota set # after the tests as they only work with tenant isolation. @@ -67,14 +69,16 @@ class BaseVolumeQuotasNegativeV2TestJSON(base.BaseVolumeAdminTest): self.demo_tenant_id, **self.shared_quota_set) - new_quota_set = {'gigabytes': 2, 'volumes': 2, 'snapshots': 1} + new_quota_set = {'gigabytes': 2 * self.default_volume_size, + 'volumes': 2, 'snapshots': 1} self.quotas_client.update_quota_set( self.demo_tenant_id, **new_quota_set) self.assertRaises(lib_exc.OverLimit, self.volumes_client.create_volume) - new_quota_set = {'gigabytes': 2, 'volumes': 1, 'snapshots': 2} + new_quota_set = {'gigabytes': 2 * self.default_volume_size, + 'volumes': 1, 'snapshots': 2} self.quotas_client.update_quota_set( self.demo_tenant_id, **self.shared_quota_set) diff --git a/tempest/api/volume/base.py b/tempest/api/volume/base.py index 8f3f1a3a2..157bd4488 100644 --- a/tempest/api/volume/base.py +++ b/tempest/api/volume/base.py @@ -102,7 +102,6 @@ class BaseVolumeTest(tempest.test.BaseTestCase): def resource_cleanup(cls): cls.clear_snapshots() cls.clear_volumes() - cls.clear_isolated_creds() super(BaseVolumeTest, cls).resource_cleanup() @classmethod diff --git a/tempest/api_schema/response/compute/baremetal_nodes.py b/tempest/api_schema/response/compute/baremetal_nodes.py index e82792cf7..82506e775 100644 --- a/tempest/api_schema/response/compute/baremetal_nodes.py +++ b/tempest/api_schema/response/compute/baremetal_nodes.py @@ -12,6 +12,8 @@ # License for the specific language governing permissions and limitations # under the License. +import copy + node = { 'type': 'object', 'properties': { @@ -41,7 +43,7 @@ list_baremetal_nodes = { } } -get_baremetal_node = { +baremetal_node = { 'status_code': [200], 'response_body': { 'type': 'object', @@ -51,3 +53,8 @@ get_baremetal_node = { 'required': ['node'] } } +get_baremetal_node = copy.deepcopy(baremetal_node) +get_baremetal_node['response_body']['properties']['node'][ + 'properties'].update({'instance_uuid': {'type': ['string', 'null']}}) +get_baremetal_node['response_body']['properties']['node'][ + 'required'].append('instance_uuid') diff --git a/tempest/api_schema/response/compute/hypervisors.py b/tempest/api_schema/response/compute/hypervisors.py index fc3b8288a..d6f2bd120 100644 --- a/tempest/api_schema/response/compute/hypervisors.py +++ b/tempest/api_schema/response/compute/hypervisors.py @@ -80,8 +80,12 @@ common_list_hypervisors_detail = { 'type': 'object', 'properties': { 'host': {'type': 'string'}, - 'id': {'type': ['integer', 'string']} + 'id': {'type': ['integer', 'string']}, + 'disabled_reason': {'type': ['string', 'null']} }, + # NOTE(gmann): 'disabled_reason' is updated in + # 'service' dict if 'os-hypervisor-status' + # extension is loaded. So this is not required. 'required': ['host', 'id'] }, 'vcpus': {'type': 'integer'}, @@ -137,8 +141,12 @@ common_show_hypervisor = { 'type': 'object', 'properties': { 'host': {'type': 'string'}, - 'id': {'type': ['integer', 'string']} + 'id': {'type': ['integer', 'string']}, + 'disabled_reason': {'type': ['string', 'null']} }, + # NOTE: 'disabled_reason' is updated in 'service' + # dict if os-hypervisor-status' extension is loaded. + # So this is not required. 'required': ['host', 'id'] }, 'vcpus': {'type': 'integer'}, diff --git a/tempest/api_schema/response/compute/parameter_types.py b/tempest/api_schema/response/compute/parameter_types.py index 4a1dfddf9..90d4c8f7d 100644 --- a/tempest/api_schema/response/compute/parameter_types.py +++ b/tempest/api_schema/response/compute/parameter_types.py @@ -65,3 +65,17 @@ addresses = { } } } + +response_header = { + 'connection': {'type': 'string'}, + 'content-length': {'type': 'string'}, + 'content-type': {'type': 'string'}, + 'status': {'type': 'string'}, + 'x-compute-request-id': {'type': 'string'}, + 'vary': {'type': 'string'}, + 'x-openstack-nova-api-version': {'type': 'string'}, + 'date': { + 'type': 'string', + 'format': 'data-time' + } +} diff --git a/tempest/api_schema/response/compute/servers.py b/tempest/api_schema/response/compute/servers.py index f9c957baf..395017325 100644 --- a/tempest/api_schema/response/compute/servers.py +++ b/tempest/api_schema/response/compute/servers.py @@ -71,6 +71,18 @@ common_show_server = { }, 'required': ['id', 'links'] }, + 'fault': { + 'type': 'object', + 'properties': { + 'code': {'type': 'integer'}, + 'created': {'type': 'string'}, + 'message': {'type': 'string'}, + 'details': {'type': 'string'}, + }, + # NOTE(gmann): 'details' is not necessary to be present + # in the 'fault'. So it is not defined as 'required'. + 'required': ['code', 'created', 'message'] + }, 'user_id': {'type': 'string'}, 'tenant_id': {'type': 'string'}, 'created': {'type': 'string'}, @@ -83,7 +95,9 @@ common_show_server = { # NOTE(GMann): 'progress' attribute is present in the response # only when server's status is one of the progress statuses # ("ACTIVE","BUILD", "REBUILD", "RESIZE","VERIFY_RESIZE") - # So it is not defined as 'required'. + # 'fault' attribute is present in the response + # only when server's status is one of the "ERROR", "DELETED". + # So they are not defined as 'required'. 'required': ['id', 'name', 'status', 'image', 'flavor', 'user_id', 'tenant_id', 'created', 'updated', 'metadata', 'links', 'addresses'] @@ -144,8 +158,11 @@ list_servers = { }, 'required': ['id', 'links', 'name'] } - } + }, + 'servers_links': parameter_types.links }, + # NOTE(gmann): servers_links attribute is not necessary to be + # present always So it is not 'required'. 'required': ['servers'] } } diff --git a/tempest/api_schema/response/compute/v2/interfaces.py b/tempest/api_schema/response/compute/v2/interfaces.py deleted file mode 100644 index 64d161d99..000000000 --- a/tempest/api_schema/response/compute/v2/interfaces.py +++ /dev/null @@ -1,29 +0,0 @@ -# Copyright 2014 NEC Corporation. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. - -from tempest.api_schema.response.compute import interfaces as common_schema - -list_interfaces = { - 'status_code': [200], - 'response_body': { - 'type': 'object', - 'properties': { - 'interfaceAttachments': { - 'type': 'array', - 'items': common_schema.interface_common_info - } - }, - 'required': ['interfaceAttachments'] - } -} diff --git a/tempest/api_schema/response/compute/v2/__init__.py b/tempest/api_schema/response/compute/v2_1/__init__.py index e69de29bb..e69de29bb 100644 --- a/tempest/api_schema/response/compute/v2/__init__.py +++ b/tempest/api_schema/response/compute/v2_1/__init__.py diff --git a/tempest/api_schema/response/compute/v2/agents.py b/tempest/api_schema/response/compute/v2_1/agents.py index d827377b7..d827377b7 100644 --- a/tempest/api_schema/response/compute/v2/agents.py +++ b/tempest/api_schema/response/compute/v2_1/agents.py diff --git a/tempest/api_schema/response/compute/v2/aggregates.py b/tempest/api_schema/response/compute/v2_1/aggregates.py index d87e4de96..d87e4de96 100644 --- a/tempest/api_schema/response/compute/v2/aggregates.py +++ b/tempest/api_schema/response/compute/v2_1/aggregates.py diff --git a/tempest/api_schema/response/compute/v2/availability_zone.py b/tempest/api_schema/response/compute/v2_1/availability_zone.py index e261d3d6c..e261d3d6c 100644 --- a/tempest/api_schema/response/compute/v2/availability_zone.py +++ b/tempest/api_schema/response/compute/v2_1/availability_zone.py diff --git a/tempest/api_schema/response/compute/v2/certificates.py b/tempest/api_schema/response/compute/v2_1/certificates.py index bda607587..bda607587 100644 --- a/tempest/api_schema/response/compute/v2/certificates.py +++ b/tempest/api_schema/response/compute/v2_1/certificates.py diff --git a/tempest/api_schema/response/compute/v2/extensions.py b/tempest/api_schema/response/compute/v2_1/extensions.py index 570cd030a..570cd030a 100644 --- a/tempest/api_schema/response/compute/v2/extensions.py +++ b/tempest/api_schema/response/compute/v2_1/extensions.py diff --git a/tempest/api_schema/response/compute/v2/fixed_ips.py b/tempest/api_schema/response/compute/v2_1/fixed_ips.py index 446633fbc..446633fbc 100644 --- a/tempest/api_schema/response/compute/v2/fixed_ips.py +++ b/tempest/api_schema/response/compute/v2_1/fixed_ips.py diff --git a/tempest/api_schema/response/compute/v2/flavors.py b/tempest/api_schema/response/compute/v2_1/flavors.py index 76c4ceea0..76c4ceea0 100644 --- a/tempest/api_schema/response/compute/v2/flavors.py +++ b/tempest/api_schema/response/compute/v2_1/flavors.py diff --git a/tempest/api_schema/response/compute/v2/floating_ips.py b/tempest/api_schema/response/compute/v2_1/floating_ips.py index 72507731f..72507731f 100644 --- a/tempest/api_schema/response/compute/v2/floating_ips.py +++ b/tempest/api_schema/response/compute/v2_1/floating_ips.py diff --git a/tempest/api_schema/response/compute/v2/hosts.py b/tempest/api_schema/response/compute/v2_1/hosts.py index 09447923b..09447923b 100644 --- a/tempest/api_schema/response/compute/v2/hosts.py +++ b/tempest/api_schema/response/compute/v2_1/hosts.py diff --git a/tempest/api_schema/response/compute/v2/hypervisors.py b/tempest/api_schema/response/compute/v2_1/hypervisors.py index cbb76984b..cbb76984b 100644 --- a/tempest/api_schema/response/compute/v2/hypervisors.py +++ b/tempest/api_schema/response/compute/v2_1/hypervisors.py diff --git a/tempest/api_schema/response/compute/v2/images.py b/tempest/api_schema/response/compute/v2_1/images.py index 43dae380b..3c0b80e79 100644 --- a/tempest/api_schema/response/compute/v2/images.py +++ b/tempest/api_schema/response/compute/v2_1/images.py @@ -78,8 +78,11 @@ list_images = { }, 'required': ['id', 'links', 'name'] } - } + }, + 'images_links': parameter_types.links }, + # NOTE(gmann): images_links attribute is not necessary to be + # present always So it is not 'required'. 'required': ['images'] } } @@ -88,15 +91,16 @@ create_image = { 'status_code': [202], 'response_header': { 'type': 'object', - 'properties': { - 'location': { - 'type': 'string', - 'format': 'uri' - } - }, - 'required': ['location'] + 'properties': parameter_types.response_header } } +create_image['response_header']['properties'].update( + {'location': { + 'type': 'string', + 'format': 'uri'} + } +) +create_image['response_header']['required'] = ['location'] delete = { 'status_code': [204] @@ -132,8 +136,11 @@ list_images_details = { 'images': { 'type': 'array', 'items': common_image_schema - } + }, + 'images_links': parameter_types.links }, + # NOTE(gmann): images_links attribute is not necessary to be + # present always So it is not 'required'. 'required': ['images'] } } diff --git a/tempest/api_schema/response/compute/v2/instance_usage_audit_logs.py b/tempest/api_schema/response/compute/v2_1/instance_usage_audit_logs.py index 658f5746c..658f5746c 100644 --- a/tempest/api_schema/response/compute/v2/instance_usage_audit_logs.py +++ b/tempest/api_schema/response/compute/v2_1/instance_usage_audit_logs.py diff --git a/tempest/api_schema/response/compute/interfaces.py b/tempest/api_schema/response/compute/v2_1/interfaces.py index fd53eb3b8..4de330986 100644 --- a/tempest/api_schema/response/compute/interfaces.py +++ b/tempest/api_schema/response/compute/v2_1/interfaces.py @@ -14,10 +14,6 @@ from tempest.api_schema.response.compute import parameter_types -delete_interface = { - 'status_code': [202] -} - interface_common_info = { 'type': 'object', 'properties': { @@ -45,3 +41,32 @@ interface_common_info = { }, 'required': ['port_state', 'fixed_ips', 'port_id', 'net_id', 'mac_addr'] } + +get_create_interfaces = { + 'status_code': [200], + 'response_body': { + 'type': 'object', + 'properties': { + 'interfaceAttachment': interface_common_info + }, + 'required': ['interfaceAttachment'] + } +} + +list_interfaces = { + 'status_code': [200], + 'response_body': { + 'type': 'object', + 'properties': { + 'interfaceAttachments': { + 'type': 'array', + 'items': interface_common_info + } + }, + 'required': ['interfaceAttachments'] + } +} + +delete_interface = { + 'status_code': [202] +} diff --git a/tempest/api_schema/response/compute/v2/keypairs.py b/tempest/api_schema/response/compute/v2_1/keypairs.py index ec26fa02f..ec26fa02f 100644 --- a/tempest/api_schema/response/compute/v2/keypairs.py +++ b/tempest/api_schema/response/compute/v2_1/keypairs.py diff --git a/tempest/api_schema/response/compute/v2/limits.py b/tempest/api_schema/response/compute/v2_1/limits.py index a7decb7b4..a7decb7b4 100644 --- a/tempest/api_schema/response/compute/v2/limits.py +++ b/tempest/api_schema/response/compute/v2_1/limits.py diff --git a/tempest/api_schema/response/compute/v2/quota_classes.py b/tempest/api_schema/response/compute/v2_1/quota_classes.py index 5474a8960..a7374df0b 100644 --- a/tempest/api_schema/response/compute/v2/quota_classes.py +++ b/tempest/api_schema/response/compute/v2_1/quota_classes.py @@ -15,7 +15,7 @@ import copy -from tempest.api_schema.response.compute.v2 import quotas +from tempest.api_schema.response.compute.v2_1 import quotas # NOTE(mriedem): os-quota-class-sets responses are the same as os-quota-sets # except for the key in the response body is quota_class_set instead of diff --git a/tempest/api_schema/response/compute/v2/quotas.py b/tempest/api_schema/response/compute/v2_1/quotas.py index 630b227c7..630b227c7 100644 --- a/tempest/api_schema/response/compute/v2/quotas.py +++ b/tempest/api_schema/response/compute/v2_1/quotas.py diff --git a/tempest/api_schema/response/compute/v2/security_group_default_rule.py b/tempest/api_schema/response/compute/v2_1/security_group_default_rule.py index 9246ab876..9246ab876 100644 --- a/tempest/api_schema/response/compute/v2/security_group_default_rule.py +++ b/tempest/api_schema/response/compute/v2_1/security_group_default_rule.py diff --git a/tempest/api_schema/response/compute/v2/security_groups.py b/tempest/api_schema/response/compute/v2_1/security_groups.py index 9a852e52e..9a852e52e 100644 --- a/tempest/api_schema/response/compute/v2/security_groups.py +++ b/tempest/api_schema/response/compute/v2_1/security_groups.py diff --git a/tempest/api_schema/response/compute/v2/servers.py b/tempest/api_schema/response/compute/v2_1/servers.py index c116bf549..ebee69769 100644 --- a/tempest/api_schema/response/compute/v2/servers.py +++ b/tempest/api_schema/response/compute/v2_1/servers.py @@ -335,7 +335,11 @@ list_servers_detail['response_body']['properties']['servers']['items'][ 'items']['properties'].update({ 'OS-EXT-IPS:type': {'type': 'string'}, 'OS-EXT-IPS-MAC:mac_addr': parameter_types.mac_address}) - +# Defining 'servers_links' attributes for V2 server schema +list_servers_detail['response_body'][ + 'properties'].update({'servers_links': parameter_types.links}) +# NOTE(gmann): servers_links attribute is not necessary to be +# present always So it is not 'required'. rebuild_server = copy.deepcopy(update_server) rebuild_server['status_code'] = [202] diff --git a/tempest/api_schema/response/compute/v2/tenant_networks.py b/tempest/api_schema/response/compute/v2_1/tenant_networks.py index 0b2868a79..0b2868a79 100644 --- a/tempest/api_schema/response/compute/v2/tenant_networks.py +++ b/tempest/api_schema/response/compute/v2_1/tenant_networks.py diff --git a/tempest/api_schema/response/compute/v2/tenant_usages.py b/tempest/api_schema/response/compute/v2_1/tenant_usages.py index 0b824a1cd..0b824a1cd 100644 --- a/tempest/api_schema/response/compute/v2/tenant_usages.py +++ b/tempest/api_schema/response/compute/v2_1/tenant_usages.py diff --git a/tempest/api_schema/response/compute/v2/volumes.py b/tempest/api_schema/response/compute/v2_1/volumes.py index 541d3ff2d..541d3ff2d 100644 --- a/tempest/api_schema/response/compute/v2/volumes.py +++ b/tempest/api_schema/response/compute/v2_1/volumes.py diff --git a/tempest/cli/simple_read_only/network/__init__.py b/tempest/cli/simple_read_only/network/__init__.py deleted file mode 100644 index e69de29bb..000000000 --- a/tempest/cli/simple_read_only/network/__init__.py +++ /dev/null diff --git a/tempest/cli/simple_read_only/network/test_neutron.py b/tempest/cli/simple_read_only/network/test_neutron.py deleted file mode 100644 index e8b355407..000000000 --- a/tempest/cli/simple_read_only/network/test_neutron.py +++ /dev/null @@ -1,285 +0,0 @@ -# Copyright 2013 OpenStack Foundation -# All Rights Reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); you may -# not use this file except in compliance with the License. You may obtain -# a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT -# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the -# License for the specific language governing permissions and limitations -# under the License. - -import re - -from oslo_log import log as logging -from tempest_lib import exceptions - -from tempest import cli -from tempest import config -from tempest import test - -CONF = config.CONF - -LOG = logging.getLogger(__name__) - - -class SimpleReadOnlyNeutronClientTest(cli.ClientTestBase): - """Basic, read-only tests for Neutron CLI client. - - Checks return values and output of read-only commands. - These tests do not presume any content, nor do they create - their own. They only verify the structure of output if present. - """ - - @classmethod - def resource_setup(cls): - if (not CONF.service_available.neutron): - msg = "Skipping all Neutron cli tests because it is not available" - raise cls.skipException(msg) - super(SimpleReadOnlyNeutronClientTest, cls).resource_setup() - - def neutron(self, *args, **kwargs): - return self.clients.neutron(*args, - endpoint_type=CONF.network.endpoint_type, - **kwargs) - - @test.attr(type='smoke') - @test.idempotent_id('84dd7190-2b98-4709-8e2c-3c1d25b9e7d2') - def test_neutron_fake_action(self): - self.assertRaises(exceptions.CommandFailed, - self.neutron, - 'this-does-not-exist') - - @test.attr(type='smoke') - @test.idempotent_id('c598c337-313a-45ac-bf27-d6b4124a9e5b') - def test_neutron_net_list(self): - net_list = self.parser.listing(self.neutron('net-list')) - self.assertTableStruct(net_list, ['id', 'name', 'subnets']) - - @test.attr(type='smoke') - @test.idempotent_id('3e172b04-2e3b-4fcf-922d-99d5c803779f') - def test_neutron_ext_list(self): - ext = self.parser.listing(self.neutron('ext-list')) - self.assertTableStruct(ext, ['alias', 'name']) - - @test.attr(type='smoke') - @test.idempotent_id('2e0de814-52d6-4f81-be17-fe327072fc23') - @test.requires_ext(extension='dhcp_agent_scheduler', service='network') - def test_neutron_dhcp_agent_list_hosting_net(self): - self.neutron('dhcp-agent-list-hosting-net', - params=CONF.compute.fixed_network_name) - - @test.attr(type='smoke') - @test.idempotent_id('8524a24a-3895-40a5-8c9d-49d4459cdda4') - @test.requires_ext(extension='agent', service='network') - def test_neutron_agent_list(self): - agents = self.parser.listing(self.neutron('agent-list')) - field_names = ['id', 'agent_type', 'host', 'alive', 'admin_state_up'] - self.assertTableStruct(agents, field_names) - - @test.attr(type='smoke') - @test.idempotent_id('97c3ef92-7303-45f1-80db-b6622f176782') - @test.requires_ext(extension='router', service='network') - def test_neutron_floatingip_list(self): - self.neutron('floatingip-list') - - @test.attr(type='smoke') - @test.idempotent_id('823e0fee-404c-49a7-8bf3-d2f0383cc649') - @test.requires_ext(extension='metering', service='network') - def test_neutron_meter_label_list(self): - self.neutron('meter-label-list') - - @test.attr(type='smoke') - @test.idempotent_id('7fb76098-01f6-417f-b9c7-e630ba3f394b') - @test.requires_ext(extension='metering', service='network') - def test_neutron_meter_label_rule_list(self): - self.neutron('meter-label-rule-list') - - @test.requires_ext(extension='lbaas_agent_scheduler', service='network') - def _test_neutron_lbaas_command(self, command): - try: - self.neutron(command) - except exceptions.CommandFailed as e: - if '404 Not Found' not in e.stderr: - self.fail('%s: Unexpected failure.' % command) - - @test.attr(type='smoke') - @test.idempotent_id('396d1d87-fd0c-4716-9ff0-f1baa54c6c61') - def test_neutron_lb_healthmonitor_list(self): - self._test_neutron_lbaas_command('lb-healthmonitor-list') - - @test.attr(type='smoke') - @test.idempotent_id('f41fa54d-5cd8-4f2c-bb4e-13abc72dccb6') - def test_neutron_lb_member_list(self): - self._test_neutron_lbaas_command('lb-member-list') - - @test.attr(type='smoke') - @test.idempotent_id('3ec04885-7573-4cce-b086-5722c0b00d85') - def test_neutron_lb_pool_list(self): - self._test_neutron_lbaas_command('lb-pool-list') - - @test.attr(type='smoke') - @test.idempotent_id('1ab530e0-ec87-498f-baf2-85f6635a2ad9') - def test_neutron_lb_vip_list(self): - self._test_neutron_lbaas_command('lb-vip-list') - - @test.attr(type='smoke') - @test.idempotent_id('e92f7362-4009-4b37-afee-f469105b24e7') - @test.requires_ext(extension='external-net', service='network') - def test_neutron_net_external_list(self): - net_ext_list = self.parser.listing(self.neutron('net-external-list')) - self.assertTableStruct(net_ext_list, ['id', 'name', 'subnets']) - - @test.attr(type='smoke') - @test.idempotent_id('ed840980-7c84-4b6e-b280-f13c5848a0e9') - def test_neutron_port_list(self): - port_list = self.parser.listing(self.neutron('port-list')) - self.assertTableStruct(port_list, ['id', 'name', 'mac_address', - 'fixed_ips']) - - @test.attr(type='smoke') - @test.idempotent_id('dded0dfa-f2ac-4c1f-bc90-69fd06dd7132') - @test.requires_ext(extension='quotas', service='network') - def test_neutron_quota_list(self): - self.neutron('quota-list') - - @test.attr(type='smoke') - @test.idempotent_id('927fca1e-4397-42a2-ba47-d738299466de') - @test.requires_ext(extension='router', service='network') - def test_neutron_router_list(self): - router_list = self.parser.listing(self.neutron('router-list')) - self.assertTableStruct(router_list, ['id', 'name', - 'external_gateway_info']) - - @test.attr(type='smoke') - @test.idempotent_id('e2e3d2d5-1aee-499d-84d9-37382dcf26ff') - @test.requires_ext(extension='security-group', service='network') - def test_neutron_security_group_list(self): - security_grp = self.parser.listing(self.neutron('security-group-list')) - self.assertTableStruct(security_grp, ['id', 'name', 'description']) - - @test.attr(type='smoke') - @test.idempotent_id('288602c2-8b59-44cd-8c5d-1ec916a114d3') - @test.requires_ext(extension='security-group', service='network') - def test_neutron_security_group_rule_list(self): - security_grp = self.parser.listing(self.neutron - ('security-group-rule-list')) - self.assertTableStruct(security_grp, ['id', 'security_group', - 'direction', 'protocol', - 'remote_ip_prefix', - 'remote_group']) - - @test.attr(type='smoke') - @test.idempotent_id('2a874a08-b9c9-4f0f-82ef-8cadb15bbd5d') - def test_neutron_subnet_list(self): - subnet_list = self.parser.listing(self.neutron('subnet-list')) - self.assertTableStruct(subnet_list, ['id', 'name', 'cidr', - 'allocation_pools']) - - @test.attr(type='smoke') - @test.idempotent_id('048e1ec3-cf6c-4066-b262-2028e03ce825') - @test.requires_ext(extension='vpnaas', service='network') - def test_neutron_vpn_ikepolicy_list(self): - ikepolicy = self.parser.listing(self.neutron('vpn-ikepolicy-list')) - self.assertTableStruct(ikepolicy, ['id', 'name', - 'auth_algorithm', - 'encryption_algorithm', - 'ike_version', 'pfs']) - - @test.attr(type='smoke') - @test.idempotent_id('bb8902b7-b2e6-49fd-b9bd-a26dd99732df') - @test.requires_ext(extension='vpnaas', service='network') - def test_neutron_vpn_ipsecpolicy_list(self): - ipsecpolicy = self.parser.listing(self.neutron('vpn-ipsecpolicy-list')) - self.assertTableStruct(ipsecpolicy, ['id', 'name', - 'auth_algorithm', - 'encryption_algorithm', - 'pfs']) - - @test.attr(type='smoke') - @test.idempotent_id('c0f33f9a-0ba9-4177-bcd5-dce34b81d523') - @test.requires_ext(extension='vpnaas', service='network') - def test_neutron_vpn_service_list(self): - vpn_list = self.parser.listing(self.neutron('vpn-service-list')) - self.assertTableStruct(vpn_list, ['id', 'name', - 'router_id', 'status']) - - @test.attr(type='smoke') - @test.idempotent_id('bb142f8a-e568-405f-b1b7-4cb458de7971') - @test.requires_ext(extension='vpnaas', service='network') - def test_neutron_ipsec_site_connection_list(self): - ipsec_site = self.parser.listing(self.neutron - ('ipsec-site-connection-list')) - self.assertTableStruct(ipsec_site, ['id', 'name', - 'peer_address', - 'peer_cidrs', - 'route_mode', - 'auth_mode', 'status']) - - @test.attr(type='smoke') - @test.idempotent_id('89baff14-8cb7-4ad8-9c24-b0278711170b') - @test.requires_ext(extension='fwaas', service='network') - def test_neutron_firewall_list(self): - firewall_list = self.parser.listing(self.neutron - ('firewall-list')) - self.assertTableStruct(firewall_list, ['id', 'name', - 'firewall_policy_id']) - - @test.attr(type='smoke') - @test.idempotent_id('996e418a-2a51-4018-9602-478ca8053e61') - @test.requires_ext(extension='fwaas', service='network') - def test_neutron_firewall_policy_list(self): - firewall_policy = self.parser.listing(self.neutron - ('firewall-policy-list')) - self.assertTableStruct(firewall_policy, ['id', 'name', - 'firewall_rules']) - - @test.attr(type='smoke') - @test.idempotent_id('d4638dd6-98d4-4400-a920-26572de1a6fc') - @test.requires_ext(extension='fwaas', service='network') - def test_neutron_firewall_rule_list(self): - firewall_rule = self.parser.listing(self.neutron - ('firewall-rule-list')) - self.assertTableStruct(firewall_rule, ['id', 'name', - 'firewall_policy_id', - 'summary', 'enabled']) - - @test.attr(type='smoke') - @test.idempotent_id('1c4551e1-e3f3-4af2-8a40-c3f551e4a536') - def test_neutron_help(self): - help_text = self.neutron('help') - lines = help_text.split('\n') - self.assertFirstLineStartsWith(lines, 'usage: neutron') - - commands = [] - cmds_start = lines.index('Commands for API v2.0:') - command_pattern = re.compile('^ {2}([a-z0-9\-\_]+)') - for line in lines[cmds_start:]: - match = command_pattern.match(line) - if match: - commands.append(match.group(1)) - commands = set(commands) - wanted_commands = set(('net-create', 'subnet-list', 'port-delete', - 'router-show', 'agent-update', 'help')) - self.assertFalse(wanted_commands - commands) - - # Optional arguments: - - @test.attr(type='smoke') - @test.idempotent_id('381e6fe3-cddc-47c9-a773-70ddb2f79a91') - def test_neutron_version(self): - self.neutron('', flags='--version') - - @test.attr(type='smoke') - @test.idempotent_id('bcad0e07-da8c-4c7c-8ab6-499e5d7ab8cb') - def test_neutron_debug_net_list(self): - self.neutron('net-list', flags='--debug') - - @test.attr(type='smoke') - @test.idempotent_id('3e42d78e-65e5-4e8f-8c29-ca7be8feebb4') - def test_neutron_quiet_net_list(self): - self.neutron('net-list', flags='--quiet') diff --git a/tempest/cmd/javelin.py b/tempest/cmd/javelin.py index 0735eeb0d..e9702493b 100755 --- a/tempest/cmd/javelin.py +++ b/tempest/cmd/javelin.py @@ -118,6 +118,7 @@ import yaml import tempest.auth from tempest import config from tempest.services.compute.json import flavors_client +from tempest.services.compute.json import floating_ips_client from tempest.services.compute.json import security_groups_client from tempest.services.compute.json import servers_client from tempest.services.identity.v2.json import identity_client @@ -194,6 +195,8 @@ class OSClient(object): **compute_params) self.flavors = flavors_client.FlavorsClientJSON(_auth, **compute_params) + self.floating_ips = floating_ips_client.FloatingIPsClientJSON( + _auth, **compute_params) self.secgroups = security_groups_client.SecurityGroupsClientJSON( _auth, **compute_params) self.objects = object_client.ObjectClient(_auth, @@ -451,15 +454,31 @@ class JavelinCheck(unittest.TestCase): # validate neutron is enabled and ironic disabled: if (CONF.service_available.neutron and not CONF.baremetal.driver_enabled): + _floating_is_alive = False for network_name, body in found['addresses'].items(): for addr in body: ip = addr['addr'] - if addr.get('OS-EXT-IPS:type', 'fixed') == 'fixed': + # If floatingip_for_ssh is at True, it's assumed + # you want to use the floating IP to reach the server, + # fallback to fixed IP, then other type. + # This is useful in multi-node environment. + if CONF.compute.use_floatingip_for_ssh: + if addr.get('OS-EXT-IPS:type', + 'floating') == 'floating': + self._ping_ip(ip, 60) + _floating_is_alive = True + elif addr.get('OS-EXT-IPS:type', 'fixed') == 'fixed': namespace = _get_router_namespace(client, network_name) self._ping_ip(ip, 60, namespace) else: self._ping_ip(ip, 60) + # if floatingip_for_ssh is at True, validate found a + # floating IP and ping worked. + if CONF.compute.use_floatingip_for_ssh: + self.assertTrue(_floating_is_alive, + "Server %s has no floating IP." % + server['name']) else: addr = found['addresses']['private'][0]['addr'] self._ping_ip(addr, 60) @@ -838,6 +857,10 @@ def create_servers(servers): # create to security group(s) after server spawning for secgroup in server['secgroups']: client.servers.add_security_group(server_id, secgroup) + if CONF.compute.use_floatingip_for_ssh: + floating_ip = client.floating_ips.create_floating_ip() + client.floating_ips.associate_floating_ip_to_server( + floating_ip['ip'], server_id) def destroy_servers(servers): @@ -852,6 +875,7 @@ def destroy_servers(servers): LOG.info("Server '%s' does not exist" % server['name']) continue + # TODO(EmilienM): disassociate floating IP from server and release it. client.servers.delete_server(response['id']) client.servers.wait_for_server_termination(response['id'], ignore_error=True) diff --git a/tempest/common/cred_provider.py b/tempest/common/cred_provider.py index 4d66ffcb5..bff9a0a6c 100644 --- a/tempest/common/cred_provider.py +++ b/tempest/common/cred_provider.py @@ -53,7 +53,7 @@ def get_configured_credentials(credential_type, fill_in=True, if identity_version == 'v3': conf_attributes.append('domain_name') # Read the parts of credentials from config - params = DEFAULT_PARAMS + params = DEFAULT_PARAMS.copy() section, prefix = CREDENTIAL_TYPES[credential_type] for attr in conf_attributes: _section = getattr(CONF, section) diff --git a/tempest/config.py b/tempest/config.py index c459d76af..12620dede 100644 --- a/tempest/config.py +++ b/tempest/config.py @@ -351,7 +351,13 @@ ComputeFeaturesGroup = [ 'images of running instances?'), cfg.BoolOpt('ec2_api', default=True, - help='Does the test environment have the ec2 api running?') + help='Does the test environment have the ec2 api running?'), + # TODO(mriedem): Remove preserve_ports once juno-eol happens. + cfg.BoolOpt('preserve_ports', + default=False, + help='Does Nova preserve preexisting ports from Neutron ' + 'when deleting an instance? This should be set to True ' + 'if testing Kilo+ Nova.') ] @@ -690,6 +696,8 @@ OrchestrationGroup = [ choices=['public', 'admin', 'internal', 'publicURL', 'adminURL', 'internalURL'], help="The endpoint type to use for the orchestration service."), + cfg.StrOpt('stack_owner_role', default='heat_stack_owner', + help='Role required for users to be able to manage stacks'), cfg.IntOpt('build_interval', default=1, help="Time in seconds between build status checks."), @@ -1199,7 +1207,6 @@ class TempestConfigProxy(object): _path = None _extra_log_defaults = [ - ('keystoneclient.session', std_logging.INFO), ('paramiko.transport', std_logging.INFO), ('requests.packages.urllib3.connectionpool', std_logging.WARN), ] diff --git a/tempest/scenario/manager.py b/tempest/scenario/manager.py index 81e771cbe..f8cc17cde 100644 --- a/tempest/scenario/manager.py +++ b/tempest/scenario/manager.py @@ -14,7 +14,6 @@ # License for the specific language governing permissions and limitations # under the License. -import os import subprocess import netaddr @@ -24,7 +23,6 @@ from tempest_lib.common.utils import data_utils from tempest_lib import exceptions as lib_exc from tempest import clients -from tempest.common import cred_provider from tempest.common import credentials from tempest.common.utils.linux import remote_client from tempest import config @@ -50,7 +48,6 @@ class ScenarioTest(tempest.test.BaseTestCase): cls.manager = clients.Manager( credentials=cls.credentials() ) - cls.admin_manager = clients.Manager(cls.admin_credentials()) @classmethod def setup_clients(cls): @@ -63,7 +60,6 @@ class ScenarioTest(tempest.test.BaseTestCase): # Compute image client cls.images_client = cls.manager.images_client cls.keypairs_client = cls.manager.keypairs_client - cls.networks_client = cls.admin_manager.networks_client # Nova security groups client cls.security_groups_client = cls.manager.security_groups_client cls.servers_client = cls.manager.servers_client @@ -542,6 +538,14 @@ class NetworkScenarioTest(ScenarioTest): super(NetworkScenarioTest, cls).skip_checks() if not CONF.service_available.neutron: raise cls.skipException('Neutron not available') + if not credentials.is_admin_available(): + msg = ("Missing Identity Admin API credentials in configuration.") + raise cls.skipException(msg) + + @classmethod + def setup_credentials(cls): + super(NetworkScenarioTest, cls).setup_credentials() + cls.admin_manager = clients.Manager(cls.admin_credentials()) @classmethod def resource_setup(cls): @@ -1283,9 +1287,17 @@ class EncryptionScenarioTest(ScenarioTest): """ @classmethod + def skip_checks(cls): + super(EncryptionScenarioTest, cls).skip_checks() + if not credentials.is_admin_available(): + msg = ("Missing Identity Admin API credentials in configuration.") + raise cls.skipException(msg) + + @classmethod def setup_clients(cls): super(EncryptionScenarioTest, cls).setup_clients() - cls.admin_volume_types_client = cls.admin_manager.volume_types_client + admin_manager = clients.Manager(cls.admin_credentials()) + cls.admin_volume_types_client = admin_manager.volume_types_client def _wait_for_volume_status(self, status): self.status_timeout( @@ -1324,49 +1336,6 @@ class EncryptionScenarioTest(ScenarioTest): control_location=control_location) -class OrchestrationScenarioTest(ScenarioTest): - """ - Base class for orchestration scenario tests - """ - - @classmethod - def skip_checks(cls): - super(OrchestrationScenarioTest, cls).skip_checks() - if not CONF.service_available.heat: - raise cls.skipException("Heat support is required") - - @classmethod - def credentials(cls): - admin_creds = cred_provider.get_configured_credentials( - 'identity_admin') - creds = cred_provider.get_configured_credentials('user') - admin_creds.tenant_name = creds.tenant_name - return admin_creds - - def _load_template(self, base_file, file_name): - filepath = os.path.join(os.path.dirname(os.path.realpath(base_file)), - file_name) - with open(filepath) as f: - return f.read() - - @classmethod - def _stack_rand_name(cls): - return data_utils.rand_name(cls.__name__ + '-') - - @classmethod - def _get_default_network(cls): - networks = cls.networks_client.list_networks() - for net in networks: - if net['label'] == CONF.compute.fixed_network_name: - return net - - @staticmethod - def _stack_output(stack, output_key): - """Return a stack output value for a given key.""" - return next((o['output_value'] for o in stack['outputs'] - if o['output_key'] == output_key), None) - - class SwiftScenarioTest(ScenarioTest): """ Provide harness to do Swift scenario tests. diff --git a/tempest/scenario/test_encrypted_cinder_volumes.py b/tempest/scenario/test_encrypted_cinder_volumes.py index eed3d0b77..e6912d821 100644 --- a/tempest/scenario/test_encrypted_cinder_volumes.py +++ b/tempest/scenario/test_encrypted_cinder_volumes.py @@ -35,8 +35,8 @@ class TestEncryptedCinderVolumes(manager.EncryptionScenarioTest): self.glance_image_create() self.nova_boot() - def create_encrypted_volume(self, encryption_provider): - volume_type = self.create_volume_type(name='luks') + def create_encrypted_volume(self, encryption_provider, volume_type): + volume_type = self.create_volume_type(name=volume_type) self.create_encryption_type(type_id=volume_type['id'], provider=encryption_provider, key_size=512, @@ -53,7 +53,8 @@ class TestEncryptedCinderVolumes(manager.EncryptionScenarioTest): def test_encrypted_cinder_volumes_luks(self): self.launch_instance() self.create_encrypted_volume('nova.volume.encryptors.' - 'luks.LuksEncryptor') + 'luks.LuksEncryptor', + volume_type='luks') self.attach_detach_volume() @test.idempotent_id('cbc752ed-b716-4717-910f-956cce965722') @@ -61,5 +62,6 @@ class TestEncryptedCinderVolumes(manager.EncryptionScenarioTest): def test_encrypted_cinder_volumes_cryptsetup(self): self.launch_instance() self.create_encrypted_volume('nova.volume.encryptors.' - 'cryptsetup.CryptsetupEncryptor') + 'cryptsetup.CryptsetupEncryptor', + volume_type='cryptsetup') self.attach_detach_volume() diff --git a/tempest/scenario/test_network_advanced_server_ops.py b/tempest/scenario/test_network_advanced_server_ops.py index bb668f7e4..3d6abff12 100644 --- a/tempest/scenario/test_network_advanced_server_ops.py +++ b/tempest/scenario/test_network_advanced_server_ops.py @@ -15,7 +15,6 @@ from oslo_log import log as logging from tempest_lib.common.utils import data_utils -from tempest_lib import decorators import testtools from tempest import config @@ -94,8 +93,8 @@ class TestNetworkAdvancedServerOps(manager.NetworkScenarioTest): self.servers_client.wait_for_server_status(self.server['id'], 'ACTIVE') self._check_network_connectivity() - @decorators.skip_because(bug="1323658") @test.idempotent_id('61f1aa9a-1573-410e-9054-afa557cab021') + @test.stresstest(class_setup_per='process') @test.services('compute', 'network') def test_server_connectivity_stop_start(self): self._setup_network_and_servers() @@ -147,7 +146,6 @@ class TestNetworkAdvancedServerOps(manager.NetworkScenarioTest): self.servers_client.resume_server(self.server['id']) self._wait_server_status_and_check_network_connectivity() - @decorators.skip_because(bug="1323658") @test.idempotent_id('719eb59d-2f42-4b66-b8b1-bb1254473967') @testtools.skipUnless(CONF.compute_feature_enabled.resize, 'Resize is not available.') diff --git a/tempest/scenario/test_network_basic_ops.py b/tempest/scenario/test_network_basic_ops.py index 7a604039f..bb1985374 100644 --- a/tempest/scenario/test_network_basic_ops.py +++ b/tempest/scenario/test_network_basic_ops.py @@ -101,13 +101,19 @@ class TestNetworkBasicOps(manager.NetworkScenarioTest): self.servers = [] def _setup_network_and_servers(self, **kwargs): + boot_with_port = kwargs.pop('boot_with_port', False) self.security_group = \ self._create_security_group(tenant_id=self.tenant_id) self.network, self.subnet, self.router = self.create_networks(**kwargs) self.check_networks() + self.port_id = None + if boot_with_port: + # create a port on the network and boot with that + self.port_id = self._create_port(self.network['id']).id + name = data_utils.rand_name('server-smoke') - server = self._create_server(name, self.network) + server = self._create_server(name, self.network, self.port_id) self._check_tenant_network_connectivity() floating_ip = self.create_floating_ip(server) @@ -141,7 +147,7 @@ class TestNetworkBasicOps(manager.NetworkScenarioTest): self.assertIn(self.router.id, seen_router_ids) - def _create_server(self, name, network): + def _create_server(self, name, network, port_id=None): keypair = self.create_keypair() self.keypairs[keypair['name']] = keypair security_groups = [{'name': self.security_group['name']}] @@ -152,6 +158,8 @@ class TestNetworkBasicOps(manager.NetworkScenarioTest): 'key_name': keypair['name'], 'security_groups': security_groups, } + if port_id is not None: + create_kwargs['networks'][0]['port'] = port_id server = self.create_server(name=name, create_kwargs=create_kwargs) self.servers.append(server) return server @@ -393,6 +401,8 @@ class TestNetworkBasicOps(manager.NetworkScenarioTest): "floating ip") @test.idempotent_id('1546850e-fbaa-42f5-8b5f-03d8a6a95f15') + @testtools.skipIf(CONF.baremetal.driver_enabled, + 'Baremetal relies on a shared physical network.') @test.attr(type='smoke') @test.services('compute', 'network') def test_connectivity_between_vms_on_different_networks(self): @@ -603,3 +613,39 @@ class TestNetworkBasicOps(manager.NetworkScenarioTest): self.check_public_network_connectivity( should_connect=True, msg="after updating " "admin_state_up of instance port to True") + + @test.idempotent_id('759462e1-8535-46b0-ab3a-33aa45c55aaa') + @testtools.skipUnless(CONF.compute_feature_enabled.preserve_ports, + 'Preserving ports on instance delete may not be ' + 'supported in the version of Nova being tested.') + @test.attr(type='smoke') + @test.services('compute', 'network') + def test_preserve_preexisting_port(self): + """Tests that a pre-existing port provided on server boot is not + deleted if the server is deleted. + + Nova should unbind the port from the instance on delete if the port was + not created by Nova as part of the boot request. + """ + # Setup the network, create a port and boot the server from that port. + self._setup_network_and_servers(boot_with_port=True) + _, server = self.floating_ip_tuple + self.assertIsNotNone(self.port_id, + 'Server should have been created from a ' + 'pre-existing port.') + # Assert the port is bound to the server. + port_list = self._list_ports(device_id=server['id'], + network_id=self.network['id']) + self.assertEqual(1, len(port_list), + 'There should only be one port created for ' + 'server %s.' % server['id']) + self.assertEqual(self.port_id, port_list[0]['id']) + # Delete the server. + self.servers_client.delete_server(server['id']) + self.servers_client.wait_for_server_termination(server['id']) + # Assert the port still exists on the network but is unbound from + # the deleted server. + port = self.network_client.show_port(self.port_id)['port'] + self.assertEqual(self.network['id'], port['network_id']) + self.assertEqual('', port['device_id']) + self.assertEqual('', port['device_owner']) diff --git a/tempest/services/baremetal/v1/json/baremetal_client.py b/tempest/services/baremetal/v1/json/baremetal_client.py index 09b6cd193..0c319f65b 100644 --- a/tempest/services/baremetal/v1/json/baremetal_client.py +++ b/tempest/services/baremetal/v1/json/baremetal_client.py @@ -131,7 +131,7 @@ class BaremetalClientJSON(base.BaremetalClient): return self._show_request('drivers', driver_name) @base.handle_errors - def create_node(self, chassis_id, **kwargs): + def create_node(self, chassis_id=None, **kwargs): """ Create a baremetal node with the specified parameters. diff --git a/tempest/services/compute/json/agents_client.py b/tempest/services/compute/json/agents_client.py index e17495f36..ff63f094d 100644 --- a/tempest/services/compute/json/agents_client.py +++ b/tempest/services/compute/json/agents_client.py @@ -16,7 +16,7 @@ import json import urllib from tempest.api_schema.response.compute import agents as common_schema -from tempest.api_schema.response.compute.v2 import agents as schema +from tempest.api_schema.response.compute.v2_1 import agents as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/aggregates_client.py b/tempest/services/compute/json/aggregates_client.py index 10955fd54..7f1c162d1 100644 --- a/tempest/services/compute/json/aggregates_client.py +++ b/tempest/services/compute/json/aggregates_client.py @@ -18,7 +18,7 @@ import json from tempest_lib import exceptions as lib_exc from tempest.api_schema.response.compute import aggregates as schema -from tempest.api_schema.response.compute.v2 import aggregates as v2_schema +from tempest.api_schema.response.compute.v2_1 import aggregates as v2_schema from tempest.common import service_client diff --git a/tempest/services/compute/json/availability_zone_client.py b/tempest/services/compute/json/availability_zone_client.py index 343c412fb..b541a2c5a 100644 --- a/tempest/services/compute/json/availability_zone_client.py +++ b/tempest/services/compute/json/availability_zone_client.py @@ -15,7 +15,8 @@ import json -from tempest.api_schema.response.compute.v2 import availability_zone as schema +from tempest.api_schema.response.compute.v2_1 import availability_zone \ + as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/certificates_client.py b/tempest/services/compute/json/certificates_client.py index 4a30f1e7b..43ec917cd 100644 --- a/tempest/services/compute/json/certificates_client.py +++ b/tempest/services/compute/json/certificates_client.py @@ -16,7 +16,7 @@ import json from tempest.api_schema.response.compute import certificates as schema -from tempest.api_schema.response.compute.v2 import certificates as v2schema +from tempest.api_schema.response.compute.v2_1 import certificates as v2schema from tempest.common import service_client diff --git a/tempest/services/compute/json/extensions_client.py b/tempest/services/compute/json/extensions_client.py index 09561b3de..5c6908530 100644 --- a/tempest/services/compute/json/extensions_client.py +++ b/tempest/services/compute/json/extensions_client.py @@ -15,7 +15,7 @@ import json -from tempest.api_schema.response.compute.v2 import extensions as schema +from tempest.api_schema.response.compute.v2_1 import extensions as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/fixed_ips_client.py b/tempest/services/compute/json/fixed_ips_client.py index 31cf5b272..dda940c22 100644 --- a/tempest/services/compute/json/fixed_ips_client.py +++ b/tempest/services/compute/json/fixed_ips_client.py @@ -15,7 +15,7 @@ import json -from tempest.api_schema.response.compute.v2 import fixed_ips as schema +from tempest.api_schema.response.compute.v2_1 import fixed_ips as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/flavors_client.py b/tempest/services/compute/json/flavors_client.py index 433c32524..25b1869a6 100644 --- a/tempest/services/compute/json/flavors_client.py +++ b/tempest/services/compute/json/flavors_client.py @@ -20,7 +20,7 @@ from tempest.api_schema.response.compute import flavors as common_schema from tempest.api_schema.response.compute import flavors_access as schema_access from tempest.api_schema.response.compute import flavors_extra_specs \ as schema_extra_specs -from tempest.api_schema.response.compute.v2 import flavors as v2schema +from tempest.api_schema.response.compute.v2_1 import flavors as v2schema from tempest.common import service_client diff --git a/tempest/services/compute/json/floating_ips_client.py b/tempest/services/compute/json/floating_ips_client.py index 0354ba4a1..5bad527bc 100644 --- a/tempest/services/compute/json/floating_ips_client.py +++ b/tempest/services/compute/json/floating_ips_client.py @@ -18,7 +18,7 @@ import urllib from tempest_lib import exceptions as lib_exc -from tempest.api_schema.response.compute.v2 import floating_ips as schema +from tempest.api_schema.response.compute.v2_1 import floating_ips as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/hosts_client.py b/tempest/services/compute/json/hosts_client.py index b06378b55..de925a979 100644 --- a/tempest/services/compute/json/hosts_client.py +++ b/tempest/services/compute/json/hosts_client.py @@ -16,7 +16,7 @@ import json import urllib from tempest.api_schema.response.compute import hosts as schema -from tempest.api_schema.response.compute.v2 import hosts as v2_schema +from tempest.api_schema.response.compute.v2_1 import hosts as v2_schema from tempest.common import service_client diff --git a/tempest/services/compute/json/hypervisor_client.py b/tempest/services/compute/json/hypervisor_client.py index 380b5cec8..bf4bc7f92 100644 --- a/tempest/services/compute/json/hypervisor_client.py +++ b/tempest/services/compute/json/hypervisor_client.py @@ -16,7 +16,7 @@ import json from tempest.api_schema.response.compute import hypervisors as common_schema -from tempest.api_schema.response.compute.v2 import hypervisors as v2schema +from tempest.api_schema.response.compute.v2_1 import hypervisors as v2schema from tempest.common import service_client diff --git a/tempest/services/compute/json/images_client.py b/tempest/services/compute/json/images_client.py index 0ceb6d1ad..1223fef7a 100644 --- a/tempest/services/compute/json/images_client.py +++ b/tempest/services/compute/json/images_client.py @@ -18,7 +18,7 @@ import urllib from tempest_lib import exceptions as lib_exc -from tempest.api_schema.response.compute.v2 import images as schema +from tempest.api_schema.response.compute.v2_1 import images as schema from tempest.common import service_client from tempest.common import waiters diff --git a/tempest/services/compute/json/instance_usage_audit_log_client.py b/tempest/services/compute/json/instance_usage_audit_log_client.py index 551d7515e..33ba76fd6 100644 --- a/tempest/services/compute/json/instance_usage_audit_log_client.py +++ b/tempest/services/compute/json/instance_usage_audit_log_client.py @@ -15,8 +15,8 @@ import json -from tempest.api_schema.response.compute.v2 import instance_usage_audit_logs \ - as schema +from tempest.api_schema.response.compute.v2_1 import \ + instance_usage_audit_logs as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/interfaces_client.py b/tempest/services/compute/json/interfaces_client.py index 0c5516c44..c3bfa99ca 100644 --- a/tempest/services/compute/json/interfaces_client.py +++ b/tempest/services/compute/json/interfaces_client.py @@ -16,9 +16,8 @@ import json import time -from tempest.api_schema.response.compute import interfaces as common_schema from tempest.api_schema.response.compute import servers as servers_schema -from tempest.api_schema.response.compute.v2 import interfaces as schema +from tempest.api_schema.response.compute.v2_1 import interfaces as schema from tempest.common import service_client from tempest import exceptions @@ -46,17 +45,19 @@ class InterfacesClientJSON(service_client.ServiceClient): resp, body = self.post('servers/%s/os-interface' % server, body=post_body) body = json.loads(body) + self.validate_response(schema.get_create_interfaces, resp, body) return service_client.ResponseBody(resp, body['interfaceAttachment']) def show_interface(self, server, port_id): resp, body = self.get('servers/%s/os-interface/%s' % (server, port_id)) body = json.loads(body) + self.validate_response(schema.get_create_interfaces, resp, body) return service_client.ResponseBody(resp, body['interfaceAttachment']) def delete_interface(self, server, port_id): resp, body = self.delete('servers/%s/os-interface/%s' % (server, port_id)) - self.validate_response(common_schema.delete_interface, resp, body) + self.validate_response(schema.delete_interface, resp, body) return service_client.ResponseBody(resp, body) def wait_for_interface_status(self, server, port_id, status): diff --git a/tempest/services/compute/json/keypairs_client.py b/tempest/services/compute/json/keypairs_client.py index 18729c340..722aefab7 100644 --- a/tempest/services/compute/json/keypairs_client.py +++ b/tempest/services/compute/json/keypairs_client.py @@ -16,7 +16,7 @@ import json from tempest.api_schema.response.compute import keypairs as common_schema -from tempest.api_schema.response.compute.v2 import keypairs as schema +from tempest.api_schema.response.compute.v2_1 import keypairs as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/limits_client.py b/tempest/services/compute/json/limits_client.py index 876990624..d2aaec6f3 100644 --- a/tempest/services/compute/json/limits_client.py +++ b/tempest/services/compute/json/limits_client.py @@ -15,7 +15,7 @@ import json -from tempest.api_schema.response.compute.v2 import limits as schema +from tempest.api_schema.response.compute.v2_1 import limits as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/quotas_client.py b/tempest/services/compute/json/quotas_client.py index ea0f423d1..89f4acddf 100644 --- a/tempest/services/compute/json/quotas_client.py +++ b/tempest/services/compute/json/quotas_client.py @@ -15,9 +15,9 @@ import json -from tempest.api_schema.response.compute.v2\ +from tempest.api_schema.response.compute.v2_1\ import quota_classes as classes_schema -from tempest.api_schema.response.compute.v2 import quotas as schema +from tempest.api_schema.response.compute.v2_1 import quotas as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/security_group_default_rules_client.py b/tempest/services/compute/json/security_group_default_rules_client.py index b370e006f..3bf3263c8 100644 --- a/tempest/services/compute/json/security_group_default_rules_client.py +++ b/tempest/services/compute/json/security_group_default_rules_client.py @@ -15,7 +15,7 @@ import json -from tempest.api_schema.response.compute.v2 import \ +from tempest.api_schema.response.compute.v2_1 import \ security_group_default_rule as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/security_groups_client.py b/tempest/services/compute/json/security_groups_client.py index 5aefa7b01..d8c8d6335 100644 --- a/tempest/services/compute/json/security_groups_client.py +++ b/tempest/services/compute/json/security_groups_client.py @@ -18,7 +18,7 @@ import urllib from tempest_lib import exceptions as lib_exc -from tempest.api_schema.response.compute.v2 import security_groups as schema +from tempest.api_schema.response.compute.v2_1 import security_groups as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/servers_client.py b/tempest/services/compute/json/servers_client.py index bd4fd0e79..bd276682a 100644 --- a/tempest/services/compute/json/servers_client.py +++ b/tempest/services/compute/json/servers_client.py @@ -21,7 +21,7 @@ import urllib from tempest_lib import exceptions as lib_exc from tempest.api_schema.response.compute import servers as common_schema -from tempest.api_schema.response.compute.v2 import servers as schema +from tempest.api_schema.response.compute.v2_1 import servers as schema from tempest.common import service_client from tempest.common import waiters from tempest import exceptions diff --git a/tempest/services/compute/json/tenant_networks_client.py b/tempest/services/compute/json/tenant_networks_client.py index c86c817c8..11251f6c8 100644 --- a/tempest/services/compute/json/tenant_networks_client.py +++ b/tempest/services/compute/json/tenant_networks_client.py @@ -14,7 +14,7 @@ import json -from tempest.api_schema.response.compute.v2 import tenant_networks as schema +from tempest.api_schema.response.compute.v2_1 import tenant_networks as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/tenant_usages_client.py b/tempest/services/compute/json/tenant_usages_client.py index bbc10515a..ff6e7a2db 100644 --- a/tempest/services/compute/json/tenant_usages_client.py +++ b/tempest/services/compute/json/tenant_usages_client.py @@ -16,7 +16,7 @@ import json import urllib -from tempest.api_schema.response.compute.v2 import tenant_usages as schema +from tempest.api_schema.response.compute.v2_1 import tenant_usages as schema from tempest.common import service_client diff --git a/tempest/services/compute/json/volumes_extensions_client.py b/tempest/services/compute/json/volumes_extensions_client.py index b2d5cf992..ba5921ea2 100644 --- a/tempest/services/compute/json/volumes_extensions_client.py +++ b/tempest/services/compute/json/volumes_extensions_client.py @@ -19,7 +19,7 @@ import urllib from tempest_lib import exceptions as lib_exc -from tempest.api_schema.response.compute.v2 import volumes as schema +from tempest.api_schema.response.compute.v2_1 import volumes as schema from tempest.common import service_client from tempest import exceptions diff --git a/tempest/stress/driver.py b/tempest/stress/driver.py index d095b53d6..e84d627d3 100644 --- a/tempest/stress/driver.py +++ b/tempest/stress/driver.py @@ -132,7 +132,14 @@ def stress_openstack(tests, duration, max_runs=None, stop_on_error=False): computes = _get_compute_nodes(controller, ssh_user, ssh_key) for node in computes: do_ssh("rm -f %s" % logfiles, node, ssh_user, ssh_key) + skip = False for test in tests: + for service in test.get('required_services', []): + if not CONF.service_available.get(service): + skip = True + break + if skip: + break if test.get('use_admin', False): manager = admin_manager else: diff --git a/tempest/stress/etc/stress-tox-job.json b/tempest/stress/etc/stress-tox-job.json index dffc469cb..9cee31672 100644 --- a/tempest/stress/etc/stress-tox-job.json +++ b/tempest/stress/etc/stress-tox-job.json @@ -15,5 +15,14 @@ "use_admin": false, "use_isolated_tenants": false, "kwargs": {} + }, + {"action": "tempest.stress.actions.unit_test.UnitTest", + "threads": 4, + "use_admin": false, + "use_isolated_tenants": false, + "required_services": ["neutron"], + "kwargs": {"test_method": "tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start", + "class_setup_per": "process"} } ] + diff --git a/tempest/tests/test_auth.py b/tempest/tests/test_auth.py index f54ff4ffe..eb63b3000 100644 --- a/tempest/tests/test_auth.py +++ b/tempest/tests/test_auth.py @@ -19,12 +19,10 @@ import datetime from oslotest import mockpatch from tempest import auth -from tempest import config from tempest import exceptions from tempest.services.identity.v2.json import token_client as v2_client from tempest.services.identity.v3.json import token_client as v3_client from tempest.tests import base -from tempest.tests import fake_config from tempest.tests import fake_credentials from tempest.tests import fake_http from tempest.tests import fake_identity @@ -46,8 +44,6 @@ class BaseAuthTestsSetUp(base.TestCase): def setUp(self): super(BaseAuthTestsSetUp, self).setUp() - self.useFixture(fake_config.ConfigFixture()) - self.stubs.Set(config, 'TempestConfigPrivate', fake_config.FakePrivate) self.fake_http = fake_http.fake_httplib2(return_type=200) self.stubs.Set(auth, 'get_credentials', fake_get_credentials) self.auth_provider = self._auth(self.credentials, diff --git a/tempest/thirdparty/boto/test.py b/tempest/thirdparty/boto/test.py index 043b230b0..cd35e7f0a 100644 --- a/tempest/thirdparty/boto/test.py +++ b/tempest/thirdparty/boto/test.py @@ -23,7 +23,6 @@ import boto from boto import ec2 from boto import exception from boto import s3 -import keystoneclient.exceptions from oslo_log import log as logging import six @@ -83,7 +82,7 @@ def decision_maker(): except lib_exc.Unauthorized: EC2_CAN_CONNECT_ERROR = "AWS credentials not set," +\ - " failed to get them even by keystoneclient" + " also failed to get it from keystone" except Exception as exc: EC2_CAN_CONNECT_ERROR = str(exc) @@ -98,7 +97,7 @@ def decision_maker(): _cred_sub_check(s3client.connection_data) except Exception as exc: S3_CAN_CONNECT_ERROR = str(exc) - except keystoneclient.exceptions.Unauthorized: + except lib_exc.Unauthorized: S3_CAN_CONNECT_ERROR = "AWS credentials not set," +\ " failed to get them even by keystoneclient" boto_logger.logger.setLevel(level) @@ -203,6 +202,9 @@ class BotoTestCase(tempest.test.BaseTestCase): super(BotoTestCase, cls).skip_checks() if not CONF.compute_feature_enabled.ec2_api: raise cls.skipException("The EC2 API is not available") + if not CONF.identity_feature_enabled.api_v2 or \ + not CONF.identity.auth_version == 'v2': + raise cls.skipException("Identity v2 is not available") @classmethod def setup_credentials(cls): @@ -277,7 +279,6 @@ class BotoTestCase(tempest.test.BaseTestCase): LOG.exception("Cleanup failed %s" % func_name) finally: del cls._resource_trash_bin[key] - cls.clear_isolated_creds() super(BotoTestCase, cls).resource_cleanup() # NOTE(afazekas): let the super called even on exceptions # The real exceptions already logged, if the super throws another, diff --git a/tools/check_uuid.py b/tools/check_uuid.py index 541e6c30c..34effe4c7 100644..100755 --- a/tools/check_uuid.py +++ b/tools/check_uuid.py @@ -1,3 +1,5 @@ +#!/usr/bin/env python + # Copyright 2014 Mirantis, Inc. # # Licensed under the Apache License, Version 2.0 (the "License"); you may @@ -119,8 +121,10 @@ class TestChecker(object): idempotent_id = None for decorator in test_node.decorator_list: if (hasattr(decorator, 'func') and - decorator.func.attr == DECORATOR_NAME and - decorator.func.value.id == DECORATOR_MODULE): + hasattr(decorator.func, 'attr') and + decorator.func.attr == DECORATOR_NAME and + hasattr(decorator.func, 'value') and + decorator.func.value.id == DECORATOR_MODULE): for arg in decorator.args: idempotent_id = ast.literal_eval(arg) return idempotent_id |