summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* [stable-only][cve] Check VMDK create-type against an allowed liststable/ussuriDan Smith2023-01-253-0/+86
| | | | | | | | | | | | NOTE(sbauza): Stable policy allows us to proactively merge a backport without waiting for the parent patch to be merged (exception to rule #4 in [1]. Marking [stable-only] in order to silence nova-tox-validate-backport [1] https://docs.openstack.org/project-team-guide/stable-branches.html#appropriate-fixes Conflicts vs victoria in: nova/conf/compute.py Related-Bug: #1996188 Change-Id: I5a399f1d3d702bfb76c067893e9c924904c8c360
* Merge "[compute] always set instance.host in post_livemigration" into ↵Zuul2023-01-163-6/+63
|\ | | | | | | stable/ussuri
| * [compute] always set instance.host in post_livemigrationSean Mooney2022-11-283-6/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change add a new _post_live_migration_update_host function that wraps _post_live_migration and just ensures that if we exit due to an exception instance.host is set to the destination host. when we are in _post_live_migration the guest has already started running on the destination host and we cannot revert. Sometimes admins or users will hard reboot the instance expecting that to fix everything when the vm enters the error state after the failed migrations. Previously this would end up recreating the instance on the source node leading to possible data corruption if the instance used shared storage. Change-Id: Ibc4bc7edf1c8d1e841c72c9188a0a62836e9f153 Partial-Bug: #1628606 (cherry picked from commit 8449b7caefa4a5c0728e11380a088525f15ad6f5) (cherry picked from commit 643b0c7d35752b214eee19b8d7298a19a8493f6b) (cherry picked from commit 17ae907569e45cc0f5c7da9511bb668a877b7b2e) (cherry picked from commit 15502ddedc23e6591ace4e73fa8ce5b18b5644b0) (cherry picked from commit 43c0e40d288960760a6eaad05cb9670e01ef40d0) (cherry picked from commit 0ac64bba8b7aba2fb358e00e970e88b32d26ef7e)
* | Merge "Adapt websocketproxy tests for SimpleHTTPServer fix" into stable/ussuriZuul2023-01-111-35/+26
|\ \ | |/ |/|
| * Adapt websocketproxy tests for SimpleHTTPServer fixmelanie witt2022-11-301-35/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In response to bug 1927677 we added a workaround to NovaProxyRequestHandler to respond with a 400 Bad Request if an open redirect is attempted: Ie36401c782f023d1d5f2623732619105dc2cfa24 I95f68be76330ff09e5eabb5ef8dd9a18f5547866 Recently in python 3.10.6, a fix has landed in cpython to respond with a 301 Moved Permanently to a sanitized URL that has had extra leading '/' characters removed. This breaks our existing unit tests which assume a 400 Bad Request as the only expected response. This adds handling of a 301 Moved Permanently response and asserts that the redirect location is the expected sanitized URL. Doing this instead of checking for a given python version will enable the tests to continue to work if and when the cpython fix gets backported to older python versions. While updating the tests, the opportunity was taken to commonize the code of two unit tests that were nearly identical. Related-Bug: #1927677 Closes-Bug: #1986545 Change-Id: I27441d15cc6fa2ff7715ba15aa900961aadbf54a (cherry picked from commit 15769b883ed4a86d62b141ea30d3f1590565d8e0) (cherry picked from commit 4a2b44c7cf55d1d79d5a2dd638bd0def3af0f5af) (cherry picked from commit 0e4a257e8636a979605c614a35e79ba47b74d870) (cherry picked from commit 3023e162e1a415ddaa70b4b8fbe24b1771dbe424) (cherry picked from commit 77bc3f004e7fe4077ea035c659630bedef1cfea1)
* | Adds a repoducer for post live migration failAmit Uniyal2022-11-282-3/+65
|/ | | | | | | | | | | | | | | | | | | | | Adds a regression test or repoducer for post live migration fail at destination, the possible casue can be fail to get instance network info or block device info changes: adds return server from _live_migrate in _integrated_helpers NOTE(auniyal): Differences * Replaced GlanceFixture with fake.stub_out_image_service in regression test, as GlanceFixture does not exist in Ussuri Related-Bug: #1628606 Change-Id: I48dbe0aae8a3943fdde69cda1bd663d70ea0eb19 (cherry picked from commit a20baeca1f5ebb0dfe9607335a6986e9ed0e1725) (cherry picked from commit 74a618a8118642c9fd32c4e0d502d12ac826affe) (cherry picked from commit 71e5a1dbcc22aeaa798d3d06ce392cf73364b8db) (cherry picked from commit 5efcc3f695e02d61cb8b881e009308c2fef3aa58) (cherry picked from commit ed1ea71489b60c0f95d76ab05f554cd046c60bac) (cherry picked from commit 6dda4f7ca3f25a11cd0178352ad24fe2e8b74785)
* For evacuation, ignore if task_state is not NoneAmit Uniyal2022-11-094-17/+35
| | | | | | | | | | | | ignore instance task state and continue with vm evacutaion. Closes-Bug: #1978983 Change-Id: I5540df6c7497956219c06cff6f15b51c2c8bc29d (cherry picked from commit db919aa15f24c0d74f3c5c0e8341fad3f2392e57) (cherry picked from commit 6d61fccb8455367aaa37ae7bddf3b8befd3c3d88) (cherry picked from commit 8e9aa71e1a4d3074a94911db920cae44334ba2c3) (cherry picked from commit 0b8124b99601e1aba492be8ed564f769438bd93d) (cherry picked from commit 3224ceb3fffc57d2375e5163d8ffbbb77529bc38)
* add regression test case for bug 1978983Amit Uniyal2022-11-092-1/+123
| | | | | | | | | | | | | | | | | | | | | | | | | | | This change add a repoducer test for evacuating a vm in the powering-off state Conflicts: nova/tests/functional/integrated_helpers.py nova/tests/functional/test_servers.py Difference: nova/tests/functional/regressions/test_bug_1978983.py NOTE(auniyal): Conflicts are due to the following changes that are not in Ussuri: * I147bf4d95e6d86ff1f967a8ce37260730f21d236 (Cyborg evacuate support) * Ia3d7351c1805d98bcb799ab0375673c7f1cb8848 (Functional tests for NUMA live migration) NOTE(auniyal): Differences * Replaced GlanceFixture with fake.stub_out_image_service in regression test, as GlanceFixture does not exist in Ussuri Related-Bug: #1978983 Change-Id: I5540df6c7497956219c06cff6f15b51c2c8bc299 (cherry picked from commit 5904c7f993ac737d68456fc05adf0aaa7a6f3018) (cherry picked from commit 6bd0bf00fca6ac6460d70c855eded3898cfe2401) (cherry picked from commit 1e0af92e17f878ce64bd16e428cb3c10904b0877) (cherry picked from commit b57b0eef218fd7604658842c9277aad782d11b45) (cherry picked from commit b6c877377f58ccaa797af3384b199002726745ea)
* Ignore plug_vifs on the ironic driverJulia Kreger2022-08-233-18/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When the nova-compute service starts, by default it attempts to startup instance configuration states for aspects such as networking. This is fine in most cases, and makes a lot of sense if the nova-compute service is just managing virtual machines on a hypervisor. This is done, one instance at a time. However, when the compute driver is ironic, the networking is managed as part of the physical machine lifecycle potentially all the way into committed switch configurations. As such, there is no need to attempt to call ``plug_vifs`` on every single instance managed by the nova-compute process which is backed by Ironic. Additionally, using ironic tends to manage far more physical machines per nova-compute service instance then when when operating co-installed with a hypervisor. Often this means a cluster of a thousand machines, with three controllers, will see thousands of un-needed API calls upon service start, which elongates the entire process and negatively impacts operations. In essence, nova.virt.ironic's plug_vifs call now does nothing, and merely issues a debug LOG entry when called. Closes-Bug: #1777608 Change-Id: Iba87cef50238c5b02ab313f2311b826081d5b4ab (cherry picked from commit 7f81cf28bf21ad2afa98accfde3087c83b8e269b) (cherry picked from commit eb6d70f02daa14920a2522e5c734a3775ea2ea7c) (cherry picked from commit f210115bcba3436b957a609cd388a13e6d77a638) (cherry picked from commit 35fb52f53fbd3f8290f775760a842d70f583fa67)
* [ironic] Minimize window for a resource provider to be lostJulia Kreger2022-08-173-1/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch is based upon a downstream patch which came up in discussion amongst the ironic community when some operators began discussing a case where resource providers had disappeared from a running deployment with several thousand baremetal nodes. Discussion amongst operators and developers ensued and we were able to determine that this was still an issue in the current upstream code and that time difference between collecting data and then reconciling the records was a source of the issue. Per Arun, they have been running this change downstream and had not seen any reoccurances of the issue since the patch was applied. This patch was originally authored by Arun S A G, and below is his original commit mesage. An instance could be launched and scheduled to a compute node between get_uuids_by_host() call and _get_node_list() call. If that happens the ironic node.instance_uuid may not be None but the instance_uuid will be missing from the instance list returned by get_uuids_by_host() method. This is possible because _get_node_list() takes several minutes to return in large baremetal clusters and a lot can happen in that time. This causes the compute node to be orphaned and associated resource provider to be deleted from placement. Once the resource provider is deleted it is never created again until the service restarts. Since resource provider is deleted subsequent boots/rebuilds to the same host will fail. This behaviour is visibile in VMbooter nodes because it constantly launches and deletes instances there by increasing the likelihood of this race condition happening in large ironic clusters. To reduce the chance of this race condition we call _get_node_list() first followed by get_uuids_by_host() method. Change-Id: I55bde8dd33154e17bbdb3c4b0e7a83a20e8487e8 Co-Authored-By: Arun S A G <saga@yahoo-inc.com> Related-Bug: #1841481 (cherry picked from commit f84d5917c6fb045f03645d9f80eafbc6e5f94bdd) (cherry picked from commit 0c36bd28ebd05ec0b1dbae950a24a2ecf339be00)
* [stable-only] Allow Train compute hosts to register with UssuriPierre Riteau2022-08-081-2/+2
| | | | | | | | | | | | | | The RPC version check in Ussuri should allow Train hosts. Use version 38 to identify them (next one after Stein which uses version 37). NOTE(elod.illes): this is a stable only fix as this bug was introduced in stable/ussrui with bug fix [1] as OLDEST_SUPPORTED_SERVICE_VERSION variable was not adjusted to the target branch. [1] https://review.opendev.org/c/openstack/nova/+/770764 Change-Id: Ieb821cb6efb3822974ad299839009e1ea8ab8db8 Closes-Bug: #1983263
* Merge "[rt] Apply migration context for incoming migrations" into stable/ussuriZuul2022-06-152-24/+40
|\
| * [rt] Apply migration context for incoming migrationsBalazs Gibizer2022-04-252-24/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a race condition between an incoming resize and an update_available_resource periodic in the resource tracker. The race window starts when the resize_instance RPC finishes and ends when the finish_resize compute RPC finally applies the migration context on the instance. In the race window, if the update_available_resource periodic is run on the destination node, then it will see the instance as being tracked on this host as the instance.node is already pointing to the dest. But the instance.numa_topology still points to the source host topology as the migration context is not applied yet. This leads to CPU pinning error if the source topology does not fit to the dest topology. Also it stops the periodic task and leaves the tracker in an inconsistent state. The inconsistent state only cleanup up after the periodic is run outside of the race window. This patch applies the migration context temporarily to the specific instances during the periodic to keep resource accounting correct. Conflicts: on resource_tracker: changed 'MigrationList.get_in_progress_and_error' call back to 'MigrationList.get_in_progress_by_host_and_node', since this change was only added by 255b3f2f918843ca5dd9b99e109ecd2189b6b749, and is not present in stable/ussuri. Change-Id: Icaad155e22c9e2d86e464a0deb741c73f0dfb28a Closes-Bug: #1953359 Closes-Bug: #1952915 (cherry picked from commit 32c1044d86a8d02712c8e3abdf8b3e4cff234a9c) (cherry picked from commit 1235dc324ebc1c6ac6dc94da0f45ffffcc546d2c) (cherry picked from commit 5f2f283a75243d2e2629d3c5f7e5ef4b3994972d) (cherry picked from commit d54bd316b331d439a26a7318ca68cab5f6280ab2)
* | Merge "Extend the reproducer for 1953359 and 1952915" into stable/ussuriZuul2022-06-151-17/+45
|\ \ | |/
| * Extend the reproducer for 1953359 and 1952915Balazs Gibizer2022-04-251-17/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch extends the original reproduction I4be429c56aaa15ee12f448978c38214e741eae63 to cover bug 1952915 as well as they have a common root cause. Change-Id: I57982131768d87e067d1413012b96f1baa68052b Related-Bug: #1953359 Related-Bug: #1952915 (cherry picked from commit 9f296d775d8f58fcbd03393c81a023268c7071cb) (cherry picked from commit 0411962938ae1de39f8dccb03efe4567f82ad671) (cherry picked from commit 94f17be190cce060ba8afcafbade4247b27b86f0) (cherry picked from commit 8d4487465b60cd165dc76dea5a9fdb3c4dbf5740)
* | Merge "Reproduce bug 1953359" into stable/ussuriZuul2022-06-153-2/+104
|\ \ | |/
| * Reproduce bug 1953359Balazs Gibizer2022-04-253-2/+104
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a functional test that reproduces a race between incoming migration and the update_available_resource periodic Conflicts: fixed conflict on test_numa_server to only add test case for 1953359 Fixes: - Changed 'start_compute' call to 'start_computes', since the former is not present in Ussuri - Added more memory to mock 'host_info', since the default would not fit the instance. Default was changed in later releases - Bumped the API version from 2.0 to 2.1 in the test, since microversion 2.47 is required creating an instance in a specific host and 2.0 is not supporting microversions. This was not needed for later releases, because the API version was bumped with some changes made by [1] - Reset the original microversion in 'create_server' after the POST request, so that subsequent calls are not affected [1] Later change that bumps API version on parent classes https://review.opendev.org/c/openstack/nova/+/741282 Co-Authored-By: Gabriel Silva Trevisan <gabriel.silvatrevisan@windriver.com> Change-Id: I4be429c56aaa15ee12f448978c38214e741eae63 Related-Bug: #1953359 (cherry picked from commit c59224d715a21998f40f72cf4e37efdc990e4d7e) (cherry picked from commit f0a6d946aaa6c30f826cfced75c2fb06fdb379a8) (cherry picked from commit d8859e4f95f5abb20c844d914f2716cba047630e) (cherry picked from commit e549fec76fd2015e6e21ee5138bf06142a71e71a)
* | [stable-only] Make sdk broken job non voting until it is fixedGhanshyam Mann2022-06-011-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | openstacksdk-functional-devstack is broken on stable/ussuri as it use master sdk and master constraints which are nto compatible with stable/ussuri one. We are discussing about how to solve it in ML and storyboard - http://lists.openstack.org/pipermail/openstack-discuss/2022-May/028763.html - https://storyboard.openstack.org/#!/story/2010057 Change-Id: Ib769f20e4937e1bfee35a9ef44ff47d7d51c7dea
* | Merge "[CI] Install dependencies for docs target" into stable/ussuriZuul2022-05-311-0/+1
|\ \
| * | [CI] Install dependencies for docs targetElod Illes2022-04-281-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When tox 'docs' target is called, first it installs the dependencies (listed in 'deps') in 'installdeps' phase, then it installs nova (with its requirements) in 'develop-inst' phase. In the latter case 'deps' is not used so that the constraints defined in 'deps' are not used. This could lead to failures on stable branches when new packages are released that break the build. To avoid this, the simplest solution is to pre-install requirements, i.e. add requirements.txt to 'docs' tox target. Conflicts: tox.ini NOTE(elod.illes): conflict is due to branch specific upper constraints file link. Change-Id: I4471d4488d336d5af0c23028724c4ce79d6a2031 (cherry picked from commit 494e8d7db6f8a3d1a952f657acab353787f57e04) (cherry picked from commit 1ac0d6984a43cddbb5a2f1a2f7bc115fd83517c9) (cherry picked from commit 64cc0848be9bf92d79e6fa7b424668d21321d593) (cherry picked from commit f66a570e946d980162a1313aa5a7e2ce5856a128) (cherry picked from commit bc92f05a6d6647e709a23a9d78b49f916874ef85)
* | | Merge "Define new functional test tox env for placement gate to run" into ↵Zuul2022-05-251-0/+10
|\ \ \ | | | | | | | | | | | | stable/ussuri
| * | | Define new functional test tox env for placement gate to runGhanshyam Mann2022-05-091-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have placement-nova-tox-functional-py38 job defined and run on placement gate[1] to run the nova functional test excluding api and notification _sample_tests, and db-related tests but that job skip those tests via tox_extra_args which is not right way to do as we currently facing error when tox_extra_args is included in tox siblings task - https://opendev.org/zuul/zuul-jobs/commit/c02c28a982da8d5a9e7b4ca38d30967f6cd1531d - https://zuul.openstack.org/build/a8c186b2c7124856ae32477f10e2b9a4 Let's define a new tox env which can exclude the required test in stestr command itself. Conflicts: tox.ini NOTE(melwitt): The conflict is because change I1d6a2986fcb0435cfabdd104d202b65329909d2b (Moving functional jobs to Victoria testing runtime) is not in Ussuri. The stestr option for the exclude regex also had to be changed because --exclude-regex is not in stestr 3.0.1, the version installed in Ussuri. [1] https://opendev.org/openstack/placement/src/commit/bd5b19c00e1ab293fc157f4699bc4f4719731c25/.zuul.yaml#L83 Change-Id: I20d6339a5203aed058f432f68e2ec1af57030401 (cherry picked from commit 7b063e4d0518af3e57872bc0288a94edcd33c19d) (cherry picked from commit 64f5c1cfb0e7223603c06e22a204716919d05294) (cherry picked from commit baf0d93e0fafcd992d37543aa9df3f6dc248a738) (cherry picked from commit d218250eb53791012f49825140e2592dab89e69c)
* | | | Merge "[stable-only] Drop lower-constraints job" into stable/ussuriZuul2022-05-183-184/+4
|\ \ \ \ | |/ / / |/| / / | |/ /
| * | [stable-only] Drop lower-constraints jobElod Illes2022-04-223-184/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During the PTG the TC discussed the topic and decided to drop the job completely. Since the latest job configuration broke all stable gate for nova (older than yoga) this is needed there to unblock our gates. For dropping the job on master let's wait to the resolution as the gate is not broken there, hence the patch is stable-only. Conflicts: .zuul.yaml lower-constraints.txt NOTE(elod.illes): conflict is due to branch specific settings (job was set to non-voting, lower constraints changes). Another change in .zuul.yaml is due to requirements-check job runs now against ubuntu-focal, that breaks tools/test_setup.sh script (fix exists in victoria: I97b0dcbb88c6ef7c22e3c55970211bed792bbd0d). This patch pins the job locally for ubuntu-bionic nodeset. Change-Id: I514f6b337ffefef90a0ce9ab0b4afd083caa277e (cherry picked from commit 15b72717f2f3bd79791b913f1b294a19ced47ca7) (cherry picked from commit ba3c5b81abce49fb86981bdcc0013068b54d4f61) (cherry picked from commit 327693af402e4dd0c03fe247c4cee7beaedd2852) (cherry picked from commit 8ff36f184dd7aedf9adfdbdf8845504557e2bef5)
* | | Fix the vGPU dynamic options raceSylvain Bauza2022-03-022-0/+19
|/ / | | | | | | | | | | | | | | | | | | | | | | As we lookup the existing dynamic options *before* creating them as _get_supported_vgpu_types() is called *before* compute init_host(), we need to make sure we call again the dynamic options registration within it. Change-Id: Ib9387c381d39fac389374c731b210815c6d4af03 Closes-Bug: #1900006 (cherry picked from commit 2bd8900d9b2600fba74341e249701051fb78eb37) (cherry picked from commit c7d9d6d9dd25e21ec76ceea294cdf1690686a086)
* | Ensure MAC addresses characters are in the same caseDmitriy Rabotyagov2022-01-202-3/+51
|/ | | | | | | | | | | | | | Currently neutron can report ports to have MAC addresses in upper case when they're created like that. In the meanwhile libvirt configuration file always stores MAC in lower case which leads to KeyError while trying to retrieve migrate_vif. Closes-Bug: #1945646 Change-Id: Ie3129ee395427337e9abcef2f938012608f643e1 (cherry picked from commit 6a15169ed9f16672c2cde1d7f27178bb7868c41f) (cherry picked from commit 63a6388f6a0265f84232731aba8aec1bff3c6d18) (cherry picked from commit 6c3d5de659e558e8f6ee353475b54ff3ca7240ee) (cherry picked from commit 28d0059c1f52e51add31bff50f1f6e443c938792)
* Merge "Fix error that cannot overwrite policy rule for 'forced_host'" into ↵ussuri-em21.2.4Zuul2021-11-031-1/+2
|\ | | | | | | stable/ussuri
| * Fix error that cannot overwrite policy rule for 'forced_host'ericxiett2021-07-261-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | When configure 'os_compute_api:servers:create:forced_host' to 'rule:admin_or_owner', but still doesn't allow. In nova/api/openstack/compute/servers.py#L669, the target is set to '{}' that is not equal None, so then it will not be set in nova/policy.py#L205. This patch configures the target param. Change-Id: I7a563386bd2f5d1930b5eb2cfc00425a19747e24 Closes-Bug: #1894975 (cherry picked from commit fd99a7ca0f56223d878142a58525d9ed64c84ed1)
* | Merge "Fix request path to query a resource provider by uuid" into stable/ussuriZuul2021-10-201-1/+1
|\ \
| * | Fix request path to query a resource provider by uuidTakashi Kajinami2021-08-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To query a resource provider by uuid, request path should look like /resource_providers?uuid=<uuid> istead of /resource_providers&uuid=<uuid> This change fixes the wrong path so that "nova-manage placement audit" command can look up the target resource provider properly. Closes-Bug: #1936278 Change-Id: I2ae7e9782316e3662e4e51e3f5ba0ef597bf281b (cherry picked from commit 1d3373dcf0a05d4a2c5b51fc1b74d41ec1bb1175) (cherry picked from commit 62a3fa4fff70a1d03998626406a71b7dc09da733) (cherry picked from commit 7dbceeceef0ca3657c72341375afc639be0b5c02)
* | | Merge "[stable-only] Pin virtualenv and setuptools" into stable/ussuriZuul2021-10-051-1/+5
|\ \ \
| * | | [stable-only] Pin virtualenv and setuptoolsBalazs Gibizer2021-10-011-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Setuptools 58.0 (bundled in virtualenv 20.8) breaks the installation of decorator 3.4.0. So this patch pins virtualenv to avoid the break. As the used 'require' feature was introduced in tox in version 3.2 [1], the required minversion has to be bumped, too. [1] https://tox.readthedocs.io/en/latest/config.html#conf-requires Change-Id: I26b2a14e0b91c0ab77299c3e4fbed5f7916fe8cf
* | | | Merge "address open redirect with 3 forward slashes" into stable/ussuri21.2.3Zuul2021-09-262-6/+35
|\ \ \ \
| * | | | address open redirect with 3 forward slashesSean Mooney2021-09-162-6/+35
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Ie36401c782f023d1d5f2623732619105dc2cfa24 was intended to address OSSA-2021-002 (CVE-2021-3654) however after its release it was discovered that the fix only worked for urls with 2 leading slashes or more then 4. This change adresses the missing edgecase for 3 leading slashes and also maintian support for rejecting 2+. Change-Id: I95f68be76330ff09e5eabb5ef8dd9a18f5547866 co-authored-by: Matteo Pozza Closes-Bug: #1927677 (cherry picked from commit 6fbd0b758dcac71323f3be179b1a9d1c17a4acc5) (cherry picked from commit 47dad4836a26292e9d34e516e1525ecf00be127c) (cherry picked from commit 9588cdbfd4649ea53d60303f2d10c5d62a070a07)
* | | | [stable-only] Set lower-constraints job as non-votingElod Illes2021-09-191-1/+4
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch sets the failing lower-constraints job to non-voting until final fix arrives for the failure. With the latest setuptools (bundled into python virtualenv) the lower constraints job started to fail on ussuri when it tries to install old version of 'decorator'. The error message is: error in decorator setup command: use_2to3 is invalid. The needed minimal version of 'decorator' would be 4.0.0, but it is only needed if the latest version of setuptools (virtualenv) is used. On old branches it would be good to pin setuptools (virtualenv), otherwise we can face similar issues. Until the final decision / fix, let's just set the lower-constraints job non-voting to unblock the gate. Change-Id: I01700882ca63f26975d4ecfd7d0a90ea95fe6614
* | | Merge "Reduce mocking in test_reject_open_redirect for compat" into ↵Zuul2021-09-141-14/+13
|\ \ \ | | | | | | | | | | | | stable/ussuri
| * | | Reduce mocking in test_reject_open_redirect for compatmelanie witt2021-07-311-14/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a followup for change Ie36401c782f023d1d5f2623732619105dc2cfa24 to reduce mocking in the unit test coverage for it. While backporting the bug fix, it was found to be incompatible with earlier versions of Python < 3.6 due to a difference in internal implementation [1]. This reduces the mocking in the unit test to be more agnostic to the internals of the StreamRequestHandler (ancestor of SimpleHTTPRequestHandler) and work across Python versions >= 2.7. Related-Bug: #1927677 [1] https://github.com/python/cpython/commit/34eeed42901666fce099947f93dfdfc05411f286 Change-Id: I546d376869a992601b443fb95acf1034da2a8f36 (cherry picked from commit 214cabe6848a1fdb4f5941d994c6cc11107fc4af) (cherry picked from commit 9c2f29783734cb5f9cb05a08d328c10e1d16c4f1) (cherry picked from commit 94e265f3ca615aa18de0081a76975019997b8709)
* | | | Fix 1vcpu error with multiqueue and vif_type=tapRodrigo Barbieri2021-08-313-13/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix for bug #1893263 introduced a regression where 1 vcpu instances would fail to build when paired with multiqueue-enabled images, in the scenario vif_type=tap. Solution is to not pass multiqueue parameter when instances.get_flavor().vcpus = 1. Closes-bug: #1939604 Change-Id: Iaccf2eeeb6e8bb80c658f51ce9ab4e8eb4093a55 (cherry picked from commit 7fc6fe6fae891eae42b36ccb9d69cd0f6d6db21d) (cherry picked from commit aa5b8d12bcacc01e5f9be45cc1eef24ac9efd2fc) (cherry picked from commit aaa56240b0311ad47ccccc3b7850ddc5b0a21702)
* | | | Merge "Dynamically archive FK related records in archive_deleted_rows" into ↵Zuul2021-08-313-95/+170
|\ \ \ \ | | | | | | | | | | | | | | | stable/ussuri
| * | | | Dynamically archive FK related records in archive_deleted_rowsmelanie witt2021-04-073-95/+170
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, it is possible to "partially archive" the database by running 'nova-manage db archive_deleted_rows' with --max_rows or by interrupting the archive process in any way. When this happens, it is possible to have archived a record with a foreign key relationship to a parent record (example: 'instance_extra' table record is archived while the 'instances' table record remains). When an instance's records become "split" in this way, any API request that can (1) access the deleted instance and (2) tries to access data that should be in a child table (example: the embedded flavor for an instance) will fail with an OrphanedObjectError and HTTP 500 to the user. Examples of APIs that are affected by this are the tenant usage APIs and listing of deleted instances as admin. In the tenant usage example, the API looks at deleted instances to calculate usage over a time period. It pulls deleted and non-deleted instances and does instance.get_flavor() to calculate their usage. The flavor data is expected to be present because expecteds_attrs=['flavor'] is used to do a join with the 'instance_extra' table and populate the instance object's flavor data. When get_flavor() is called, it tries to access the instance.flavor attribute (which hasn't been populated because the 'instance_extra' record is gone). That triggers a lazy-load of the flavor which loads the instance from the database again with expected_attrs=['flavor'] again which doesn't populate instance.flavor (again) because the 'instance_extra' record is gone. Then the Instance._load_flavor code intentionally orphans the instance record to avoid triggering lazy-loads while it attempts to populate instance.flavor, instance.new_flavor, and instance.old_flavor. Finally, another lazy-load is triggered (because instance.flavor is still not populated) and fails with OrphanedObjectError. One way to solve this problem is to make it impossible for archive_deleted_records to orphan records that are related by foreign key relationships. The approach is to process parent tables first (opposite of today where we process child tables first) and find all of the tables that refer to it by foreign keys, create and collect insert/delete statements for those child records, and then put them all together in a single database transaction to archive all related records "atomically". The idea is that if anything were to interrupt the transaction (errors or other) it would roll back and keep all the related records together. Either all or archived or none are archived. This changes the logic of the per table archive to discover tables that refer to the table by foreign keys and generates insert/delete query statements to execute in the same database transaction as the table archive itself. The extra records archived along with the table are added to the rows_archived result. The existing code for "archiving records if instance is deleted" also has to be removed along with this because the new logic does the same thing dynamically and makes it obsolete. Finally, some assertions in the unit tests need to be changed or removed because they were assuming certain types of archiving failures due to foreign key constraint violations that can no longer occur with the new dynamic logic for archiving child records. Closes-Bug: #1837995 Change-Id: Ie653e5ec69d16ae469f1f8171fee85aea754edff (cherry picked from commit becb94ae643ab4863daa564783646921b4a2b372) (cherry picked from commit 7b4f4796478941eafa9c0997f7ef03293c442d94)
* | | | | Merge "Add functional test for bug 1837995" into stable/ussuriZuul2021-08-311-0/+47
|\ \ \ \ \ | |/ / / /
| * | | | Add functional test for bug 1837995melanie witt2021-04-061-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a functional test to reproduce the bug where when calling an API that will examine a deleted instance record (examples: GET /servers/detail and GET /os-simple-tenant-usage), if that instance's related records have only been partially archived (example: 'instance_extra' record was archived but 'instances' record has not been archived yet) then the API call fails with the following error: Unexpected API Error ... <class 'nova.exception.OrphanedObjectError'> (HTTP 500) Related-Bug: #1837995 Change-Id: Ia225ee86536248d2540de0d7a1cd012333e708a2 (cherry picked from commit 3d4e53f832792cc49001f1588164e28b84c16d7a) (cherry picked from commit 21241b38dd4bc2c74a69aed489ada5c611395920)
* | | | | Merge "Centralize sqlite FK constraint enforcement" into stable/ussuriZuul2021-08-314-30/+22
|\ \ \ \ \ | |/ / / /
| * | | | Centralize sqlite FK constraint enforcementmelanie witt2021-04-064-30/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's a TODO in the code to centralize foreign key constraint enforcement for sqlite for unit and functional tests and we're missing enforcement of FK constraints in a couple of test classes that should have it. This resolves the TODO and turns on FK constraint enforcement where it is missing. Do this to enhance testing in preparation for a proposed change to the database archiving logic later in this patch series. Change-Id: Idcf026d020e63e4e6ece1db46e4cdc7b7742b76f (cherry picked from commit 172024db7174bdac05a3d3172c645b0db83ae41e) (cherry picked from commit 382d64ea36e5cf3b47929fee0b93539d2f126975)
* | | | | Merge "Initialize global data separately and run_once in WSGI app init" into ↵Zuul2021-08-215-2/+256
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | stable/ussuri
| * | | | | Initialize global data separately and run_once in WSGI app initHervé Beraud2021-07-085-2/+256
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | NOTE(melwitt): This is a combination of two changes to avoid intermittent test failure that was introduced by the original bug fix, and was fixed by change I2bd360dcc6501feea7baf02d4510b282205fc061. We have discovered that if an exception is raised at any point during the running of the init_application WSGI script in an apache/mod_wsgi Daemon Mode environment, it will prompt apache/mod_wsgi to re-run the script without starting a fresh python process. Because we initialize global data structures during app init, subsequent runs of the script blow up as some global data do *not* support re-initialization. It is anyway not safe to assume that init of global data is safe to run multiple times. This mod_wsgi behavior appears to be a special situation that does not behave the same as a normal reload in Daemon Mode as the script file is being reloaded upon failure instead of the daemon process being shutdown and restarted as described in the documentation [1]. In order to handle this situation, we can move the initialization of global data structures to a helper method that is decorated to run only once per python interpreter instance. This way, we will not attempt to re-initialize global data that are not safe to init more than once. Co-Authored-By: Michele Baldessari <michele@redhat.com> Co-Authored-By: melanie witt <melwittt@gmail.com> Conflicts: nova/test.py NOTE(melwitt): The conflict is because change I1fea14d5be10bb4e884f52e0ae8be722519ddd3f (Poison netifaces.interfaces() in tests) is not in Ussuri. Closes-Bug: #1882094 [1] https://modwsgi.readthedocs.io/en/develop/user-guides/reloading-source-code.html#reloading-in-daemon-mode Reset global wsgi app state in unit test Since I2bd360dcc6501feea7baf02d4510b282205fc061 there is a global state set during the wsgi_app init making our unit test cases non-deterministic based on the order of them. This patch makes sure that the global state is reset for each test case. Closes-Bug: #1921098 (cherry picked from commit bc2c19bb2db901af0c48d34fb15a335f4e343361) Change-Id: I2bd360dcc6501feea7baf02d4510b282205fc061 (cherry picked from commit 7c9edc02eda45aafbbb539b759e6b92f7aeb5ea8) (cherry picked from commit e3085fa6310ddeaafa493c3f718aab0ce64f0994)
* | | | | | Merge "Honor [neutron]http_retries in the manual client" into stable/ussuriZuul2021-08-202-0/+3
|\ \ \ \ \ \ | |_|_|_|/ / |/| | | | |
| * | | | | Honor [neutron]http_retries in the manual clientmelanie witt2021-06-072-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change Ifb3afb13aff7e103c2e80ade817d0e63b624604a added a nova side config option for specifying neutron client retries that maps to the ksa connect_retries config option to provide parity with the cinder and glance clients that have nova side config options. That change missed passing CONF.neutron.http_retries to the manual client used for calling the port binding API. This sets the connect_retries attribute on the manual ksa client so http_retries will be honored. Closes-Bug: #1929886 Related-Bug: #1866937 Change-Id: I8296e4be9f0706fab043451b856efadbb7bd41f6 (cherry picked from commit 56eb253e9febccf721df6bca4eb851ad26cb70a6) (cherry picked from commit 46aa3f4ec769e948d9eb73604bf9b66f4b0230b0) (cherry picked from commit f20346bc00a30c914cbefb48009db776f8e00b09)
* | | | | | Merge "Move 'check-cherry-picks' test to gate, n-v check" into stable/ussuriZuul2021-08-033-8/+23
|\ \ \ \ \ \ | |_|_|_|/ / |/| | | | |
| * | | | | Move 'check-cherry-picks' test to gate, n-v checkStephen Finucane2021-06-183-8/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This currently runs in the 'check' pipeline, as part of the pep8 job, which causes otherwise perfectly valid backports to report as failing CI. There's no reason a stable core shouldn't be encouraged to review these patches: we simply want to prevent them *merging* before their parent(s). Resolve this conflict by moving the check to separate voting job in the 'gate' pipeline as well as a non-voting job in the 'check' pipeline to catch more obvious issues. NOTE(lyarwood): Conflicts as I1d6a2986fcb0435cfabdd104d202b65329909d2b and If6758c83a6bd95aefb7703529cbccd14e3adfee2 are not present on stable/ussuri. Conflicts: .zuul.yaml Change-Id: Id3e4452883f6a3cf44ff58b39ded82e882e28c23 Signed-off-by: Stephen Finucane <stephenfin@redhat.com> (cherry picked from commit 98b01c9a59df4912f5a162c2c52d1f00c84d24c2) (cherry picked from commit fef0305abefbf165fecb883f03bce97f525a790a) (cherry picked from commit b7677ae08ae151858ecb0e67039e54bb3df89700)
View Lorry Controller Status