summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
| * | | | Move 'check-cherry-picks' test to gate, n-v checkStephen Finucane2021-06-183-8/+23
| |/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This currently runs in the 'check' pipeline, as part of the pep8 job, which causes otherwise perfectly valid backports to report as failing CI. There's no reason a stable core shouldn't be encouraged to review these patches: we simply want to prevent them *merging* before their parent(s). Resolve this conflict by moving the check to separate voting job in the 'gate' pipeline as well as a non-voting job in the 'check' pipeline to catch more obvious issues. NOTE(lyarwood): Conflict as I672904e9bfb45a66a82331063c7d49c4bc0439df isn't present on stable/victoria. Conflicts: .zuul.yaml Change-Id: Id3e4452883f6a3cf44ff58b39ded82e882e28c23 Signed-off-by: Stephen Finucane <stephenfin@redhat.com> (cherry picked from commit 98b01c9a59df4912f5a162c2c52d1f00c84d24c2) (cherry picked from commit fef0305abefbf165fecb883f03bce97f525a790a)
* | | | Merge "Reject open redirection in the console proxy" into stable/victoriaZuul2021-07-263-0/+76
|\ \ \ \
| * | | | Reject open redirection in the console proxymelanie witt2021-05-173-0/+76
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Our console proxies (novnc, serial, spice) run in a websockify server whose request handler inherits from the python standard SimpleHTTPRequestHandler. There is a known issue [1] in the SimpleHTTPRequestHandler which allows open redirects by way of URLs in the following format: http://vncproxy.my.domain.com//example.com/%2F.. which if visited, will redirect a user to example.com. We can intercept a request and reject requests that pass a redirection URL beginning with "//" by implementing the SimpleHTTPRequestHandler.send_head() method containing the vulnerability to reject such requests with a 400 Bad Request. This code is copied from a patch suggested in one of the issue comments [2]. Closes-Bug: #1927677 [1] https://bugs.python.org/issue32084 [2] https://bugs.python.org/issue32084#msg306545 Conflicts: nova/console/websocketproxy.py nova/tests/unit/console/test_websocketproxy.py NOTE(melwitt): The conflicts are because the following changes are not in Victoria: Ib2c406327fef2fb4868d8050fc476a7d17706e23 (Remove six.moves) I58b0382c86d4ef798572edb63d311e0e3e6937bb (Refactor and rename test_tcp_rst_no_compute_rpcapi) Change-Id: Ie36401c782f023d1d5f2623732619105dc2cfa24 (cherry picked from commit 781612b33282ed298f742c85dab58a075c8b793e) (cherry picked from commit 470925614223c8dd9b1233f54f5a96c02b2d4f70)
* | | | | Merge "Stop leaking ceph df cmd in RBD utils" into stable/victoria22.2.2Zuul2021-07-092-1/+14
|\ \ \ \ \
| * | | | | Stop leaking ceph df cmd in RBD utilsTobias Urdin2021-06-252-1/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the ceph df command fails in the get_pool_info method of RBD utils the actual command executed if seen by the users in the fault error message. This hides the command behind a StorageError exception and logs the exception instead of leaking it to the users. Change-Id: I6e3a73f2e04d1a7636daf96d5af73c9cf2fbe220 Closes-Bug: 1926978 (cherry picked from commit 86af7feed06f08ddb3ef65122089216708d53a06) (cherry picked from commit 5ede75c65edbcb27557831ae6f5c3a9f81f23a0e)
* | | | | | Merge "Honor [neutron]http_retries in the manual client" into stable/victoriaZuul2021-07-082-0/+3
|\ \ \ \ \ \ | |/ / / / / |/| | | | |
| * | | | | Honor [neutron]http_retries in the manual clientmelanie witt2021-06-072-0/+3
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change Ifb3afb13aff7e103c2e80ade817d0e63b624604a added a nova side config option for specifying neutron client retries that maps to the ksa connect_retries config option to provide parity with the cinder and glance clients that have nova side config options. That change missed passing CONF.neutron.http_retries to the manual client used for calling the port binding API. This sets the connect_retries attribute on the manual ksa client so http_retries will be honored. Closes-Bug: #1929886 Related-Bug: #1866937 Change-Id: I8296e4be9f0706fab043451b856efadbb7bd41f6 (cherry picked from commit 56eb253e9febccf721df6bca4eb851ad26cb70a6) (cherry picked from commit 46aa3f4ec769e948d9eb73604bf9b66f4b0230b0)
* | | | | Merge "Initialize global data separately and run_once in WSGI app init" into ↵Zuul2021-06-255-2/+256
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | stable/victoria
| * | | | | Initialize global data separately and run_once in WSGI app initHervé Beraud2021-06-085-2/+256
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | NOTE(melwitt): This is a combination of two changes to avoid intermittent test failure that was introduced by the original bug fix, and was fixed by change I2bd360dcc6501feea7baf02d4510b282205fc061. We have discovered that if an exception is raised at any point during the running of the init_application WSGI script in an apache/mod_wsgi Daemon Mode environment, it will prompt apache/mod_wsgi to re-run the script without starting a fresh python process. Because we initialize global data structures during app init, subsequent runs of the script blow up as some global data do *not* support re-initialization. It is anyway not safe to assume that init of global data is safe to run multiple times. This mod_wsgi behavior appears to be a special situation that does not behave the same as a normal reload in Daemon Mode as the script file is being reloaded upon failure instead of the daemon process being shutdown and restarted as described in the documentation [1]. In order to handle this situation, we can move the initialization of global data structures to a helper method that is decorated to run only once per python interpreter instance. This way, we will not attempt to re-initialize global data that are not safe to init more than once. Co-Authored-By: Michele Baldessari <michele@redhat.com> Co-Authored-By: melanie witt <melwittt@gmail.com> Conflicts: nova/api/openstack/wsgi_app.py NOTE(melwitt): The conflict is because change If4783adda92da33d512d7c2834f0bb2e2a9b9654 (Support sys.argv in wsgi app) is not in Victoria. Closes-Bug: #1882094 [1] https://modwsgi.readthedocs.io/en/develop/user-guides/reloading-source-code.html#reloading-in-daemon-mode Reset global wsgi app state in unit test Since I2bd360dcc6501feea7baf02d4510b282205fc061 there is a global state set during the wsgi_app init making our unit test cases non-deterministic based on the order of them. This patch makes sure that the global state is reset for each test case. Closes-Bug: #1921098 (cherry picked from commit bc2c19bb2db901af0c48d34fb15a335f4e343361) Change-Id: I2bd360dcc6501feea7baf02d4510b282205fc061 (cherry picked from commit 7c9edc02eda45aafbbb539b759e6b92f7aeb5ea8)
* | | | | | Merge "rbd: Get rbd_utils unit tests running again" into stable/victoriaZuul2021-06-252-1/+1
|\ \ \ \ \ \
| * | | | | | rbd: Get rbd_utils unit tests running againmelanie witt2021-06-032-1/+1
| | |/ / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Awhile back, change I25baf5edd25d9e551686b7ed317a63fd778be533 moved rbd_utils out from the libvirt driver and into a central location under nova/storage. This move missed adding a __init__.py file under nova/tests/unit/storage, so unit test discovery wasn't picking up the rbd_utils tests and couldn't run them. This adds a __init__.py file under nova/tests/unit/storage to get the tests running again. This also fixes a small bug introduced by change I3032bbe6bd2d6acc9ba0f0cac4d00ed4b4464ceb in RbdTestCase.setUp() that passed nonexistent self.images_rbd_pool to self.flags. It should be self.rbd_pool. Closes-Bug: #1928007 Change-Id: Ic03a5336abdced883f62f395690c0feac12075c8 (cherry picked from commit 8b647f1b3f56879be221b3925570790a1e0e77f8) (cherry picked from commit 8f018d754d5c55e432cd51df99278382b527283e)
* | | | | | [neutron] Get only ID and name of the SGs from NeutronSlawek Kaplonski2021-06-211-1/+7
| |_|/ / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During the VM booting process Nova asks Neutron for the security groups of the project. If there are no any fields specified, Neutron will prepare list of security groups with all fields, including rules. In case if project got many SGs, it may take long time as rules needs to be loaded separately for each SG on Neutron's side. During booting of the VM, Nova really needs only "id" and "name" of the security groups so this patch limits request to only those 2 fields. This lazy loading of the SG rules was introduced in Neutron in [1] and [2]. [1] https://review.opendev.org/#/c/630401/ [2] https://review.opendev.org/#/c/637407/ Related-Bug: #1865223 Change-Id: I15c3119857970c38541f4de270bd561166334548 (cherry picked from commit 388498ac5fa15ed8deef06ec061ea47e4a1b7377) (cherry picked from commit 4f49545afaf3cd453796d48ba96b9a82d11c01bf)
* | | | | Merge "Ignore PCI devices with 32bit domain" into stable/victoriaZuul2021-06-175-16/+77
|\ \ \ \ \
| * | | | | Ignore PCI devices with 32bit domainBalazs Gibizer2021-05-175-16/+77
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Nova and QEMU[1] supports PCI devices with a PCI address that has 16 bit domain. However there are hypervisors that reports PCI addresses with 32 bit domain. While today we cannot assign these to guests this should not prevent the nova-compute service to start. This patch changes the PCI manager to ignore such PCI devices. Please note that this patch does not change fact that nova does not allow specifying PCI addresses with 32bit domain in the [pci]/passthrough_whitelist configuration. Such configuration is still rejected at nova-compute service startup. Closes-Bug: #1897528 [1] https://github.com/qemu/qemu/blob/f2a1cf9180f63e88bb38ff21c169da97c3f2bad5/hw/core/qdev-properties.c#L993 Change-Id: I59a0746b864610b6a314078cf5661d3d2b84b1d4 (cherry picked from commit 8c9d6fc8f073cde78b79ae259c9915216f5d59b0)
* | | | | | Merge "Reproduce bug 1897528" into stable/victoriaZuul2021-06-171-0/+37
|\ \ \ \ \ \ | |/ / / / /
| * | | | | Reproduce bug 1897528Balazs Gibizer2021-05-171-0/+37
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The nova-compute fails to start if the hypervisor has PCI addresses 32bit domain. Change-Id: I48dcb7faa17fe9f8346445a1746cff5845baf358 Related-Bug: #1897528 (cherry picked from commit 976ac722d36439d16ea4ec1bf5037c482c89ef55)
* | | | | Error anti-affinity violation on migrationsRodrigo Barbieri2021-06-093-24/+211
| |/ / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Error-out the migrations (cold and live) whenever the anti-affinity policy is violated. This addresses violations when multiple concurrent migrations are requested. Added detection on: - prep_resize - check_can_live_migration_destination - pre_live_migration The improved method of detection now locks based on group_id and considers other migrations in-progress as well. Closes-bug: #1821755 Change-Id: I32e6214568bb57f7613ddeba2c2c46da0320fabc (cherry picked from commit 33c8af1f8c46c9c37fcc28fb3409fbd3a78ae39f) (cherry picked from commit 8b62a4ec9bf617dfb2da046c25a9f76b33516508)
* | | | zuul: Replace grenade and nova-grenade-multinode with grenade-multinodeLee Yarwood2021-06-039-512/+16
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If2608406776e0d5a06b726e65b55881e70562d18 dropped the single node grenade job from the integrated-gate-compute template as it duplicates the existing grenade-multinode job. However it doesn't remove the remianing single node grenade job still present in the Nova project. This change replaces the dsvm based nova-grenade-multinode job with the zuulv3 native grenade-multinode based job. Various legacy playbooks and hook scripts are also removed as they are no longer used. Note that this does result in a loss of coverage for ceph that should be replaced as soon as a zuulv3 native ceph based multinode job is available. Change-Id: I02b2b851a74f24816d2f782a66d94de81ee527b0 (cherry picked from commit 91e53e4c2b90ea57aeac4ec522dd7c8c54961d09) (cherry picked from commit c45bedd98d50af865d727b7456c974c8e27bff8b)
* | | Merge "guestfs: With libguestfs >= v1.41.1 decode returned bytes to string" ↵22.2.1Zuul2021-04-303-4/+11
|\ \ \ | | | | | | | | | | | | into stable/victoria
| * | | guestfs: With libguestfs >= v1.41.1 decode returned bytes to stringLee Yarwood2021-04-253-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | libguestfs >= v1.41.1 and commit 0ee02e0117527 changed the return type of read_file from string to bytes. https://github.com/libguestfs/libguestfs/commit/0ee02e0117527b86a31b2a88a14994ce7f15571f As we don't check the version of libguestfs installed this change handles both the original behaviour where a string is returned and the newer behaviour by decoding any returned bytes to a string. Closes-Bug: #1882421 Change-Id: I1c12b2903c1e5bf3a88394493456ad33233f3cd8 (cherry picked from commit 606d588e3eca1d88ad26b4c2cfa3f2e1d5ed553e) (cherry picked from commit e98935f705a931e3ae666820387a419be1622b64)
* | | | Merge "Dynamically archive FK related records in archive_deleted_rows" into ↵Zuul2021-04-273-95/+170
|\ \ \ \ | | | | | | | | | | | | | | | stable/victoria
| * | | | Dynamically archive FK related records in archive_deleted_rowsmelanie witt2021-04-073-95/+170
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, it is possible to "partially archive" the database by running 'nova-manage db archive_deleted_rows' with --max_rows or by interrupting the archive process in any way. When this happens, it is possible to have archived a record with a foreign key relationship to a parent record (example: 'instance_extra' table record is archived while the 'instances' table record remains). When an instance's records become "split" in this way, any API request that can (1) access the deleted instance and (2) tries to access data that should be in a child table (example: the embedded flavor for an instance) will fail with an OrphanedObjectError and HTTP 500 to the user. Examples of APIs that are affected by this are the tenant usage APIs and listing of deleted instances as admin. In the tenant usage example, the API looks at deleted instances to calculate usage over a time period. It pulls deleted and non-deleted instances and does instance.get_flavor() to calculate their usage. The flavor data is expected to be present because expecteds_attrs=['flavor'] is used to do a join with the 'instance_extra' table and populate the instance object's flavor data. When get_flavor() is called, it tries to access the instance.flavor attribute (which hasn't been populated because the 'instance_extra' record is gone). That triggers a lazy-load of the flavor which loads the instance from the database again with expected_attrs=['flavor'] again which doesn't populate instance.flavor (again) because the 'instance_extra' record is gone. Then the Instance._load_flavor code intentionally orphans the instance record to avoid triggering lazy-loads while it attempts to populate instance.flavor, instance.new_flavor, and instance.old_flavor. Finally, another lazy-load is triggered (because instance.flavor is still not populated) and fails with OrphanedObjectError. One way to solve this problem is to make it impossible for archive_deleted_records to orphan records that are related by foreign key relationships. The approach is to process parent tables first (opposite of today where we process child tables first) and find all of the tables that refer to it by foreign keys, create and collect insert/delete statements for those child records, and then put them all together in a single database transaction to archive all related records "atomically". The idea is that if anything were to interrupt the transaction (errors or other) it would roll back and keep all the related records together. Either all or archived or none are archived. This changes the logic of the per table archive to discover tables that refer to the table by foreign keys and generates insert/delete query statements to execute in the same database transaction as the table archive itself. The extra records archived along with the table are added to the rows_archived result. The existing code for "archiving records if instance is deleted" also has to be removed along with this because the new logic does the same thing dynamically and makes it obsolete. Finally, some assertions in the unit tests need to be changed or removed because they were assuming certain types of archiving failures due to foreign key constraint violations that can no longer occur with the new dynamic logic for archiving child records. Closes-Bug: #1837995 Conflicts: nova/db/sqlalchemy/api.py NOTE(melwitt): The conflict is because change I23bb9e539d08f5c6202909054c2dd49b6c7a7a0e (Remove six.text_type (1/2)) is not in Victoria. Change-Id: Ie653e5ec69d16ae469f1f8171fee85aea754edff (cherry picked from commit becb94ae643ab4863daa564783646921b4a2b372)
* | | | | Merge "Add functional test for bug 1837995" into stable/victoriaZuul2021-04-271-0/+47
|\ \ \ \ \ | |/ / / /
| * | | | Add functional test for bug 1837995melanie witt2021-04-061-0/+47
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a functional test to reproduce the bug where when calling an API that will examine a deleted instance record (examples: GET /servers/detail and GET /os-simple-tenant-usage), if that instance's related records have only been partially archived (example: 'instance_extra' record was archived but 'instances' record has not been archived yet) then the API call fails with the following error: Unexpected API Error ... <class 'nova.exception.OrphanedObjectError'> (HTTP 500) Related-Bug: #1837995 Change-Id: Ia225ee86536248d2540de0d7a1cd012333e708a2 (cherry picked from commit 3d4e53f832792cc49001f1588164e28b84c16d7a)
* | | | | Merge "Centralize sqlite FK constraint enforcement" into stable/victoriaZuul2021-04-274-30/+22
|\ \ \ \ \ | |/ / / / | | / / / | |/ / / |/| | |
| * | | Centralize sqlite FK constraint enforcementmelanie witt2021-04-064-30/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's a TODO in the code to centralize foreign key constraint enforcement for sqlite for unit and functional tests and we're missing enforcement of FK constraints in a couple of test classes that should have it. This resolves the TODO and turns on FK constraint enforcement where it is missing. Do this to enhance testing in preparation for a proposed change to the database archiving logic later in this patch series. Conflicts: nova/test.py NOTE(melwitt): The conflicts are because of the following changes not in Victoria: * Ib2c406327fef2fb4868d8050fc476a7d17706e23 (Remove six.moves) * Ide65686cf02463045f5c32771ca949802b19636f (Remove six.binary_type/integer_types/string_types) Change-Id: Idcf026d020e63e4e6ece1db46e4cdc7b7742b76f (cherry picked from commit 172024db7174bdac05a3d3172c645b0db83ae41e)
* | | | Fix the vGPU dynamic options raceSylvain Bauza2021-04-062-0/+19
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As we lookup the existing dynamic options *before* creating them as _get_supported_vgpu_types() is called *before* compute init_host(), we need to make sure we call again the dynamic options registration within it. Change-Id: Ib9387c381d39fac389374c731b210815c6d4af03 Closes-Bug: #1900006 (cherry picked from commit 2bd8900d9b2600fba74341e249701051fb78eb37)
* | | Merge "api: Log os-resetState as an instance action" into stable/victoriaZuul2021-03-314-2/+44
|\ \ \
| * | | api: Log os-resetState as an instance actionLee Yarwood2021-01-264-2/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It appears this was missed as this action never hits the compute API and is fully handled within the API itself. This change simply records an instance action directly in the API for this admin action providing a breadcrumb to operators and users. Closes-Bug: #1911924 Change-Id: Ifd9657f3357bc39cb904caa65168d38def8c9184 (cherry picked from commit 534b4f38958af2fbf4392e445ddb1b2bad4179ed)
* | | | Merge "Add config parameter 'live_migration_scheme' to live migration with ↵Zuul2021-03-221-1/+10
|\ \ \ \ | | | | | | | | | | | | | | | tls guide" into stable/victoria
| * | | | Add config parameter 'live_migration_scheme' to live migration with tls guideJosephine Seifert2021-03-191-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the config option 'live_migration_scheme = tls' to the secure live migration guide. To let the live migration use the qemu native tls, some configuration of the compute nodes is needed. The guide describes this but misses the 'live_migration_scheme' config option. It is necessary to set 'live_migration_scheme' to tls to use the connection uri for encrypted traffic. Without this parameter everything seems to work, but the unencrypted tcp-connection is still used for the live migration. Closes-Bug: #1919357 Change-Id: Ia5130d411706bf7e1c983156158011a3bc6d5cd6 (cherry picked from commit 5d5ff82bab1dfb12e6223446b1cf20db6a348f38)
* | | | | Merge "libvirt: Skip encryption metadata lookups if secret already exists on ↵Zuul2021-03-213-1/+42
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | host" into stable/victoria
| * | | | | libvirt: Skip encryption metadata lookups if secret already exists on hostLee Yarwood2020-12-213-1/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When connecting an encrypted volume to a host the _attach_encryptor method will be called in order to either call a legacy os-brick encryptor *or* configure a libvirt secret used by libvirt and QEMU to natively decrypt LUKSv1 encrypted volumes. To create this libvirt secret the configured key manager will be queried to provide and then decode the associated secret before this is stashed within libvirt. This change simply skips the above when an existing libvirt secret associated with the target volume is found on the host already. While this obviously optimises basic instance lifecycle flows such as a simple power off and on it additionally resolves a more convoluted use case when the ``[DEFAULT]/resume_guests_state_on_host_boot`` configurable is enabled. In this case the compute service has no request context with which to query the key manager when attempting to restart instances with encrypted volumes attached. As a result any attempt by the compute service to restart an instance with an attached encrypted volume would previously fail. Closes-Bug: #1905701 Change-Id: Ia2007bc63ef09931ea0197cef29d6a5614ed821a (cherry picked from commit a107a5099e86c3da80a6feeca6f840d5a3ad11b9)
* | | | | | Merge "Use absolute path during qemu img rebase" into stable/victoriaZuul2021-03-202-6/+33
|\ \ \ \ \ \
| * | | | | | Use absolute path during qemu img rebaseBalazs Gibizer2021-01-192-6/+33
| | |_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During an assisted volume snapshot delete request from Cinder nova removes the snapshot from the backing file chain. During that nova checks the existence of such file. However in some cases (see the bug report) the path is relative and therefore os.path.exists fails. This patch makes sure that nova uses the volume absolute path to make the backing file path absolute as well. Closes-Bug #1885528 Change-Id: I58dca95251b607eaff602783fee2fc38e2421944 (cherry picked from commit b9333125790682f9d60bc74fdbb12a098565e7c2)
* | | | | | Merge "libvirt: Use specific user when probing encrypted rbd disks during ↵Zuul2021-03-192-2/+9
|\ \ \ \ \ \ | |_|_|/ / / |/| | | | | | | | | | | extend" into stable/victoria
| * | | | | libvirt: Use specific user when probing encrypted rbd disks during extendLee Yarwood2021-02-012-2/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I0c3f14100a18107f7e416293f3d4fcc641ce5e55 introduced new logic when extending LUKSv1 encrypted rbd volumes. As part of this qemu-img is used to probe the rbd volume to determine the size of the LUKSv1 header. The URI used to point to the rbd volume did not provide a user and assumed that n-cpu/privsep would have access to the admin keyring. This isn't always the case in most environments and would result in a failure to probe the disk when the admin keyring wasn't available. This change resolves this by appending the `id:$username` option to the end of the URI provided to qemu-img using the `auth_username` found in the connection_info from Cinder. Closes-Bug: #1913575 Change-Id: Ia6d6dcdd7042f2aef6b3abeb5cd0f7525678a3b7 (cherry picked from commit b62a1abd616cea7fdfcad18e1cf548767d67b6dd)
* | | | | | Handle instance = None in _local_delete_cleanup22.2.0melanie witt2021-02-233-36/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change I4d3193d8401614311010ed0e055fcb3aaeeebaed added some additional local delete cleanup to prevent leaking of placement allocations. The change introduced a regression in our "delete while booting" handling as the _local_delete_cleanup required a valid instance object to do its work and in two cases, we could have instance = None from _lookup_instance if we are racing with a create request and the conductor has deleted the instance record while we are in the middle of processing the delete request. This handles those scenarios by doing two things: (1) Changing the _local_delete_cleanup and _update_queued_for_deletion methods to take an instance UUID instead of a full instance object as they really only need the UUID to do their work (2) Saving a copy of the instance UUID before doing another instance lookup which might return None and passing that UUID to the _local_delete_cleanup and _update_queued_for_deletion methods Closes-Bug: #1914777 Change-Id: I03cf285ad83e09d88cdb702a88dfed53c01610f8 (cherry picked from commit 123f6262f63477d3f50dfad09688978e044bd9e0)
* | | | | | Add regression test for bug 1914777melanie witt2021-02-231-0/+151
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds two tests to cover a regression where racing create and delete requests could result in a user receiving a 500 error when attempting to delete an instance: Unexpected exception in API method: AttributeError: 'NoneType' object has no attribute 'uuid' Related-Bug: #1914777 Change-Id: I8249c572c6f727ef4ca434843106b9b57c47e585 (cherry picked from commit f7975d640ce1e9fa06d045d35177f07451716f0c)
* | | | | | Merge "tools: Allow check-cherry-picks.sh to be disabled by an env var" into ↵Zuul2021-02-182-0/+7
|\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | stable/victoria
| * | | | | | tools: Allow check-cherry-picks.sh to be disabled by an env varLee Yarwood2021-02-172-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The checks performed by this script aren't always useful to downstream consumers of the repo so allow them to disable the script without having to make changes to tox.ini. NOTE(lyarwood): This backport has Ie8a672fd21184c810bfe9c0e3a49582189bf2111 squashed into it to ensure the introduced env var is passed into the pep8 tox env. tox: Add passenv DISABLE_CHERRY_PICK_CHECK to pep8 I4f551dc4b57905cab8aa005c5680223ad1b57639 introduced the environment variable to disable the check-cherry-pick.sh script but forgot to allow it to be passed into the pep8 tox env. Change-Id: I4f551dc4b57905cab8aa005c5680223ad1b57639 (cherry picked from commit 610396f8ad5fe8c3abb7731fcd42c81e5246a938)
* | | | | | | Merge "Disallow CONF.compute.max_disk_devices_to_attach = 0" into ↵Zuul2021-02-173-1/+19
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | stable/victoria
| * | | | | | | Disallow CONF.compute.max_disk_devices_to_attach = 0melanie witt2021-01-293-1/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The CONF.compute.max_disk_devices_to_attach option controls the maximum number of disk devices allowed to attach to an instance. If it is set to 0, it will literally allow no disk device for instances, preventing them from being able to boot. This adds a note to the config option help to call this out and changes nova-compute to raise InvalidConfiguration during init_host if [compute]max_disk_devices_to_attach has been set to 0. The nova-compute service will fail to start if the option is set to 0. Note: there doesn't appear to be any way to disallow particular values in a oslo.config IntOpt other than the min/max values. Here we need the min value to be -1 to represent unlimited. There is a 'choices' kwarg available but that is only for enumerating valid values and we need to allow any integer >= 1 as well. Change-Id: I6e30468bc28f661ddc17937ab1de04a706f05063 Closes-Bug: #1897950 (cherry picked from commit 25a632a4e1daa1941a6297ddb51088972f23ce6d)
* | | | | | | | Merge "only wait for plugtime events in pre-live-migration" into stable/victoriaZuul2021-02-143-11/+48
|\ \ \ \ \ \ \ \
| * | | | | | | | only wait for plugtime events in pre-live-migrationSean Mooney2021-02-033-11/+48
| | |_|_|/ / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change modifies _get_neutron_events_for_live_migration to filter the event to just the subset that will be sent at plug-time. Currently neuton has a bug where by the dhcp agent send a network-vif-plugged event during live migration after we update the port profile with "migrating-to:" this cause a network-vif-plugged event to be sent for configuration where vif_plugging in nova/os-vif is a noop. when that is corrected the current logic in nova cause the migration to time out as its waiting for an event that will never arrive. This change filters the set of events we wait for to just the plug time events. This backport has squashed the follow up change I37c712ba9a0ab88c44d10f80da3254ab6c463a68 to remove the unused migration paramater orginally added by this patch to _get_neutron_events_for_live_migration Related-Bug: #1815989 Closes-Bug: #1901707 Change-Id: Id2d8d72d30075200d2b07b847c4e5568599b0d3b (cherry picked from commit 8b33ac064456482158b23c2a2d52f819ebb4c60e)
* | | | | | | | Merge "Prevent archiving of pci_devices records because of 'instance_uuid'" ↵Zuul2021-02-032-1/+20
|\ \ \ \ \ \ \ \ | |_|_|_|/ / / / |/| | | | | | | | | | | | | | | into stable/victoria
| * | | | | | | Prevent archiving of pci_devices records because of 'instance_uuid'melanie witt2020-10-192-1/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently in the archive_deleted_rows code, we will attempt to clean up "residue" of deleted instance records by assuming any table with a 'instance_uuid' column represents data tied to an instance's lifecycle and delete such records. This behavior poses a problem in the case where an instance has a PCI device allocated and someone deletes the instance. The 'instance_uuid' column in the pci_devices table is used to track the allocation association of a PCI with an instance. There is a small time window during which the instance record has been deleted but the PCI device has not yet been freed from a database record perspective as PCI devices are freed during the _complete_deletion method in the compute manager as part of the resource tracker update call. Records in the pci_devices table are anyway not related to the lifecycle of instances so they should not be considered residue to clean up if an instance is deleted. This adds a condition to avoid archiving pci_devices on the basis of an instance association. Closes-Bug: #1899541 Change-Id: Ie62d3566230aa3e2786d129adbb2e3570b06e4c6 (cherry picked from commit 1c256cf774693e2395ae8fe4a7a2f416a7aeb03a)
* | | | | | | | Merge "Fallback to same-cell resize with qos ports" into stable/victoria22.1.0Zuul2021-01-306-32/+74
|\ \ \ \ \ \ \ \ | |_|_|/ / / / / |/| | | | | | |
| * | | | | | | Fallback to same-cell resize with qos portsBalazs Gibizer2021-01-116-32/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The cross-cell resize code does not consider neutron ports with resource request. To avoid migration failures this patch makes nova to fall back to same cell resize if the instance has neutron ports with resource request. The stable only difference is due to change in the reproduction patch Id91d2e817ef6bd21124bb840bdb098054e9753b8 on stable. Change-Id: Icaad4b2375b491c8a7e87fb6f4977ae2e13e8190 Closes-Bug: #1907522 Closes-Bug: #1907511 (cherry picked from commit 1c056c384d1b89dd877e1aa3108b1797d7881223)
* | | | | | | | Merge "Use subqueryload() instead of joinedload() for (system_)metadata" ↵Zuul2021-01-272-1/+24
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | into stable/victoria