summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* | | | | Merge "Add a workaround config toggle to refuse ceph image upload" into ↵Zuul2021-06-254-0/+104
|\ \ \ \ \ | |/ / / / |/| | | | | | | | | stable/train
| * | | | Add a workaround config toggle to refuse ceph image uploadDan Smith2020-10-214-0/+104
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a compute node is backed by ceph, and the image is not clone-able in that same ceph, nova will try to download the image from glance and upload it to ceph itself. This is nice in that it "just works", but it also means we store that image in ceph in an extremely inefficient way. In a glance multi-store case with multiple ceph clusters, the user is currently required to make sure that the image they are going to use is stored in a backend local to the compute node they land on, and if they do not (or can not), then nova will do this non-COW inefficient copy of the image, which is likely not what the operator expects. Per the discussion at the Denver PTG, this adds a workaround flag which allows the operators to direct nova to *not* do this behavior and instead refuse to boot the instance entirely. Conflicts: nova/conf/workarounds.py NOTE(melwitt): The conflict is because this patch originally landed on ussuri and change If874f018ea996587e178219569c2903c2ee923cf (Reserve DISK_GB resource for the image cache) landed afterward and was backported to stable/train. Related-Bug: #1858877 Change-Id: I069b6b1d28eaf1eee5c7fb8d0fdef9c0c229a1bf (cherry picked from commit 80191e6d828cf823ce3aa7c6176da5e531694900)
* | | | | Merge "Add missing exception" into stable/trainZuul2021-06-252-1/+10
|\ \ \ \ \ | |_|/ / / |/| | | |
| * | | | Add missing exceptionStephen Finucane2020-11-192-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Change Idd49b0c70caedfcd42420ffa2ac926a6087d406e added support for discovery of PMEM devices by the libvirt driver. Some error handling code in this was expected to raise a 'GetPMEMNamespacesFailed' exception, however, a typo meant the exception was actually called 'GetPMEMNamespaceFailed' (singular). This exception was later removed in change I6fd027fb51823b8a8a24ed7b864a2191c4e8e8c0 because it had no references. Re-introduce the exception, this time with the correct name, and add some unit tests to prevent us regressing. Conflicts: nova/exception.py NOTE(stephenfin): Conflicts are because change I6fd027fb51823b8a8a24ed7b864a2191c4e8e8c0 doesn't exist on this branch, meaning the misnamed exception still exists and simply needs to be renamed. Change-Id: I3b597a46314a1b29a952fc0f7a9c4537341e37b8 Signed-off-by: Stephen Finucane <stephenfin@redhat.com> Closes-Bug: #1904446 (cherry picked from commit 160ed6ff652b80cd0a86d41dc3f66c15cee66290) (cherry picked from commit 82d415d200e39a5c11fe134e9ef62e427988f2ba) (cherry picked from commit 8f65de96a07623c6a8a9cd49db942ad6d4d3ab03)
* | | | | Merge "Set migrate_data.vifs only when using multiple port bindings" into ↵Zuul2021-06-047-23/+140
|\ \ \ \ \ | | | | | | | | | | | | | | | | | | stable/train
| * | | | | Set migrate_data.vifs only when using multiple port bindingsroot2021-04-077-23/+140
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In the rocky cycle nova was enhanced to support the multiple port binding live migration workflow when neutron supports the binding-extended API extension. When the migration_data object was extended to support multiple port bindings, populating the vifs field was used as a sentinel to indicate that the new workflow should be used. In the train release I734cc01dce13f9e75a16639faf890ddb1661b7eb (SR-IOV Live migration indirect port support) broke the semantics of the migrate_data object by unconditionally populating the vifs field This change restores the rocky semantics, which are depended on by several parts of the code base, by only conditionally populating vifs if neutron supports multiple port bindings. Changes to patch: - unit/virt/libvirt/fakelibvirt.py: Include partial pick from change Ia3d7351c1805d98bcb799ab0375673c7f1cb8848 to add the jobStats, complete_job and fail_job to fakelibvirt. The full change was not cherry-picked as it was part of the numa aware live migration feature in Victoria. - renamed import of nova.network.neutron to nova.network.neutronv2.api - mocked nova.virt.libvirt.guest.Guest.get_job_info to return fakelibvirt.VIR_DOMAIN_JOB_COMPLETED - replaced from urllib import parse as urlparse with import six.moves.urllib.parse as urlparse for py2.7 Conflicts: nova/tests/functional/regressions/test_bug_1888395.py nova/tests/unit/compute/test_compute.py nova/tests/unit/compute/test_compute_mgr.py nova/tests/unit/virt/test_virt_drivers.py Co-Authored-By: Sean Mooney <work@seanmooney.info> Change-Id: Ia00277ac8a68a635db85f9e0ce2c6d8df396e0d8 Closes-Bug: #1888395 (cherry picked from commit b8f3be6b3c5af91d215b4a0cecb9be098e8d8799) (cherry picked from commit afa843c8a7e128489a8245ed7a1b391c022b3305)
* | | | | | Merge "add functional regression test for bug #1888395" into stable/trainZuul2021-06-043-1/+130
|\ \ \ \ \ \ | |/ / / / /
| * | | | | add functional regression test for bug #1888395Sean Mooney2021-04-023-1/+130
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds a funcitonal regression test that assert the broken behavior when trying to live migrate with a neutron backend that does not support multiple port bindings. Conflicts/Changes: nova/tests/functional/regressions/test_bug_1888395.py: - specify api major version to allow block_migration 'auto' - use TempDir fixture for instances path - worked around lack of create_server and start_computes in integrated helpers in train by inlining the behavior in setUp and test_live_migrate - reverted to python2 compatiable super() syntax nova/tests/unit/virt/libvirt/fake_imagebackend.py: - include portion of change Ia3d7351c1805d98bcb799ab0375673c7f1cb8848 which stubs out the is_file_in_instance_path method. That was included in a feature patch set so just pulling the necessary bit. Change-Id: I470a016d35afe69809321bd67359f466c3feb90a Partial-Bug: #1888395 (cherry picked from commit 71bc6fc9b89535679252ffe5a737eddad60e4102) (cherry picked from commit bea55a7d45bdc97679cf08c9faec789cfc90de27)
* | | | | | Merge "Update image_base_image_ref during rebuild." into stable/trainZuul2021-05-233-0/+12
|\ \ \ \ \ \
| * | | | | | Update image_base_image_ref during rebuild.Alexandre Arents2021-04-273-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In different location we assume system_metadata.image_base_image_ref exists, because it is set during instance creation in method _populate_instance_for_create But once instance is rebuild, all system_metadata image property a dropped and replace by new image property and without setting back image_base_image_ref. This change propose to set image_base_image_ref during rebuild. In specific case of shelve/unshelve in Qcow2 backend, image_base_image_ref is used to rebase disk image, so we ensure this property is set as instance may have been rebuild before the fix was apply. Related-Bug: #1732428 Closes-Bug: #1893618 Change-Id: Ia3031ea1f7db8b398f02d2080ca603ded8970200 (cherry picked from commit fe52b6c25bebdd1b459c7a59fbb8d9f6de200c9d) (cherry picked from commit 560414036d9f2ebcfeb8626029d0bd849d6cad44)
* | | | | | | Merge "Rebase qcow2 images when unshelving an instance" into stable/trainZuul2021-05-232-0/+106
|\ \ \ \ \ \ \ | |/ / / / / /
| * | | | | | Rebase qcow2 images when unshelving an instanceAlexandre Arents2021-04-272-0/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During unshelve, instance is spawn with image created by shelve and is deleted just after, instance.image_ref still point to the original instance build image. In qcow2 environment, this is an issue because instance backing file don't match anymore instance.image_ref and during live-migration/resize, target host will fetch image corresponding to instance.image_ref involving instance corruption. This change fetches original image and rebase instance disk on it. This avoid image_ref mismatch and bring back storage benefit to keep common image in cache. If original image is no more available in glance, backing file is merged into disk(flatten), ensuring instance integrity during next live-migration/resize operation. NOTE(lyarwood): Test conflicts caused by If56842da51688 not being present in stable/train. Conflicts: nova/tests/unit/virt/libvirt/test_driver.py Change-Id: I1a33fadf0b7439cf06c06cba2bc06df6cef0945b Closes-Bug: #1732428 (cherry picked from commit 8953a689467f8c3e996086392251de67953a45ba) (cherry picked from commit 7003618884b5e033f822685541da6bc7955fe55a)
* | | | | | | Merge "Make _rebase_with_qemu_img() generic" into stable/trainZuul2021-05-222-24/+37
|\ \ \ \ \ \ \ | |/ / / / / /
| * | | | | | Make _rebase_with_qemu_img() genericAlexandre Arents2021-04-272-24/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move volume_delete related logic away from this method, in order to make it generic and usable elsewhere. NOTE(lyarwood): Conflict caused by I52fbbcac9dc386f24ee81b3321dd0d8355e01976 landing in stbale/ussuri. Conflicts: nova/tests/unit/virt/libvirt/test_driver.py Change-Id: I17357d85f845d4160cb7c7784772530a1e92af76 Related-Bug: #1732428 (cherry picked from commit ce2203456660083119cdbb7e73c1ad15e6e0a074) (cherry picked from commit 2e89699c3301bf801784c637b6919752fcd3503f)
* | | | | | | Merge "libvirt: 'video.vram' property must be an integer" into stable/traintrain-em20.6.1Zuul2021-04-282-7/+11
|\ \ \ \ \ \ \ | |/ / / / / / |/| | | | | |
| * | | | | | libvirt: 'video.vram' property must be an integerStephen Finucane2020-10-122-7/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The 'vram' property of the 'video' device must be an integer else libvirt will spit the dummy out, e.g. libvirt.libvirtError: XML error: cannot parse video vram '8192.0' The division operator in Python 3 results in a float, not an integer like in Python 2. Use the truncation division operator instead. Change-Id: Iebf678c229da4f455459d068cafeee5f241aea1f Signed-off-by: Stephen Finucane <stephenfin@redhat.com> Closes-Bug: #1896496 (cherry picked from commit f2ca089bce842127e7d0644b38a11da9278db8ea) (cherry picked from commit fd7c66f61c0b07cff82d93e0f884777949170857) (cherry picked from commit 121e481a880b48ce800aafdeca0b8b086e2cddf3)
* | | | | | | Merge "libvirt: Increase incremental and max sleep time during device ↵Zuul2021-04-241-2/+2
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | detach" into stable/train
| * | | | | | | libvirt: Increase incremental and max sleep time during device detachLee Yarwood2021-04-131-2/+2
| |/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug #1894804 outlines how DEVICE_DELETED events were often missing from QEMU on Focal based OpenStack CI hosts as originally seen in bug #1882521. This has eventually been tracked down to some undefined QEMU behaviour when a new device_del QMP command is received while another is still being processed, causing the original attempt to be aborted. We hit this race in slower OpenStack CI envs as n-cpu rather crudely retries attempts to detach devices using the RetryDecorator from oslo.service. The default incremental sleep time currently being tight enough to ensure QEMU is still processing the first device_del request on these slower CI hosts when n-cpu asks libvirt to retry the detach, sending another device_del to QEMU hitting the above behaviour. Additionally we have also seen the following check being hit when testing with QEMU >= v5.0.0. This check now rejects overlapping device_del requests in QEMU rather than aborting the original: https://github.com/qemu/qemu/commit/cce8944cc9efab47d4bf29cfffb3470371c3541b This change aims to avoid this situation entirely by raising the default incremental sleep time between detach requests from 2 seconds to 10, leaving enough time for the first attempt to complete. The overall maximum sleep time is also increased from 30 to 60 seconds. Future work will aim to entirely remove this retry logic with a libvirt event driven approach, polling for the the VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED and VIR_DOMAIN_EVENT_ID_DEVICE_REMOVAL_FAILED events before retrying. Finally, the cleanup of unused arguments in detach_device_with_retry is left for a follow up change in order to keep this initial change small enough to quickly backport. Closes-Bug: #1882521 Related-Bug: #1894804 Change-Id: Ib9ed7069cef5b73033351f7a78a3fb566753970d (cherry picked from commit dd1e6d4b0cee465fd89744e306fcd25228b3f7cc) (cherry picked from commit 4819f694b2e2d5688fdac7e850f1a6c592253d6b) (cherry picked from commit f32286c62e11d43f6b01bc82caacf588447862e5)
* | | | | | | Merge "Improve error log when snapshot fails" into stable/trainZuul2021-04-173-0/+12
|\ \ \ \ \ \ \ | |_|/ / / / / |/| | | | | |
| * | | | | | Improve error log when snapshot failsVu Tran2021-03-253-0/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If snapshot creation via glance fails due to lack of space or over quota, we want to have a clearer error message. Change-Id: Ic9133f6bc14d4fe766d37a438bf52c33e89da768 Closes-Bug: #1613770 (cherry picked from commit 024bf10d8aec5e58111793a9652b16682eb61b7c)
* | | | | | | Merge "Test for disabling greendns" into stable/trainZuul2021-03-311-0/+45
|\ \ \ \ \ \ \
| * | | | | | | Test for disabling greendnsArtom Lifshitz2020-11-191-0/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In commit 7c1d964faa we fixed how we disable greendns. This patch adds a test for this. It also lays down the groundwork for future tests of how we manage eventlet's monkeypatching. How and what eventlet monkeypatches can be controlled by environment variables that are processed by eventlet at import-time (for exmaple, EVENTLET_NO_GREENDNS). Nova manages all of this in nova.monkey_patch. Therefore, nova.monkey_patch must be the first thing to import eventlet. As nova.tests.functional.__init__ imports nova.monkey_patch, our new test can go in the functional tree. Related-bug: 1895322 Change-Id: I5b6c45b7b9a9eca3c13ecfaa5f50942922b69270 (cherry picked from commit 6f35e4fd2afdb8e26682fc5bd73dbe23236d25c7) (cherry picked from commit 9ac794b93b827dde1d9545b88fd7913b7c00dacb)
* | | | | | | | Add config parameter 'live_migration_scheme' to live migration with tls guideJosephine Seifert2021-03-241-1/+10
| |/ / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds the config option 'live_migration_scheme = tls' to the secure live migration guide. To let the live migration use the qemu native tls, some configuration of the compute nodes is needed. The guide describes this but misses the 'live_migration_scheme' config option. It is necessary to set 'live_migration_scheme' to tls to use the connection uri for encrypted traffic. Without this parameter everything seems to work, but the unencrypted tcp-connection is still used for the live migration. Closes-Bug: #1919357 Change-Id: Ia5130d411706bf7e1c983156158011a3bc6d5cd6 (cherry picked from commit 5d5ff82bab1dfb12e6223446b1cf20db6a348f38) (cherry picked from commit 276b8db5afd945bfc56ccfadd25bcce1b6df9cb7) (cherry picked from commit a968289b1f89b52d7396b66294f768c732583849)
* | | | | | | Merge "Prevent archiving of pci_devices records because of 'instance_uuid'" ↵Zuul2021-03-222-1/+20
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | into stable/train
| * | | | | | | Prevent archiving of pci_devices records because of 'instance_uuid'melanie witt2020-11-022-1/+20
| |/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently in the archive_deleted_rows code, we will attempt to clean up "residue" of deleted instance records by assuming any table with a 'instance_uuid' column represents data tied to an instance's lifecycle and delete such records. This behavior poses a problem in the case where an instance has a PCI device allocated and someone deletes the instance. The 'instance_uuid' column in the pci_devices table is used to track the allocation association of a PCI with an instance. There is a small time window during which the instance record has been deleted but the PCI device has not yet been freed from a database record perspective as PCI devices are freed during the _complete_deletion method in the compute manager as part of the resource tracker update call. Records in the pci_devices table are anyway not related to the lifecycle of instances so they should not be considered residue to clean up if an instance is deleted. This adds a condition to avoid archiving pci_devices on the basis of an instance association. Closes-Bug: #1899541 Change-Id: Ie62d3566230aa3e2786d129adbb2e3570b06e4c6 (cherry picked from commit 1c256cf774693e2395ae8fe4a7a2f416a7aeb03a) (cherry picked from commit 09784db62fcd01124a101c4c69cab6e71e1ac781) (cherry picked from commit 79df36fecf8c8be5ae9d59397882ac844852043e)
* | | | | | | Merge "compute: Lock by instance.uuid lock during swap_volume" into stable/trainZuul2021-03-191-2/+26
|\ \ \ \ \ \ \
| * | | | | | | compute: Lock by instance.uuid lock during swap_volumeLee Yarwood2021-03-111-2/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The libvirt driver is currently the only virt driver implementing swap volume within Nova. While libvirt itself does support moving between multiple volumes attached to the same instance at the same time the current logic within the libvirt driver makes a call to virDomainGetXMLDesc that fails if there are active block jobs against any disk attached to the domain. This change simply uses an instance.uuid based lock in the compute layer to serialise requests to swap_volume to avoid this from being possible. Closes-Bug: #1896621 Change-Id: Ic5ce2580e7638a47f1ffddb4edbb503bf490504c (cherry picked from commit 6cf449bdd0d4beb95cf12311e7d2f8669e625fac) (cherry picked from commit eebf94b6540fcd16826067fac544b5a3238842a3) (cherry picked from commit f7ba1aab5f6f76ba88d6cc63cde2ec246ee61ec5)
* | | | | | | | Merge "Use subqueryload() instead of joinedload() for (system_)metadata" ↵Zuul2021-03-132-1/+24
|\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | into stable/train
| * | | | | | | | Use subqueryload() instead of joinedload() for (system_)metadatamelanie witt2021-01-272-1/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, when we "get" a single instance from the database and we load metadata and system_metadata, we do so using a joinedload() which does JOINs with the respective tables. Because of the one-to-many relationship between an instance and (system_)metadata records, doing the database query this way can result in a large number of additional rows being returned unnecessarily and cause a large data transfer. This is similar to the problem addressed by change I0610fb16ccce2ee95c318589c8abcc30613a3fe9 which added separate queries for (system_)metadata when we "get" multiple instances. We don't, however, reuse the same code for this change because _instances_fill_metadata converts the instance database object to a dict, and some callers of _instance_get_by_uuid need to be able to access an instance database object attached to the session (example: instance_update_and_get_original). By using subqueryload() [1], we can perform the additional queries for (system_)metadata to solve the problem with a similar approach. Closes-Bug: #1799298 [1] https://docs.sqlalchemy.org/en/13/orm/loading_relationships.html#subquery-eager-loading Change-Id: I5c071f70f669966e9807b38e99077c1cae5b4606 (cherry picked from commit e728fe668a6de886455f2dbaf655c8a151462c8c) (cherry picked from commit 63d2e62c3a223f883ca810f4c66a2a236cf3d483) (cherry picked from commit e7a45e0335e4cf44fec7f7b8d2505f5b95445cf9)
* | | | | | | | | Merge "replace the "hide_hypervisor_id" to "hw:hide_hypervisor_id"" into ↵Zuul2021-03-124-9/+37
|\ \ \ \ \ \ \ \ \ | |_|/ / / / / / / |/| | | | | | | | | | | | | | | | | stable/train
| * | | | | | | | replace the "hide_hypervisor_id" to "hw:hide_hypervisor_id"ramboman2021-03-014-9/+37
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we use the flavor extra_specs "hide_hypervisor_id" in AggregateInstanceExtraSpecsFilter, then will retrun False. So we need correct the extra_specs. NOTE: The first two files do not exist in stable/train due to extra specs validators (patch Ib64a1348cce1dca995746214616c4f33d9d664bd) was introduced in Ussuri. The last one file has conflict due to the same bp/flavor-extra-sepc-validators feature (patch: I8da84b48e4d630eeb91d92346aa2323e25e28e3b) added in Ussuri. Conflicts: nova/api/validation/extra_specs/hw.py nova/api/validation/extra_specs/null.py nova/tests/unit/api/openstack/compute/test_flavors_extra_specs.py Change-Id: I9d8d8c3a30cf6da7e8fb48374347e069ab075df2 Closes-Bug: 1841932 (cherry picked from commit bf488a8630702160021b5848bf6e86fbb8015205) (cherry picked from commit 9d28d7ec808469ec129b66c69b9e63cd9537a63f)
* | | | | | | | | [stable-only] gate: Pin CEPH_RELEASE to nautilus in LM hook20.6.0Lee Yarwood2021-03-112-0/+6
|/ / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I1edd5a50079f325fa143a7e0d51b3aa3bb5ed45d moved the branchless devstack-plugin-ceph project to the Octopus release of Ceph that drops support for py2. As this was still the default on stable/train this breaks the nova-live-migration and nova-grenade jobs. This change works around this by pinning the CEPH_RELEASE to nautilus within the LM hook as was previously used prior to the above landing. Note that the devstack-plugin-ceph-tempest job from the plugin repo continues to pass as it is correctly pinned to the Luminous release that supports py2. If anything the above enforces the need to move away from these hook scripts and instead inherit our base ceph jobs from this repo in the future to avoid the Ceph release jumping around like this. Change-Id: I1d029ebe78b16ed2d4345201b515baf3701533d5
* | | | | | | | Merge "Disallow CONF.compute.max_disk_devices_to_attach = 0" into stable/trainZuul2021-02-233-1/+19
|\ \ \ \ \ \ \ \ | |_|_|_|_|_|/ / |/| | | | | | |
| * | | | | | | Disallow CONF.compute.max_disk_devices_to_attach = 0melanie witt2021-01-293-1/+19
| |/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The CONF.compute.max_disk_devices_to_attach option controls the maximum number of disk devices allowed to attach to an instance. If it is set to 0, it will literally allow no disk device for instances, preventing them from being able to boot. This adds a note to the config option help to call this out and changes nova-compute to raise InvalidConfiguration during init_host if [compute]max_disk_devices_to_attach has been set to 0. The nova-compute service will fail to start if the option is set to 0. Note: there doesn't appear to be any way to disallow particular values in a oslo.config IntOpt other than the min/max values. Here we need the min value to be -1 to represent unlimited. There is a 'choices' kwarg available but that is only for enumerating valid values and we need to allow any integer >= 1 as well. Change-Id: I6e30468bc28f661ddc17937ab1de04a706f05063 Closes-Bug: #1897950 (cherry picked from commit 25a632a4e1daa1941a6297ddb51088972f23ce6d) (cherry picked from commit 8e12b81839e1fb364b3598699fecf57c78f81df3) (cherry picked from commit 4ad7e5e2630542bce4cfa53adafd1da299adb0f6)
* | | | | | | Merge "libvirt: Only ask tpool.Proxy to autowrap vir* classes" into stable/trainZuul2021-02-203-5/+16
|\ \ \ \ \ \ \
| * | | | | | | libvirt: Only ask tpool.Proxy to autowrap vir* classesLee Yarwood2020-11-033-5/+16
| | |/ / / / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I668643c836d46a25df46d4c99a973af5e50a39db attempted to fix service wide pauses by providing a more complete list of classes to tpool.Proxy. While this excluded libvirtError it can include internal libvirt-python classes pointed to by private globals that have been introduced with the use of type checking within the module. Any attempt to wrap these internal classes will result in the failure seen in bug #1901383. As a result this change simply ignores any class found during inspection that doesn't start with the `vir` string, used by libvirt to denote public methods and classes. Closes-Bug: #1901383 Co-Authored-By: Daniel Berrange <berrange@redhat.com> Change-Id: I568b0c4fd6069b9118ff116532f14abb46cc42ab (cherry picked from commit 0d2ca53bb86b8e4a3c44855cb5ef57f223462543) (cherry picked from commit 048a3337a8c98ec8fc138083376430ffb9027f67) (cherry picked from commit 36cb57d97be07a5560bef4c090f8cab47992c042)
* | | | | | | tools: Allow check-cherry-picks.sh to be disabled by an env varLee Yarwood2021-02-172-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The checks performed by this script aren't always useful to downstream consumers of the repo so allow them to disable the script without having to make changes to tox.ini. NOTE(lyarwood): This backport has Ie8a672fd21184c810bfe9c0e3a49582189bf2111 squashed into it to ensure the introduced env var is passed into the pep8 tox env. tox: Add passenv DISABLE_CHERRY_PICK_CHECK to pep8 I4f551dc4b57905cab8aa005c5680223ad1b57639 introduced the environment variable to disable the check-cherry-pick.sh script but forgot to allow it to be passed into the pep8 tox env. Change-Id: I4f551dc4b57905cab8aa005c5680223ad1b57639 (cherry picked from commit 610396f8ad5fe8c3abb7731fcd42c81e5246a938) (cherry picked from commit 3d86df068a7a78f8faa9ac6437ff4fc19f5cebcc) (cherry picked from commit cb96119e59df64d6884431c8f2e8f6ced8318bd7)
* | | | | | | Merge "Use cell targeted context to query BDMs for metadata" into stable/trainZuul2021-02-052-11/+17
|\ \ \ \ \ \ \
| * | | | | | | Use cell targeted context to query BDMs for metadataBalazs Gibizer2021-01-192-11/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The metadata service supports a multicell deployment in a configuration where the nova-api service implements the metadata API. In this case the metadata query needs to be cell targeted. This was partly implemented already. The instance itself is queried from the cell DB properly. However the BDM data used the non targeted context resulting in an empty BDM returned by the metadata service. Functional reproduction test is not added as I did not find a way to have a cell setup in the functional test that reproduce the problem. I reproduced the bug and tested the fix in a devstack. Change-Id: I48f57082edaef3ec4722bd31ce29a90b94d32523 Closes-Bug: #1881944 (cherry picked from commit 1390eecf8dec4c3b022a9b1e259a908197566738) (cherry picked from commit 9a5b6249d65c96e387a9eb0ecdefcc2057c21869) (cherry picked from commit c727cfc36c9d352102e04b9c94bebc7e25a1ab97)
* | | | | | | | Merge "Fix a hacking test" into stable/trainZuul2021-02-041-1/+1
|\ \ \ \ \ \ \ \ | |_|_|_|_|_|_|/ |/| | | | | | |
| * | | | | | | Fix a hacking testTakashi Natsume2021-01-091-1/+1
| |/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In test_useless_assertion, the useless_assertion method should be checked instead of nonexistent_assertion_methods_and_attributes. Change-Id: Ifd19f636f58ae353d912bde57cba2cd0a29a9baa Signed-off-by: Takashi Natsume <takanattie@gmail.com> (cherry picked from commit 117508129461436e13c148bb068b0775d67e85d3) (cherry picked from commit f4d62e1a0b77f9611a3be8427adafd96caf24bb1) (cherry picked from commit 7562e64dee54a36162c3c181214aa269068f1712)
* | | | | | | Handle disabled CPU features to fix live migration failuresAndrew Bonney2021-02-022-2/+47
| |_|/ / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When performing a live migration between hypervisors running libvirt, where one or more CPU features are disabled, nova does not take account of these. This results in migration failures as none of the available hypervisor targets appear compatible. This patch ensures that the libvirt 'disable' poicy is taken account of, at least in a basic sense, by explicitly ignoring items flagged in this way when enumerating CPU features. Closes-Bug: #1898715 Change-Id: Iaf14ca97cfac99dd280d1114123f2d4bb6292b63 (cherry picked from commit eeeca4ceff576beaa8558360c8a6a165d716f996) (cherry picked from commit 45a4110d20a574d0c43431a4b87497920c9cbe06) (cherry picked from commit b6c473159ec45e0aa715edd45cde28f77484a5f7)
* | | | | | Update pci stat pools based on PCI device changes20.5.0Hemanth Nakkina2021-01-116-33/+222
|/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | At start up of nova-compute service, the PCI stat pools are populated based on information in pci_devices table in Nova database. The pools are updated only when new device is added or removed but not on any device changes like device type. If an existing device is configured as SRIOV and nova-compute is restarted, the pci_devices table gets updated but the device is still listed under the old pool in pci_tracker.stats.pool (in-memory object). This patch looks for device type updates in existing devices and updates the pools accordingly. Conflicts: nova/tests/functional/libvirt/test_pci_sriov_servers.py nova/tests/unit/virt/libvirt/fakelibvirt.py nova/tests/functional/libvirt/base.py To avoid the conflicts and make the new functional test execute, following changes are performed - Modified the test case to use flavor extra spec pci_passthrough :alias to create a server with sriov port instead of creating a sriov port and passing port information during server creation. - Removed changes in nova/tests/functional/libvirt/base.py as they are required only if neutron sriov port is created in the test case. Change-Id: Id4ebb06e634a612c8be4be6c678d8265e0b99730 Closes-Bug: #1892361 (cherry picked from commit b8695de6da56db42b83b9d9d4c330148766644be) (cherry picked from commit d8b8a8193b6b8228f6e7d6bde68b5ea6bb53dd8b) (cherry picked from commit f58399cf496566e39d11f82a61e0b47900f2eafa)
* | | | | [stable-only] Cap bandit to 1.6.2 and raise hacking, flake8 and stestrLee Yarwood2020-12-232-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The 1.6.3 [1] release has dropped support for py2 [2] so cap to 1.6.2 when using py2. This change also raises hacking to 1.1.0 in lower-constraints.txt after it was bumped by I35c654bd39f343417e0a1124263ff31dcd0b05c9. This also means that flake8 is bumped to 2.6.0. stestr is also bumped to 2.0.0 as required by oslotest 3.8.0. All of these changes are squashed into a single change to pass the gate. [1] https://github.com/PyCQA/bandit/releases/tag/1.6.3 [2] https://github.com/PyCQA/bandit/pull/615 Depends-On: https://review.opendev.org/c/openstack/devstack/+/768256 Depends-On: https://review.opendev.org/c/openstack/swift/+/766214 Closes-Bug: #1907438 Closes-Bug: #1907756 Change-Id: Ie5221bf37c6ed9268a4aa0737ffcdd811e39360a
* | | | | Merge "Change default num_retries for glance to 3" into stable/trainZuul2020-12-014-8/+19
|\ \ \ \ \
| * | | | | Change default num_retries for glance to 3Keigo Noha2020-11-134-8/+19
| |/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, the default value of num_retries for glance is 0. It means that the request to glance is sent only one time. On the other hand, neutron and cinder clients set the default value to 3. To align the default value for retry to other components, we should change the default value to 3. Closes-Bug: #1888168 Change-Id: Ibbd4bd26408328b9e1a1128b3794721405631193 (cherry picked from commit 662af9fab6eacb46bcaee38d076d33c2c0f82b9b) (cherry picked from commit 1f9dd694b937cc55a81a64fdce442829f009afb3)
* | | | | Merge "Validate id as integer for os-aggregates" into stable/trainZuul2020-11-282-21/+85
|\ \ \ \ \
| * | | | | Validate id as integer for os-aggregatesJohannes Kulik2020-11-272-21/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | According to the api-ref, the id passed to calls in os-aggregates is supposed to be an integer. No function validated this, so any value passed to these functions would directly reach the DB. While this is fine for SQLite, making a query with a string for an integer column on other databases like PostgreSQL results in a DBError exception and thus a HTTP 500 instead of 400 or 404. This commit adds validation for the id parameter the same way it's already done for other endpoints. Conflicts: nova/api/openstack/compute/aggregates.py Changes: nova/tests/unit/api/openstack/compute/test_aggregates.py NOTE(stephenfin): Conflicts are due to absence of change I4ab96095106b38737ed355fcad07e758f8b5a9b0 ("Add image caching API for aggregates") which we don't want to backport. A test related to this feature must also be removed. Change-Id: I83817f7301680801beaee375825f02eda526eda1 Closes-Bug: 1865040 (cherry picked from commit 2e70a1717f25652912886cbefa3f40e6df908c00)
* | | | | | docs: Clarify configuration steps for PF devicesStephen Finucane2020-11-262-5/+27
|/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Devices that report SR-IOV capabilities cannot be used without special configuration - namely, the addition of "'device_type': 'type-PF'" or "'device_type': 'type-VF'" to the '[pci] alias' configuration option. Spell this out in the docs. Change-Id: I4abbe30505a5e4ccba16027addd6d5f45066e31b Signed-off-by: Stephen Finucane <sfinucan@redhat.com> Closes-Bug: #1852727 (cherry picked from commit 810aafc5ec9a7d25b33cf6c137c47b117c91269a)
* | | | | docs: Change order of PCI configuration stepsStephen Finucane2020-11-261-72/+70
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It doesn't really make sense to describe the "higher level" configuration steps necessary for PCI passthrough before describing things like BIOS configuration. Simply switch the ordering. Change-Id: I4ea1d9a332d6585ce2c0d5a531fa3c4ad9c89482 Signed-off-by: Stephen Finucane <sfinucan@redhat.com> Related-Bug: #1852727 (cherry picked from commit 557728abaf0c822f2b1a5cdd4fb2e11e19d8ead7)