summaryrefslogtreecommitdiff
path: root/glance_store/tests
Commit message (Collapse)AuthorAgeFilesLines
* Add force to os-brick disconnect4.4.0Brian Rosmaita2023-05-102-2/+2
| | | | | | | | | In order to be sure that devices are being removed from the host, we should be using the 'force' parameter with os-brick's disconnect_volume() method. Closes-bug: #2004555 Change-Id: I63d09ad9ef465bc154c85a9ea125449c039d1b90
* Merge "move attachment_update to try block"Zuul2023-05-031-4/+15
|\
| * move attachment_update to try blockanguoming2023-02-161-4/+15
| | | | | | | | | | | | | | | | | | move attachment_update to try block, so when there is some error calling attachment_update the finally block will be called, then the attachment created could be deleted. Closes-bug: #1983238 Change-Id: I2dc3888a56d802424c6d62a656b2e5fef9dabb3e
* | Merge "RBD: Wrap RBD calls in native threads"Zuul2023-04-261-0/+50
|\ \
| * | RBD: Wrap RBD calls in native threadsRajat Dhasmana2023-04-061-0/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | librbd methods call lower level C code which runs in native thread and isn't aware about the eventlet threads hence hangs the eventlet loop until the native thread is executed. This could cause problems when we are creating multiple images with large size where one call to librados can cause the process to hang and other operations can starve for execution and error out. This patch wraps each RBD call in it's own native thread that won't affect other RBD call from executing. Change-Id: I8efb0460df9fcba050b5ce949eb10caea325c851
* | | Run cinder driver unit testsBrian Rosmaita2023-04-181-0/+0
|/ / | | | | | | | | | | We have all these nice tests, might as well execute them. Change-Id: Iaf5131927bc2a0a953ec87c8b4d2e40f5d61d878
* | Fix misuse of assertTrueTakashi Natsume2023-02-122-22/+22
|/ | | | | | | Replace assertTrue with assertIsInstance or assertEqual. Change-Id: Ied3d6ba3941745a6bebda50780d16be2908a619c Signed-off-by: Takashi Natsume <takanattie@gmail.com>
* Merge "Cinder: Add support to extend attached volumes"4.3.0Zuul2023-02-086-9/+41
|\
| * Cinder: Add support to extend attached volumesRajat Dhasmana2023-01-186-9/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While creating an image, if we want to extend the volume (to accomodate the image), we need to first detach the volume, perform the extend and attach it again. This is inefficient for backends that support extending attached volumes since we are performing 3 cinder operations (the attachment call includes 3 API calls which sum to a total of 5 API calls for 3 operations) instead of directly extending it which requires only 1 API call to cinder. The support for extending attached volumes was added in cinder microversion 3.42. This patch adds a new boolean config option ``cinder_do_extend_attached`` which allows operators to specify if the cinder backend they are using supports extending attached (in-use) volumes. By default this will be ``false``. Based on the parameter, we will perform the extend operation, online (if cinder_do_extend_attached is true) and offline otherwise. Spec: https://review.opendev.org/c/openstack/glance-specs/+/868901 Depends-On: https://review.opendev.org/c/openstack/glance/+/869021 Depends-On: https://review.opendev.org/c/openstack/cinder/+/869051 Change-Id: I5e70824e9abc5277ea25ba95704b358fe3686037
* | Merge "Rbd: Deprecate unused rados_connect_timeout"Zuul2023-02-082-16/+0
|\ \ | |/ |/|
| * Rbd: Deprecate unused rados_connect_timeoutCyril Roelandt2022-08-042-16/+0
| | | | | | | | | | | | | | | | | | | | | | This option determines the value we pass as the "timeout" argument to rados.Rados.connect. Unfortunately, this argument is silently ignored by librados[1]. [1] https://docs.ceph.com/en/latest/rados/api/python/#rados.Rados.connect Closes-Bug: #1983499 Change-Id: I819df987f690c7a8c0a1153ad0e30bd5dbeb7b57
* | Refactor/restructure glance cinder storewhoami-rajat2023-01-177-49/+280
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is an effort to decouple some of the cinder backend specific code (like nfs, scaleio) from the generic logic. The purpose is to make the code modular and any change for a particular cinder backend should not affect the code path of other backends thereby reducing regression. This is also required for another use case of supporting extend of attached volumes added in [1]. Following are the major changes done in this patch: 1) Move cinder store to a new directory 'cinder' and rename 'cinder.py' to 'store.py' (similar to swift) 2) Create new files for nfs and scaleio backends for moving code specific to these backends into their own separate file. This also fixes one bug when using sparse files in nfs and we wait for file size to be equal to volume size (initially done for scaleio/powerflex backend) but this will never happen for nfs sparse files. See bug: 2000584 3) Move cinder tests to 'tests/unit/cinder' directory and add tests for base, nfs and scaleio files. 4) Modify/fix existing tests Closes-Bug: #2000584 [1] https://review.opendev.org/c/openstack/glance_store/+/868742 Depends-On: https://review.opendev.org/c/openstack/glance/+/869021 Change-Id: I26c272b6c503e98fbbafca411d3eec47283bd6fc
* | Add region_name option to s3 storeArnaud Morin2022-12-023-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | Add an option to let the operator choose a proper region_name. Some operator are not using S3 from amazon but from other provider. In such case, the get_s3_location cannot give good region_name. Adding an option from config seems a better option that relying on hardcoded stuff in the code. Signed-off-by: Arnaud Morin <arnaud.morin@ovhcloud.com> Change-Id: I71352e75f6415494709d8fb62c8b13f3ea7c6016
* | [test-only] OverflowError running on 32-bit systemsBrian Rosmaita2022-10-121-6/+18
| | | | | | | | | | | | | | | | | | | | | | Change I59b1579dc9877668b82d4195431c1 added tests that mock an input buffer, but the mocked return_value for __len__ exceeds the allowable maximum value for an index in the CPython implementation on 32-bit systems. Fix this by using half the value and allowing the buffer to be called twice during the tests. Change-Id: Iaba20c01fb4d994cd738a570197f8f6f04f22606 Closes-bug: #1991406
* | Merge "Tests: Mock sleep in cinder test_attachment_create_retries"4.1.0Zuul2022-08-231-0/+1
|\ \
| * | Tests: Mock sleep in cinder test_attachment_create_retriesEric Harney2022-08-221-0/+1
| | | | | | | | | | | | | | | | | | | | | This brings the time to run this unit test from 6+ seconds to 0.003s. Change-Id: I0c40e33bb7f35555b50ca04f1b048d9b716b77b5
* | | Merge "Swift: Honor *_domain_name parameters"Zuul2022-08-224-14/+169
|\ \ \ | |/ / |/| |
| * | Swift: Honor *_domain_name parametersTakashi Kajinami2022-08-184-14/+169
| |/ | | | | | | | | | | | | | | | | The *_domain_id parmaeters should not have any default. Otherwise keystoneauth ignores the *_domain_name parameters and it requires only *_domain_id parameters are used. Closes-Bug: #1620999 Change-Id: I1f8c9184761313f9fc5fda2f257e52233e0196d1
* | Do not loose url queries on redirectsPavlo Shchelokovskyy2022-08-041-0/+9
|/ | | | | | | | | when fetching images with http driver, the redirect URL can have mandatory query in it, which must be kept intact to successfully fetch the image. Change-Id: I2a9d4d026b935ea6c5e5a3a46c86f70ce1e39ae7 Closes-Bug: #1633860
* Add debug logs to cinder storewhoami-rajat2022-07-201-1/+2
| | | | | | | | | | When debugging issues related to glance cinder store, there are several calls to cinder and it becomes hard to determine which step we are currently executing without going through cinder logs. This patch adds some useful debug logs for the new attachment's code to understand which stage of attachment we are on. Change-Id: I491b7292a511c47c1d6148dab69ae04269e50c85
* Remove Python 2 supportliyou012022-05-2515-232/+223
| | | | | | | | | Python 2 has been deprecated for almost two years, and has not been guaranteed to work with glance_store for a while. This patch removes all traces of six, unicode strings and Python 2 tweaks. Co-Authored-By: Cyril Roelandt <cyril@redhat.com> Change-Id: Ifa78924d7ecf4f2d9a54c677888ab2926530c487
* Merge "Cinder: Correct exception logging during attach"Zuul2022-05-051-0/+9
|\
| * Cinder: Correct exception logging during attachwhoami-rajat2022-04-281-0/+9
| | | | | | | | | | | | | | | | | | | | There is a syntax error in the exception logging when attaching a volume fails in attachment state manager. This patch corrects it and adds a test to guard against similar changes in future. Closes-Bug: #1970698 Change-Id: I43c407046a49bb37631113e2ea65d05450f9365d
* | Add coverage for add methodwhoami-rajat2022-03-223-0/+173
| | | | | | | | | | | | | | This patch adds test coverage of code paths like extend volume, exception blocks, when image_size is zero, volume delete call etc Change-Id: I59b1579dc9877668b82d4195431c14cc41cfe892
* | Add exception coverage for get, get_size, deletewhoami-rajat2022-03-223-28/+71
| | | | | | | | | | | | | | | | | | | | | | | | This patch adds coverage for the various exceptions handled and raised in get, get_size and delete methods. It also corrects the behavior of _test_cinder_get_size method where client.volumes returned dictionary instead of a MagicMock and it worked due to the existing method "get" in dictionary which has same name as "get" method in cinderclient. The dictionary object is replaced with the appropriate MagicMock object in this patch. Change-Id: If63a6c810b5aab992e54857bc81f5052c2c593c4
* | Add coverage for helper methodswhoami-rajat2022-03-221-0/+33
|/ | | | | | | Add coverage for helper methods like ``get_hash_str``, ``_get_mount_path`` and ``_get_host_ip``. Change-Id: Idb232f2d7b0f74e9c65e42955c767fb7bb6ae004
* Add coverage for get_cinderclient and _check_contextwhoami-rajat2022-03-223-0/+64
| | | | | | | | | | This patch adds coverage for some parts in get_cinderclient method like ``cinder_endpoint_template`` option and keystone exception case when catalog info for cinder could not be found. Also it adds tests for complete coverage of _check_context method. Change-Id: I18b1e5e8fd818824a2dda2ad14d9456190fe9ff4
* Add coverage for StoreLocationwhoami-rajat2022-03-173-0/+49
| | | | | | | | | | | | This patch adds code coverage (UTs) for methods in StoreLocation class: 1) process_specs 2) get_uri 3) parse_uri It also adds a missing method (_set_url_prefix) coverage in Multistore cinder tests (test_multistore_cinder). Change-Id: I8ced5af11669fb131c665ce53be57143f2c7b518
* Add coverage for get_cinder_sessionwhoami-rajat2022-03-143-20/+72
| | | | | | | | Add test coverage for two config options used to create a keystone session object for cinderclient namely ``cinder_api_insecure`` and ``cinder_ca_certificates_file``. Change-Id: Idb6db9f87ef7290441dfe60ca2eb9a59c9ee7bd3
* Remove six usagewhoami-rajat2022-03-143-12/+9
| | | | | | | | | | six has been used to maintain compatibility between python2 and python3 code. Since current openstack development branches doesn't support python2, we don't need the six library and can use python3 code directly. This patch removes six usage from cinder store tests. Change-Id: I4deb193b44394ae9f99f57e31fb81fe04394e247
* Refactor cinder store tests[2/2]whoami-rajat2022-03-143-191/+126
| | | | | | | | | | | | | | | | | | | | | This patch aims at a refactoring effort that would remove duplicate tests (current and future) by moving them into a common base class which is called via both single and multi store test modules with their specific configurations. This has a lot of benefits: 1) Removes duplicate code 2) Makes addition of new tests easier and cleaner 3) Ensuring a new method/code path added is tested in both single and multi store configurations 4) Fixing issues detected while refactoring methods (Eg: tests for add method in test_multistore_cinder were not passing the hashing_algo parameter which is currently handled by the backward compat code (back_compat_add decorator) but those tests will break when we remove backward compatibility) Change-Id: I12569af5623f1cd7803c00a6c3b9eb211f15b6fd
* Refactor cinder store tests[1/2]whoami-rajat2022-03-113-639/+414
| | | | | | | | | | | | | | | | | | | This patch aims at a refactoring effort that would remove duplicate tests (current and future) by moving them into a common base class which is called via both single and multi store test modules with their specific configurations. This has a lot of benefits: 1) Removes duplicate code 2) Makes addition of new tests easier and cleaner 3) Ensuring a new method/code path added is tested in both single and multi store configurations 4) Fixing issues detected while refactoring methods (Eg: fake_chown accepted an optional backend parameter which does not match the signature of original method temporary_chown) Change-Id: Iaacb3e117cac2a661750bdb21bd0a7496078ea26
* Replace FakeObject with MagicMock[2/2]whoami-rajat2022-03-101-63/+68
| | | | | | | | | | | | | | | | | | FakeObject is used to set dynamic attributes to a fake class which works in most cases but fails when an attribute is accessed which is not defined/passed explicitly. This causes failure of tests whenever a new attribute is accessed and it's hard to maintain the set of attributes for every object used. MagicMock provides the same functionality with additional features and handles the case effectively when an attribute is accessed which is not explicitly defined in the tests. This change will help safeguard against cases seen earlier in[1]. This patch replaces FakeObject with MagicMock in test_multistore_cinder.py [1] https://review.opendev.org/c/openstack/glance/+/805974 Change-Id: I193bb03d174e6c010132702dedef1dea05f40e41
* Replace FakeObject with MagicMock[1/2]whoami-rajat2022-03-101-53/+49
| | | | | | | | | | | | | | | | | | FakeObject is used to set dynamic attributes to a fake class which works in most cases but fails when an attribute is accessed which is not defined/passed explicitly. This causes failure of tests whenever a new attribute is accessed and it's hard to maintain the set of attributes for every object used. MagicMock provides the same functionality with additional features and handles the case effectively when an attribute is accessed which is not explicitly defined in the tests. This change will help safeguard against cases seen earlier in[1]. This patch replaces FakeObject with MagicMock in test_cinder_store.py [1] https://review.opendev.org/c/openstack/glance/+/805974 Change-Id: Ifd0905b139cc90c2bd7444bc9e7638f3af6879a7
* Cinder store: Wait for device resize3.0.0whoami-rajat2022-02-072-4/+96
| | | | | | | | | | | | | | | | | | | | | | | When we have an image with size > 1 GB, we follow the following steps to accomodate the image: 1) Detach the volume 2) extend the volume 3) Attach the volume 4) Open the volume device as a file and resume writing the image Sometimes due to several reasons (mostly network related), the size of the device file could mismatch with the actual volume size (or the backend LUN size). This can happen if the extend was performed (i.e. the control path) but it takes the time to reflect that into the mapped device (i.e. the data path). This mismatch can cause the issue "IOError: [Errno 28] No space left on device". To avoid this scenario, we check if the device size is less than the volume size, we wait for the extended LUN to show up in mapped device and then continue the image writing operation. Closes-Bug: #1959913 Change-Id: I206580f6be615ebc5e15b546b9c23728d4116a5d
* Merge "Correct attachment_complete call"Zuul2022-01-072-4/+0
|\
| * Correct attachment_complete callwhoami-rajat2022-01-032-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cinder's attachment_complete API is called to mark the volume from "attaching" state to "in-use" state stating that the volume is ready to use. In the current code, attachment_complete is called before the brick connection is complete. This should be done after the connection to brick is successful and we have a volume path to write the image into. This patch corrects the behavior. The wrong sequence of execution won't have any issues functionally since we open the volume after all steps are completed but flow wise it is incorrect. This change won't have any end user impact. Change-Id: Ia9652a4ff6d7efbabb58511f0ce93a87b3a4dfa8
* | Pass valid IP address to os-brickwhoami-rajat2021-12-232-10/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | While fetching connector information the cinder driver of glance-store is passing 'hostname' as IP address whereas it actually needs IP address. This path fetches the IPV4 or IPV6 address from the available 'hostname' and passes it to os-brick. This worked for other cinder backends which don't rely specifically on IP address for initializing connection like fibre channel, nfs etc and even for some iscsi backends since, in some environments, the hostname is same as the ip address. This is not the case always and we should pass the correct IPv4/IPv6 address of the host to avoid this issue. Closes-Bug: #1955668 Change-Id: Ic79815972e654a8bfe2775f57c68cfa0bf115e2f
* | [RBD] Clone v2: Image is unusable if deletion failsAbhishek Kekane2021-12-212-0/+30
|/ | | | | | | | | | | | | | | | | | | | Recently cinder has utilized Ceph clone v2 support for its RBD backend, since then if you attempt to delete an image from glance that has a dependent volume, all future uses of that image will fail in error state. Despite the fact that image itself is still inside of Ceph/Glance. This issue is reproducible if you are using ceph client version greater than 'luminous' To resolve this issue glance RBD driver now checks whether snapshot of original image has any external references before deleting/removing it's snapshot and returns 409 Conflict response if it has any. NOTE: To check this dependency glance osd needs 'read' access to cinder and nova side RBD pool. Closes-Bug: #1954883 Depends-on: https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/819476 Change-Id: If30b7bd7acac148b6f89ce46abbe128c678c90e7
* Merge "Add volume multiattach handling"Zuul2021-08-174-12/+173
|\
| * Add volume multiattach handlingwhoami-rajat2021-08-124-12/+173
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We implemented cinder's new attachment API support with patch[1]. It was needed to add multiattach volume handling added with this patch. There is no special configuration change or user interference needed. If a volume is of a multiattach type, then it will be handled as a multiattach volume. [1] https://review.opendev.org/c/openstack/glance_store/+/782200 Implements: blueprint attachment-api-and-multiattach-support Closes-Bug: #1904546 Change-Id: Iffb825492a20fd877476acad05f817b399072f01
* | Merge "Raise correct exception from "Quota full""Zuul2021-08-161-0/+27
|\ \
| * | Raise correct exception from "Quota full"Erno Kuvaja2021-08-161-0/+27
| | | | | | | | | | | | | | | | | | | | | | | | Swift driver reraises the client exception causing glance-api to return 500. Lets raise g_s StoreFull correctly instead. Change-Id: I5be151afe242a75142e74d488c4787a35929e189 Closes-bug: #1926404
* | | Merge "swift: Take into account swift_store_endpoint"Zuul2021-08-112-0/+26
|\ \ \
| * | | swift: Take into account swift_store_endpointAlexandre Arents2021-03-012-0/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In SingleTenant authv3 context, connection manager does not evaluate swift_store_endpoint and always takes endpoint from catalog. The change ensures CONF.glance_store.swift_store_endpoint will take over catalog value also in that case. Closes-Bug: #1885651 Change-Id: Ib18ff19cd539e0117909f849672036b8c9e5f049
* | | | Merge "Doc: Use Block Storage API v3"Zuul2021-08-111-1/+1
|\ \ \ \ | |_|_|/ |/| | |
| * | | Doc: Use Block Storage API v3Takashi Kajinami2021-07-251-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ... because v2 API was deprecated during Pike and was already removed during Xena[1]. [1] e05b261af7dcd24096b229860df65dff1d385910 Change-Id: I0e6d72ce82ce89aaaa8c56862bb8266d0351cd86
* | | | Glance cinder nfs: Block creating qcow2 volumesRajat Dhasmana2021-07-282-33/+110
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There's an issue when cinder nfs is configured as glance backend and the image is > 1GB i.e. extend operation is performed on volume. Currently glance writes the qcow2 header in the raw volume and cinder nfs driver used to work on format autodetection which is changed to manual detection here[1] This fixes the extend volume problem for raw volumes, however, if cinder nfs is configured to create qcow2 volumes then we will run into the same problem. This patch blocks creating images when the nfs volume is qcow2. Refactoring work: 1) attachment_delete call is removed in nfs related code as it was already done in the finally block 2) handle_exceptions decorator is removed from volume delete method as it raises BackendException and incase when volume is not found (possibly deleted manually), the image deletion fails so we don't want to raise any error in this case [1] https://review.opendev.org/c/openstack/cinder/+/761152 Closes-Bug: #1901138 Change-Id: I8ce6f36f1cb4b0ed6bcc5f3869fab3bb64fe3390
* | | | Merge "Add cinder's new attachment support"Zuul2021-07-233-26/+221
|\ \ \ \ | |/ / / |/| | |
| * | | Add cinder's new attachment supportRajat Dhasmana2021-07-233-26/+221
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Cinder introduced new attachment API flow in microversion 3.27 (also attachment_complete added in mv 3.44 and support for passing mode added in mv 3.54) which provides a clean interface to interact with cinder for attachments and is also required for multiattach volume support (Related future work). Nova uses it since a long time and is proven to be stable, this patch implements the same for glance. The create volume and delete volume calls are also moved to cinder_utils file to use the generic exception handler and keep similar code together for consistency. Partially Implements: blueprint attachment-api-and-multiattach-support Change-Id: I2758ed1d5b8e0981faa3eff6f83e1ce5975a01d2
View Lorry Controller Status