summaryrefslogtreecommitdiff
path: root/tests
Commit message (Collapse)AuthorAgeFilesLines
* Fix SLO re-uploadTim Burke2019-07-291-2/+42
| | | | | | | | | | | | | | | | | Previously, if you uploaded a file as an SLO then re-uploaded it with the same segment size and mtime, the second upload would go delete the segments it just (re)uploaded. This was due to us tracking old_slo_manifest_paths and new_slo_manifest_paths in different formats; one would have a leading slash while the other would not. Now, normalize to the stripped-slash version so we stop deleting segments we just uploaded. Change-Id: Ibcbed3df4febe81cdf13855656e2daaca8d521b4 (cherry picked from commit 9021a58c240e156f54ffafdc4609868f348d3ebc) (cherry picked from commit 2cd8b86075997f2997118bd92286849ddb0c93d7) (cherry picked from commit d76aa4d810f948279ccffa6d3255f7200aed0323)
* Allow for object uploads > 5GB from stdin.Timur Alperovich2018-01-182-0/+235
| | | | | | | | | | | | | When uploading from standard input, swiftclient should turn the upload into an SLO in the case of large objects. This patch picks the threshold as 10MB (and uses that as the default segment size). The consumers can also supply the --segment-size option to alter that threshold and the SLO segment size. The patch does buffer one segment in memory (which is why 10MB default was chosen). (test is updated) Change-Id: Ib13e0b687bc85930c29fe9f151cf96bc53b2e594
* Merge "Allow --meta on upload"Zuul2017-12-082-28/+13
|\
| * Allow --meta on uploadTim Burke2017-07-062-28/+13
| | | | | | | | | | | | | | Previously, the --meta option was only allowed on post or copy subcommands. Change-Id: I87bf0338c34b5e89aa946505bee68dbeb37d784c Closes-Bug: #1616238
* | Merge "Add support for versionless endpoints"Jenkins2017-08-291-0/+20
|\ \
| * | Add support for versionless endpointsChristian Schwede2017-06-131-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Newer deployments are using versionless Keystone endpoints, and most OpenStack clients already support this. This patch enables this for Swift: if an auth_url without any path component is found, it assumes a versionless endpoint will be used. In this case the v3 suffix will be appended to the path if none auth_version is set, and v2.0 is appended if auth_version requires v2. Closes-Bug: 1554885 Related-Bug: 1691106 Change-Id: If7ecb67776cb77828f93ad8278cc5040015216b7
* | | Allow for uploads from standard input.Timur Alperovich2017-07-261-1/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If "-" is passed in for the source, python-swiftclient will upload the object by reading the contents of the standard input. The object name option must be set, as well, and this cannot be used in conjunction with other files. This approach stores the entire contents as one object. A follow on patch will change this behavior to upload from standard input as SLO, unless the segment size is larger than the content size. Change-Id: I1a8be6377de06f702e0f336a5a593408ed49be02
* | | Buffer reads from diskTim Burke2017-07-112-10/+10
| | | | | | | | | | | | | | | | | | | | | Otherwise, Python defaults to 8k reads which seems kinda terrible. Change-Id: I3160626e947083af487fd1c3cb0aa6a62646527b Closes-Bug: #1671621
* | | Option to ignore mtime metadata entry.Christopher Bartz2017-07-061-0/+46
| |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, the swiftclient upload command passes a custom metadata header for each object (called object-meta-mtime), whose value is the current UNIX timestamp. When downloading such an object with the swiftclient, the mtime header is parsed and passed as the atime and mtime for the newly created file. There are use-cases where this is not desired, for example when using tmp or scratch directories in which files older than a specific date are deleted. This commit provides a boolean option for ignoring the mtime header. Change-Id: If60b389aa910c6f1969b999b5d3b6d0940375686
* | Merge "Skip checksum validation on partial downloads"Jenkins2017-06-221-0/+10
|\ \
| * | Skip checksum validation on partial downloadsTim Burke2017-04-211-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | If we get back some partial content, we can't validate the MD5. That's OK. Change-Id: Ic1d65272190af0d3d982f3cd06833cac5c791a1e Closes-Bug: 1642021
* | | Merge "Tolerate RFC-compliant ETags"Jenkins2017-06-221-15/+23
|\ \ \ | |/ /
| * | Tolerate RFC-compliant ETagsTim Burke2017-04-211-15/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Since time immemorial, Swift has returned unquoted ETags for plain-old Swift objects -- I hear tell that we once tried to change this, but quickly backed it out when some clients broke. However, some proxies (such as nginx) apparently may force the ETag to adhere to the RFC, which states [1]: An entity-tag consists of an opaque *quoted* string (emphasis mine). See the related bug for an instance of this happening. Since we can still get the original ETag easily, we should tolerate the more-compliant format. [1] https://tools.ietf.org/html/rfc2616.html#section-3.11 or, if you prefer the new ones, https://tools.ietf.org/html/rfc7232#section-2.3 Change-Id: I7cfacab3f250a9443af4b67111ef8088d37d9171 Closes-Bug: 1681529 Related-Bug: 1678976
* | | Merge "Stop sending X-Static-Large-Object headers"Jenkins2017-06-141-1/+0
|\ \ \
| * | | Stop sending X-Static-Large-Object headersTim Burke2017-04-101-1/+0
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If we were to include this in a normal PUT, it would 400, but only if slo is actually in the pipeline. If it's *not*, we'll create a normal Swift object and the header sticks. - This is really confusing for users; see the related bug. - If slo is later enabled in the cluster, Swift starts responding 500 with a KeyError because the client and on-disk formats don't match! Change-Id: I1d80c76af02f2ca847123349224ddc36d2a6996b Related-Change: I986c1656658f874172860469624118cc63bff9bc Related-Bug: #1680083
* | | Merge "Do not set Content-Type to '' with new requests."Jenkins2017-06-131-2/+17
|\ \ \ | |_|/ |/| |
| * | Do not set Content-Type to '' with new requests.Timur Alperovich2017-06-131-2/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, python-swiftclient worked around a requests issue where Content-Type could be set to application/x-www-form-urlencoded when using python3. This issue has been resolved and a fix released in requests 2.4 (fixed in subsequent releases as well). The patch makes the workaround conditional on the requests version, so that with sufficiently new requests libraries, the Content-Type is not set. For reference, requests 2.4 was released August 29th, 2014. The specific issue filed in the requests tracker is: https://github.com/requests/requests/issues/2071. Related-Change: I035f8b4b9c9ccdc79820b907770a48f86d0343b4 Closes-Bug: #1433767 Change-Id: Ieb2243d2ff5326920a27ce8c3c6f0f5c396701ed
* | | Merge "Fix MockHttpResponse to be more like the Real"Jenkins2017-06-121-44/+31
|\ \ \ | |/ / |/| |
| * | Fix MockHttpResponse to be more like the RealClay Gerrard2017-03-081-44/+31
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change pulls out that relatively new [1] little string to pull at in the MockHttpResponse that I think is sorta ugly. And replaces it with the correct behavior that's representative of the Real for which it's standing in (which is sadly our wrapper to make a requests response feel like a httplib.HTTPResponse). It's not clear (to me) the history which allowed this difference in the behavior of the Real and Fake to persist - it seems to have always been this way [2]. I also reworded a relatively new test [1] to cover more code, and make assertions on the desired behavior of the client instead of "just" the http_log method. FWIW, I don't think there was necessarily anything wrong with the scope of the new test [1] - and it certainly makes sense to see new tests copy nearby existing tests. But I subjectively think this smaller test is more demonstrative of the desired behavior. 1. Related-Change-Id: I6d7ccbf4ef9b46e890ecec58842c5cdd2804c7a9 2. Related-Change-Id: If07af46cb377f3f3d70f6c4284037241d360a8b7 Change-Id: Ib99a029c1bd1ea1efa8060fe8a11cb01deea41c6
* | | Merge "ISO 8601 timestamps for tempurl"Jenkins2017-05-182-16/+151
|\ \ \
| * | | ISO 8601 timestamps for tempurlChristopher Bartz2017-03-292-16/+151
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | Client-side implementation for ISO 8601 timestamp support of tempurl middleware. Please see https://review.openstack.org/#/c/422679/ Change-Id: I76da28b48948475ec1bae5258e0b39a316553fb7
* | | respect bulk delete page size and fix logic errorJohn Dickinson2017-04-202-20/+126
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, using SwiftService to delete "many" objects would use bulk delete if available, but it would not respect the bulk delete page size. If the number of objects to delete exceeded the bulk delete page size, SwiftService would ignore the error and nothing would be deleted. This patch changes _should_bulk_delete() to be _bulk_delete_page_size(); instead of returning a simple True/False, it returns the page size for the bulk deleter, or 1 if objects should be deleted one at a time. Delete SDK calls are then spread across multiple bulk DELETEs if the requested number of objects to delete exceeds the returned page size. Fixed the logic in _should_bulk_delete() so that if the object list is exactly 2x the thread count, it will not bulk delete. This is the natural conclusion following the logic that existed previously: if the delete request can be satisfied by every worker thread doing one or two tasks, don't bulk delete. But if it requires a worker thread to do three or more tasks, do a bulk delete instead. Previously, the logic would mean that if every worker thread did exactly two tasks, it would bulk delete. This patch changes a "<" to a "<=". Closes-Bug: 1679851 Change-Id: I3c18f89bac1170dc62187114ef06dbe721afcc2e
* | Close file handle after upload jobKazufumi Noto2017-03-161-47/+72
|/ | | | | | | | The opened file for upload is not closed. This fix prevents possible file handle leak. Closes-Bug: #1559079 Change-Id: Ibc58667789e8f54c74ae2bbd32717a45f7b30550
* Fix logging of the gzipped bodyVitaly Gridnev2017-03-082-1/+61
| | | | | Change-Id: I6d7ccbf4ef9b46e890ecec58842c5cdd2804c7a9 Closes-bug: 1670620
* Merge "Make functests py3-compatible"Jenkins2017-01-241-15/+20
|\
| * Make functests py3-compatibleTim Burke2016-08-241-15/+20
| | | | | | | | Change-Id: I2b3bf17e874cf049eccab4c85ceac7da10d258ef
* | Merge "Accept more types of input for headers/meta"Jenkins2017-01-241-3/+16
|\ \
| * | Accept more types of input for headers/metaTim Burke2016-11-181-3/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, we only accepted iterables of strings like 'Header: Value'. Now, we'll also accept lists of tuples like ('Header', 'Value') as well as dictionaries like {'Header': 'Value'}. This should be more intuitive for application developers, who are already used to being able to pass dicts or lists of tuples to libraries like requests. Change-Id: I93ed2f1e8305f0168b7a4bd90c205b04730da836
* | | prefix-based tempurls supportChristopher Bartz2017-01-192-2/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | Implements client-side functionality for prefix-based tempurls. Please see: https://review.openstack.org/#/c/274048/ Change-Id: I8d7701daee888ed1120271a96c0660b01543ca2d
* | | Merge "modify 'swift <sub_command> —help' display"Jenkins2016-12-131-2/+5
|\ \ \ | |/ / |/| |
| * | modify 'swift <sub_command> —help' displayShashirekha Gundur2016-12-131-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | In python swiftclient: swift <sub_command> —help will now display st_<sub_command>_options + st_<sub_command>_help texts e.g. http://paste.openstack.org/show/589752/ Change-Id: I34e4b2ac29ef395f8ca474ce7a82f59a1fd8c7f4 Closes-Bug: #1621415
* | | Replace 'assertEqual(None, ...)' with 'assertIsNone(...)'howardlee2016-11-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [H203] Use assertIs(Not)None to check for None (off by default) Unit test assertions tend to give better messages for more specific assertions. As a result, assertIsNone(...) is preferred over assertEqual(None, ...) and assertIs(None, ...), and assertIsNotNone(...) is preferred over assertNotEqual(None, ...) and assertIsNot(None, ...). Off by default. More details, see: http://docs.openstack.org/developer/hacking/#unit-tests-and-assertraises Trivial fix. Change-Id: Icd268b96dea5e5bb9bd344f597dfcd9cc82253f0
* | | Merge "Add additional headers for HEAD/GET/DELETE requests."Jenkins2016-11-083-50/+307
|\ \ \
| * | | Add additional headers for HEAD/GET/DELETE requests.Charles Hsu2016-11-073-50/+307
| | | | | | | | | | | | | | | | | | | | Change-Id: I69276ba711057c122f97deac412e492e313c34dd Closes-Bug: 1615830
* | | | Merge "Add v1password keystoneauth plugin"Jenkins2016-11-081-0/+246
|\ \ \ \
| * | | | Add v1password keystoneauth pluginTim Burke2016-10-241-0/+246
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This lets us use Keystone sessions against endpoints like swauth and tempauth with code like: import keystoneauth1.loading import keystoneauth1.session import swiftclient loader = keystoneauth1.loading.get_plugin_loader('v1password') auth_plugin = loader.load_from_options( auth_url='http://saio:8080/auth/v1.0', username='test:tester', password='testing') keystone_session = keystoneauth1.session.Session(auth_plugin) conn = swiftclient.Connection(session=keystone_session) The plugin includes an optional project_name option, which may be used to override the swift account from the storage url that was returned. Additionally, it includes enough infrastructure to support some commands in python-openstackclient>=3.0: export OS_AUTH_TYPE=v1password export OS_AUTH_URL=http://saio:8080/auth/v1.0 export OS_PROJECT_NAME=AUTH_test2 export OS_USERNAME=test:tester export OS_PASSWORD=testing openstack token issue openstack catalog list openstack catalog show object-store openstack object store account show openstack container list openstack container create <container> openstack container save <container> openstack container show <container> openstack container delete <container> openstack object list <container> openstack object create <container> <file> openstack object save <container> <object> opsentack object show <container> <object> openstack object delete <container> <object> Change-Id: Ia963dc44415f72a6518227e86d9528a987e07491
* | | | | Merge "Adding keystoneauth sessions support"Jenkins2016-10-261-0/+42
|\ \ \ \ \ | |/ / / / | | | / / | |_|/ / |/| | |
| * | | Adding keystoneauth sessions supportPaulo Ewerton2016-05-191-0/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch allows authentication in swiftclient with a keystonauth session. Co-Authored-By: Tim Burke <tim@swiftstack.com> Change-Id: Ia3fd947ff619c11ff0ce474897533dcf7b49d9b3 Closes-Bug: 1518938
* | | | Merge "Use ConfigParser instead of SafeConfigParser"Jenkins2016-10-171-1/+1
|\ \ \ \
| * | | | Use ConfigParser instead of SafeConfigParserPallavi2016-10-171-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The SafeConfigParser class has been renamed to ConfigParser in Python 3.2. This alias will be removed in future versions.So we can use ConfigParser directly instead. Change-Id: Ie0292271229ee77d1bc57dd897ebbbb3d3c059aa Closes-Bug: #1618666
* | | | | Replace assertTrue(a in b) with assertIn(a, b)Anh Tran2016-10-031-4/+4
| |_|/ / |/| | | | | | | | | | | Change-Id: I9726c9c051eea9264200ab27b3e556c68d5c927f
* | | | Merge "Make tempurl subcommand insist on whole number seconds"Jenkins2016-09-151-3/+18
|\ \ \ \
| * | | | Make tempurl subcommand insist on whole number secondsAlistair Coles2016-09-141-3/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously the tempurl subcommand would dump a traceback due to a TypeError if the seconds option was not an int value. With this patch it will now return the same error message as if the seconds option were negative or not a number. Also changes the error message to state that the seconds option should be a "whole number" rather than a "positive integer", since 0 is a valid value. Change-Id: Ie940d470f2be8006aa8eb7fe242f092457aeae21 Closes-Bug: #1621817
* | | | | Merge "Fix intermittent test failure"Jenkins2016-09-141-3/+7
|\ \ \ \ \
| * | | | | Fix intermittent test failureTim Burke2016-09-011-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The mock needs to be initialized before calling main, or we get a race to create it between all the uu_threads Change-Id: If7649da13ed9276d7f0e005e999770e09c022a3f
* | | | | | Merge "Make tempurl command check for valid object path"Jenkins2016-09-142-2/+40
|\ \ \ \ \ \ | | |/ / / / | |/| / / / | |_|/ / / |/| | | |
| * | | | Make tempurl command check for valid object pathAlistair Coles2016-09-142-2/+40
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the supplied path is not of the form /v1/a/c/o then swift tempurl subcommand will now return an error message. Also removes redundant check for seconds parameter being an int from shell.py because the same check is made when calling utils.generate_temp_url. Drive-by fix for missing param definition for generate_temp_url. Change-Id: I41f4389948b01fadaa5fc4939ea12e0ed2167345 Related-Change: I0fb2ce125fe12d660e4deb778265016bdd5ff31b
* | | | | Fix order of arguments in assertIsJi-Wei2016-09-091-6/+6
|/ / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some tests used incorrect order assertIs(observed, expected). The correct order expected by testtools is assertIs(expected, observed, message='') Assert that expected is observed. Parameters: expected The expected value. observed The observed value. message An optional message describing the error. https://docs.python.org/2/library/unittest.html Change-Id: I5f406336b7804c13ca042dd0c6b30645a711c5f2
* | | | Make options arguments truely optionalChristian Schwede2016-09-061-0/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the options help, info and debug must exist in the options dictionary; however it might be that this is not the case if a method is imported externally. This patch fixes this. Also support using a storage URL or path as an argument for tempurl, and adding a test to verify correct output in both cases. Related-Bug: 1607519 Related-Bug: 1607523 Closes-Bug: 1607521 Change-Id: I0fb2ce125fe12d660e4deb778265016bdd5ff31b
* | | | Merge "Accept gzip-encoded API responses"Jenkins2016-09-012-3/+55
|\ \ \ \
View Lorry Controller Status