summaryrefslogtreecommitdiff
path: root/swiftclient/client.py
Commit message (Collapse)AuthorAgeFilesLines
* Add support for versionless endpointsChristian Schwede2017-09-191-2/+15
| | | | | | | | | | | | | | | Newer deployments are using versionless Keystone endpoints, and most OpenStack clients already support this. This patch enables this for Swift: if an auth_url without any path component is found, it assumes a versionless endpoint will be used. In this case the v3 suffix will be appended to the path if none auth_version is set, and v2.0 is appended if auth_version requires v2. Closes-Bug: 1554885 Related-Bug: 1691106 Change-Id: If7ecb67776cb77828f93ad8278cc5040015216b7 (cherry picked from commit 2ff3102cf779a507af8fa39296154a17a60f540a)
* Do not set Content-Type to '' with new requests.Timur Alperovich2017-06-131-2/+4
| | | | | | | | | | | | | | | | | | Previously, python-swiftclient worked around a requests issue where Content-Type could be set to application/x-www-form-urlencoded when using python3. This issue has been resolved and a fix released in requests 2.4 (fixed in subsequent releases as well). The patch makes the workaround conditional on the requests version, so that with sufficiently new requests libraries, the Content-Type is not set. For reference, requests 2.4 was released August 29th, 2014. The specific issue filed in the requests tracker is: https://github.com/requests/requests/issues/2071. Related-Change: I035f8b4b9c9ccdc79820b907770a48f86d0343b4 Closes-Bug: #1433767 Change-Id: Ieb2243d2ff5326920a27ce8c3c6f0f5c396701ed
* Fix logging of the gzipped bodyVitaly Gridnev2017-03-081-2/+5
| | | | | Change-Id: I6d7ccbf4ef9b46e890ecec58842c5cdd2804c7a9 Closes-bug: 1670620
* Merge "Add additional headers for HEAD/GET/DELETE requests."Jenkins2016-11-081-23/+42
|\
| * Add additional headers for HEAD/GET/DELETE requests.Charles Hsu2016-11-071-23/+42
| | | | | | | | | | Change-Id: I69276ba711057c122f97deac412e492e313c34dd Closes-Bug: 1615830
* | Low-level API: Don't log just before raising an exceptionTim Burke2016-10-281-7/+2
| | | | | | | | | | | | | | | | | | | | | | | | The only logging we should do is when we've encountered a problem *and we've dealt with it ourselves*. When we're raising an exception, it should be up to the caller to decide whether to log anything about it. Anything else is just rude. Change-Id: I1c96b76d90a78b7a10ffe63e4a7440c8f579147c Closes-Bug: 1213179 Related-Bug: 1202229
* | Merge "Adding keystoneauth sessions support"Jenkins2016-10-261-10/+25
|\ \
| * | Adding keystoneauth sessions supportPaulo Ewerton2016-05-191-10/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch allows authentication in swiftclient with a keystonauth session. Co-Authored-By: Tim Burke <tim@swiftstack.com> Change-Id: Ia3fd947ff619c11ff0ce474897533dcf7b49d9b3 Closes-Bug: 1518938
* | | Remove redundant space in docstringAnh Tran2016-09-281-4/+4
| | | | | | | | | | | | | | | | | | TrivialFix Change-Id: I7012fb7bbbedfba30b8e0450a449787e9076f4ef
* | | TrivialFix: Removed redundant 'the'Anh Tran2016-09-271-2/+2
| |/ |/| | | | | Change-Id: I3b3c0e7e4d5c8a1934378e2083a64bb686754f18
* | Merge "Accept gzip-encoded API responses"Jenkins2016-09-011-3/+5
|\ \
| * | Accept gzip-encoded API responsesTim Burke2016-08-301-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, we would accept gzip-encoded responses, but only because we were letting requests decode *all* responses (even object data). This restores the previous capability, but with tighter controls about which requests will accept gzipped responses and where the decoding happens. Change-Id: I4fd8b97207b9ab01b1bcf825cc16efd8ad46344a Related-Bug: 1282861 Related-Bug: 1338464
* | | Merge "Convert numeric and boolean header values to strings"Jenkins2016-08-311-0/+7
|\ \ \ | |/ / |/| |
| * | Convert numeric and boolean header values to stringsTim Burke2016-08-251-0/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Recently, requests got a bit more picky about what types of data it will accept as header values [1]. The reasons for this are generally sound; str()ing arbitrary objects just before pushing them out a socket may not do what the developer wanted/expected. However, there are a few standard types that developers may be sending that we should convert for them as a convenience. Now, we'll convert all int, float, and bool values to strings before sending them on to requests. Change-Id: I6c2f451009cb03cb78812f54e4ed8566076de821 Closes-Bug: 1614932
* | | Merge "client: renew token on 401 even if retries is 0"Jenkins2016-08-261-4/+4
|\ \ \ | |/ / |/| |
| * | client: renew token on 401 even if retries is 0Julien Danjou2016-06-071-4/+4
| |/ | | | | | | | | | | | | | | | | | | | | | | Gnocchi uses a client with retries=0 to maximize throughtput and not retry N times on e.g. 404 when checking existence of an object. However, this as the side effect of never renewing the token since there' no retry on 401 either. This patches change the behavior so that 401 errors are always retried, whatever the retries value is. Closes-Bug: #1589926 Change-Id: Ie06adf4cf17ea4592b5bbd7bbde9828e5e134e3e
* | Merge "Add copy object method"Jenkins2016-08-241-0/+71
|\ \
| * | Add copy object methodMarek Kaleta2016-08-231-0/+71
| |/ | | | | | | | | | | | | | | | | | | | | Implement copy object method in swiftclient Connection, Service and CLI. Although COPY functionality can be accomplished with 'X-Copy-From' header in PUT request, using copy is more convenient especially when using copy for updating object metadata non-destructively. Closes-Bug: 1474939 Change-Id: I1338ac411f418f4adb3d06753d044a484a7f32a4
* | Merge "Query string functionality for containers"Jenkins2016-06-141-9/+25
|\ \ | |/ |/|
| * Query string functionality for containersAndrew Welleck2016-06-091-9/+25
| | | | | | | | | | | | | | | | | | Added functionality for arbitrary query strings to be passed into container functions. Additionally a minor typo correction in the README. Added unit tests for query string functionality. Closes-Bug: #1542459 Change-Id: Ica2cb3ea439632588388e748d8d2e944e9ed4fa4
* | Merge "Support client certificate/key"Jenkins2016-05-191-3/+36
|\ \
| * | Support client certificate/keyCedric Brandily2016-04-101-3/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | This change enables to specify a client certificate/key with: * usual CLI options (--os-cert/--os-key) * usual environment variables ($OS_CERT/$OS_KEY) Closes-Bug: #1565112 Change-Id: I12e151adcb6084d801c6dfed21d82232a3259aea
* | | Correct doc string for http_conn argumentKota Tsuyuzaki2016-05-121-27/+27
| |/ |/| | | | | | | | | | | | | Previous doc string describes "http_conn" for a bunch of functions as "HTTPConnection object" but it is actually "a tuple of (parsed_url, HTTPConnection object)" so now this patch correct the mis-consistency. Change-Id: I6ff90a941e6df6a1b3b5dc74fd5f4262654c9f0a
* | Merge "Check responses when retrying bodies"Jenkins2016-05-111-0/+17
|\ \ | |/ |/|
| * Check responses when retrying bodiesTim Burke2016-05-041-0/+17
| | | | | | | | | | | | | | | | | | | | | | | | Previously, if a Range request came back 200 OK (rather than 206 Partial Content), we would mangle the response body. This could happen if there was a middleware that would silently drop Range headers, for example. Now, if the response does not include a Content-Range header, we will log a warning and seek to our previous position in the stream. If the Content-Range header has an unexpected value, we will raise an exception. Change-Id: I94d4536cc1489968d45a2b6ba7edd70c85800275
* | Include response headers in ClientExceptionsTim Burke2016-03-031-71/+19
| | | | | | | | | | | | | | | | | | | | | | Now, client applications can get to things like transaction IDs for failures without needing to turn on all of logging. While we're at it, add a from_response factory method for ClientException. Co-Authored-By: Alexander Corwin <ancorwin@gmail.com> Change-Id: Ib46d5f8fc7f36f651f5908bb9d900316fdaebce3
* | Merge "Fix wrong args for get_container with full listing"Jenkins2016-02-291-2/+2
|\ \
| * | Fix wrong args for get_container with full listingAlistair Coles2016-02-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In client get_container(), when full_listing is true, the calls back to get_container() pass service_token as a positional arg which maps its value to the full_listing arg. It should use a keyword. Change-Id: Iac2af45df124ff33fcb7fbaf1ba959ef06c96378 Closes-Bug: #1496093
* | | Merge "Force header keys/values to bytes/unicode before coercing to unicode"Jenkins2016-02-271-0/+2
|\ \ \
| * | | Force header keys/values to bytes/unicode before coercing to unicodeTim Burke2016-02-261-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, parse_header_string was only called with data coming out of requests, which would be either bytes or unicode. Now that we're sending it request headers as well (see related change), we need to be more defensive. If the value given is neither bytes nor unicode, convert it to a native string. This will allow developers using the client API to continue sending header dicts like {'X-Delete-After': 2} ...as in Swift's test/probe/test_object_expirer.py Change-Id: Ie57a93274507b184af5cad4260f244359a585f09 Related-Change: I43dd7254f7281d4db59b286aa2145643c64e1705
* | | | Fix test for redacting sensitive data in client.http_log()Joel Wright2016-02-261-1/+1
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | The test should have included utf8 encoded unicode data to test that encoded unicode data stored in headers was parsed correctly. Also fixes the docstring for swiftclient.safe_value() Change-Id: Id0def0b3af7a364f1257cc22f67b71c0cc5d8479
* | | Follow-up to patch 282363Tim Burke2016-02-251-7/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | * Improve some formatting * Be more explicit about how much will be revealed when * Rename redact_sensitive_tokens to redact_sensitive_headers, as it affects more than tokens. Change-Id: I02b375d914e9f0a210d038ecb31188d09a8ffce3
* | | Do not reveal auth token in swiftclient log messages by defaultJoel Wright2016-02-221-6/+65
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the swiftclient logs sensitive info in headers when logging HTTP requests. This patch hides sensitive info in headers such as 'X-Auth-Token' in a similar way to swift itself (we add a 'reveal_sensitive_prefix' configuration to the client). With this patch, tokens are truncated by removing the specified number of characters, after which '...' is appended to the logged token to indicate that it has been redacted. Co-Authored-By: Li Cheng <shcli@cn.ibm.com> Co-Authored-By: Zack M. Davis <zdavis@swiftstack.com> Change-Id: I43dd7254f7281d4db59b286aa2145643c64e1705 Closes-bug: #1516692
* | Merge "_RetryBody doesn't need to take explicit etag/content-length"Jenkins2016-02-181-11/+7
|\ \ | |/
| * _RetryBody doesn't need to take explicit etag/content-lengthTim Burke2016-01-111-11/+7
| | | | | | | | | | | | | | Also, don't try to do int(None) for chunk-encoded responses (like DLOs that are longer than a single container listing). Change-Id: Ibacd75d5ee46135d62388786903c895fda8ed3ba
* | Merge "Accept token and tenant_id for authenticating against KS"Jenkins2016-02-101-0/+1
|\ \
| * | Accept token and tenant_id for authenticating against KSPratik Mallya2016-01-181-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Allow swiftclient to authenticate against keystone using tenant name/id and token only. Without this patch, the password is required, which may not always be available. Authentication against keystone is required to get the service catalog, which includes the endpoints for swift. Change-Id: I4477af445474c5fa97ff864c4942f1330b59e5d6 Closes-Bug: #1476002
* | | Merge "Use bulk-delete middleware when available"Jenkins2016-01-221-3/+15
|\ \ \
| * | | Use bulk-delete middleware when availableTim Burke2016-01-121-3/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When issuing `delete` commands that would require three or more individual deletes, check whether the cluster supports bulk deletes and use that if it's available. Additionally, a new option is added to the `delete` command: * --prefix <prefix> Delete all objects that start with <prefix>. This is similar to the --prefix option for the `list` command. Example: $ swift delete c --prefix obj_prefix/ ...will delete from container "c" all objects whose name begins with "obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar". Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
* | | | Merge "Fix the http request headers being overwritten in logging"Jenkins2016-01-151-1/+0
|\ \ \ \ | |/ / / |/| | |
| * | | Fix the http request headers being overwritten in loggingMin Min Ren2016-01-141-1/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Fix the http request headers in put_object being overwritten in logging Change-Id: Id0d1e36561a61ed1ce30d93c801ec32f058a6fa4 Closes-bug: #1501292
* | | | Merge "Retry file uploads via SwiftService"Jenkins2016-01-121-1/+3
|\ \ \ \ | |_|_|/ |/| | |
| * | | Retry file uploads via SwiftServiceTim Burke2016-01-111-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When we introduced LengthWrapper, we neglected to make it resettable. As a result, upload failures result in errors like: put_object(...) failure and no ability to reset contents for reupload. Now, LengthWrappers will be resettable if their _readable has seek/tell. Related-Change: I6c8bc1366dfb591a26d934a30cd21c9e6b9a04ce Change-Id: I21f43f06e8c78b24d1fc081efedf2687942e042f
* | | | Retry download of object bodyStuart McLaren2016-01-081-6/+82
|/ / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently the swift client retries establishing a connection to the server (by default up to 5 times). However, when downloading an object, once the connection has been established and the inital headers have been returned, no attempt is made to retry. So, for example, if 99MB of a 100MB object have been downloaded and the connection is then lost, the download will fail. This patch changes the behaviour to re-establish the connection and fetch the remaining bytes using the 'Range' header to offset. Data retry is not yet supported if the original request is for a subset of the object data (ie uses the 'Range' header), or if resp_chunk_size has not been set. The object's etag is checked using If-Match to make sure the object data hasn't changed since the start of the download. Change-Id: Iab47f10081ff39f6d344dbc2479cbc3bfd1c5b29
* | | Merge "Wrap raw iterators to ensure we send entire contents to server"Jenkins2016-01-061-1/+5
|\ \ \
| * | | Wrap raw iterators to ensure we send entire contents to serverTim Burke2015-12-301-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, if you attempt to stream an upload from an iterator, as in def data(): yield 'foo' yield '' yield 'bar' conn.put_object('c', 'o', data()) ... requests will faithfully emit a zero-length chunk, ending the transfer. Swift will then close the connection, possibly (if Connection: keep-alive was set) after attempting to parse the next chunk as a new request. Now, Swift will receive all of the bytes from the iterable, and any zero-byte chunks will be ignored. This will be fixed in requests [1], but not until an eventual 3.0.0 release. [1] https://github.com/kennethreitz/requests/pull/2631 Change-Id: I19579ed7a0181ac3f488433e7c1839f7f7a040b8
* | | | Fix some typosTim Burke2015-12-301-4/+4
|/ / / | | | | | | | | | Change-Id: Iaf7f30a7ae0c2ac76fc5cdcee31ea74c08ce601e
* | | Wrong usage of "an"hgangwx2015-12-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Wrong usage of "an" in the messages: "the optional os_options paramater includes an non-empty" "We are allowing to have an tenant_name argument" Should be: "the optional os_options paramater includes a non-empty" "We are allowing to have a tenant_name argument" Totally 2 occurrences in python-swiftclient base code. Change-Id: I2f2f7e07432fedfee5ccb418d9505250b3fed597
* | | Merge "Stop passing attr to keystoneclient when there's no filter_value"Jenkins2015-12-221-4/+7
|\ \ \
| * | | Stop passing attr to keystoneclient when there's no filter_valueTim Burke2015-12-071-4/+7
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | It was dropping warnings like "UserWarning: Providing attr without filter_value to get_urls() is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release. Either both should be provided or neither should be provided." Change-Id: Iead0bcf36b4a46bf465a55a33a21fd7f14f0ac40
View Lorry Controller Status