summaryrefslogtreecommitdiff
path: root/swiftclient/shell.py
Commit message (Collapse)AuthorAgeFilesLines
* Follow-up to patch 282363Tim Burke2016-02-251-1/+1
| | | | | | | | | * Improve some formatting * Be more explicit about how much will be revealed when * Rename redact_sensitive_tokens to redact_sensitive_headers, as it affects more than tokens. Change-Id: I02b375d914e9f0a210d038ecb31188d09a8ffce3
* Do not reveal auth token in swiftclient log messages by defaultJoel Wright2016-02-221-0/+2
| | | | | | | | | | | | | | | | Currently the swiftclient logs sensitive info in headers when logging HTTP requests. This patch hides sensitive info in headers such as 'X-Auth-Token' in a similar way to swift itself (we add a 'reveal_sensitive_prefix' configuration to the client). With this patch, tokens are truncated by removing the specified number of characters, after which '...' is appended to the logged token to indicate that it has been redacted. Co-Authored-By: Li Cheng <shcli@cn.ibm.com> Co-Authored-By: Zack M. Davis <zdavis@swiftstack.com> Change-Id: I43dd7254f7281d4db59b286aa2145643c64e1705 Closes-bug: #1516692
* Merge "Display proper name when failing to create segments container"Jenkins2016-02-171-1/+1
|\
| * Display proper name when failing to create segments containerTim Burke2016-02-101-1/+1
| | | | | | | | | | | | Previously, we displayed the base container's name. Change-Id: I70f1949a44ba61158e31178e4536f229c37aab47
* | Merge "Fix debug and info option parsing"Jenkins2016-02-111-8/+7
|\ \
| * | Fix debug and info option parsingChristian Schwede2015-12-301-8/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The debug and info options need to be set before a subcommand method is called, otherwise they are simply ignored. This is kind of irritating - other options (for example -U, -A, -K) are usable after a positional command. This patch fixes this, and commands like these are no longer ignoring --debug or --info: swift stat --debug swift list container --info Co-Authored-By: Alistair Coles <alistair.coles@hpe.com> Change-Id: Ib19b05deef7a015881f1eed4a3946025e16bf922
* | | Support --os-identity-api-version optionAlistair Coles2016-02-051-6/+17
| |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add support for the auth-version to be specified using --os-identity-api-version or OS_IDENTITY_API_VERSION for compatibility with other openstack client command line options. The auth version used will be selected as follows: - if either --auth-version or --os-identity-api-version is set, use that value - otherwise use the value of ST_AUTH_VERSION, if set - otherwise use the value of OS_AUTH_VERSION, if set - otherwise (new behaviour) use the value of OS_IDENTITY_API_VERSION, if set - otherwise default to 1.0 Note that before this change the auth version might have defaulted to 1.0 despite OS_IDENTITY_API_VERSION being set, but with this change OS_IDENTITY_API_VERSION is preferred. Change-Id: Ifba4c4e43560ede3013337b8cdbc77dc2de6e8ff Closes-Bug: #1541273
* | Merge "Convert http response(byte string) to string in python3."Jenkins2016-01-221-1/+2
|\ \
| * | Convert http response(byte string) to string in python3.Charles Hsu2015-09-061-1/+2
| | | | | | | | | | | | | | | | | | | | | Avoid a TypeError exception in python3. Change-Id: I4039e3f2a88b5f681288b5ca8dd5c63c13b7764f Closes-bug: #1457012
* | | Use bulk-delete middleware when availableTim Burke2016-01-121-15/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When issuing `delete` commands that would require three or more individual deletes, check whether the cluster supports bulk deletes and use that if it's available. Additionally, a new option is added to the `delete` command: * --prefix <prefix> Delete all objects that start with <prefix>. This is similar to the --prefix option for the `list` command. Example: $ swift delete c --prefix obj_prefix/ ...will delete from container "c" all objects whose name begins with "obj_prefix/", such as "obj_prefix/foo" and "obj_prefix/bar". Change-Id: I6b9504848d6ef562cf4f570bbcd17db4e3da8264
* | | Fix some typosTim Burke2015-12-301-1/+1
| |/ |/| | | | | Change-Id: Iaf7f30a7ae0c2ac76fc5cdcee31ea74c08ce601e
* | Merge "Unification of manpages and docstrings"Jenkins2015-12-011-9/+12
|\ \
| * | Unification of manpages and docstringsPeter Lisák2015-11-131-9/+12
| | | | | | | | | | | | | | | | | | | | | * adding missing options * unification of help Change-Id: I2365e66433b63de8fd4da205611d9c1bf3bb6730
* | | Merge "Suppress iso8601 logging from --debug output"Jenkins2015-12-011-0/+1
|\ \ \ | |/ / |/| |
| * | Suppress iso8601 logging from --debug outputQiu Yu2015-09-121-0/+1
| | | | | | | | | | | | | | | | | | | | | This change silences logging from iso8601 when --debug option set Change-Id: Ib8b8423012d43ef78d7138609fa98f40d46e7d4b Closes-bug: #1324470
* | | Add content-type in list of container contentPeter Lisák2015-11-111-1/+3
| | | | | | | | | | | | Change-Id: Ie0787d5ffbee0a7d2429cb285fa6ecdf722e4ae1
* | | Miscellaneous (mostly test) cleanupTim Burke2015-10-071-10/+10
|/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Always use testtools.TestCase, since we're relying on testtools * Always use mock (as opposed to unittest.mock) since we're relying on mock * Add note about when a missing logging handler was added * Stop %-formatting the giant usage string that doesn't actually need any formatting * Prefer assertIs, assertIn, assertIsInstance over assertTrue * Use else-self.fail instead of sentinel values * Check resp.get('error') is None before checking resp['success'] is True, so test failures actually tell you something useful * Tighten some isinstance assertions * Import MockHttpTest from correct location * Only populate clean_os_environ once * Use setUp for setup, not __init__ * Replace assertIn(key, dict) and assertEqual(foo, dict[key]) with assertEqual(foo, dict.get(key)) when key is a literal and foo is not None * Use mock.patch.object instead of manually patching for tests * Use six.binary_type instead of type(''.encode('utf-8')) * Stop shadowing builtin bytes * Reclaim some margin * Stop checking the return-type of encode_utf8; we already know it's bytes Change-Id: I2138ea553378ce88810b7353147c8645a8f8c90e
* | absolute expiry option for tempURL generationZack M. Davis2015-09-041-3/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The `tempurl` subcommand's second positional argument is called `seconds` and has heretofore interpreted as the number of seconds for which the tempURL should be valid, counting from the moment of running the command. This is indeed a common, if not the most common, use-case. But some users, occasionally, might want to generate a tempURL that expires at some particular ("absolute") time, rather than a particular amount of time relative to the moment of happening to run the command. (One might make an analogy to the way in which Swift's expiring object support supports an `X-Delete-At` header in addition to `X-Delete-After`—and it's the former that must be regarded as ontologically prior.) Thus, this commit adds an `--absolute` optional argument to the `tempurl` subcommand; if present, the `seconds` argument will be interpreted as a Unix timestamp of when the tempURL should be expire, rather than a duration for which the tempURL should be valid starting from "now". Change-Id: If9ded96f2799800958d5063127f3de812f50ef06
* | Merge "Reduce memory usage for download/delete and add --no-shuffle option ↵Jenkins2015-08-281-0/+18
|\ \ | | | | | | | | | to st_download"
| * | Reduce memory usage for download/delete and add --no-shuffle option to ↵Joel Wright2015-07-201-0/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | st_download The current code builds a full object listing before performing either a multiple download or delete operation (and also shuffles this complete list in the case of a download). This patch removes the creation of the full object list and adds the ability to turn off shuffle for files when downloading. Also added is a limit on the number of list results that can be queued by a single call to service.list without consuming results (reduces memory overhead for large listings). Some tests added for service.py download and list. Change-Id: Ie737cbb7f8b1fa8a79bbb88914730b05aa7f2906
* | | flake8 ignores same hacks as swiftHiroshi Miura2015-08-181-4/+4
| |/ |/| | | | | | | | | | | | | | | | | | | - blacklisted flake8 hacking - fix against E122 continuation line missing indentation or outdented Closes-bug: #1475516 Change-Id: I708d0a3466a1f85c84e478873e142821ce0774cb Signed-off-by: Hiroshi Miura <miurahr@nttdata.co.jp>
* | Merge "swiftclient: add short options to help message"Jenkins2015-07-311-34/+43
|\ \ | |/ |/|
| * swiftclient: add short options to help messageHiroshi Miura2015-07-171-34/+43
| | | | | | | | | | | | | | | | | | | | | | | | - add usage strings for short option such as '-a' for '--all' This add all short options to usage text. - add missing --header usage help for download command - some cometic changes Closes-bug: #1475511 Change-Id: Ibfecac8764669540fa025787548133a50fa50b10 Signed-off-by: Hiroshi Miura <miurahr@nttdata.co.jp>
* | Merge "Always decode command-line arguments as UTF-8"2.5.0Jenkins2015-07-161-6/+9
|\ \ | |/ |/|
| * Always decode command-line arguments as UTF-8Tim Burke2015-07-071-6/+9
| | | | | | | | | | | | | | | | | | There was always an implicit assumption that they were UTF-8 before, and by converting them to unicode we close another hole allowing raw bytes to appear in user-facing messages. Closes-Bug: #1431866 Change-Id: If2e41d9a592c3ad02818e9c6f0959fd4b73cd0e0
* | Merge "Correct the help message of swift tempurl"Jenkins2015-07-111-4/+4
|\ \
| * | Correct the help message of swift tempurlYangLei2015-06-301-4/+4
| | | | | | | | | | | | | | | | | | | | | correct the help message of swift tempurl use <> instead of [] in Positional arguments. Change-Id: Ib60ce97cef03e0423082c497604525eba2300fa9
* | | Merge "Add some bash helpers for auth stuff"Jenkins2015-07-111-4/+50
|\ \ \
| * | | Add some bash helpers for auth stuffClay Gerrard2015-06-151-4/+50
| | | | | | | | | | | | | | | | Change-Id: If61ac9a050e7a115f37dbf4e74b904ac5dfd2052
* | | | Merge "Add ability to download objects to particular folder."Jenkins2015-07-111-2/+24
|\ \ \ \ | |_|_|/ |/| | |
| * | | Add ability to download objects to particular folder.Charles Hsu2015-07-081-2/+24
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds "--output-dir" and "--remove-prefix" options to the "download" command and unit tests for it. Example: $ swift list example --prefix swift2.2 swift2.2/bin/swift-object-auditor swift2.2/bin/swift-object-expirer swift2.2/bin/swift-object-info swift2.2/bin/swift-object-replicator swift2.2/bin/swift-object-server swift2.2/bin/swift-object-updater When given "--output-dir <directory>", client downloads objects to <directory>. $ swift download example --prefix swift2.2 \ --output-dir new/swift/dir The folder structure: . └── new └── swift └── dir └── swift2.2 └── bin ├── swift-object-auditor ├── swift-object-expirer ├── swift-object-info ├── swift-object-replicator ├── swift-object-server └── swift-object-updater When given "--remove-prefix", client downloads objects without <prefix>. $ swift download example --prefix swift2.2 \ --remove-prefix \ --output-dir swift The folder structure: . └── swift └── bin ├── swift-object-auditor ├── swift-object-expirer ├── swift-object-info ├── swift-object-replicator ├── swift-object-server └── swift-object-updater Co-Authored-By: Clay Gerrard <clay.gerrard@gmail.com> Change-Id: I7463fe2941cc94f9a50a4756a97c2ccdf946294d Implements: blueprint swiftclient-download-pseudo-folder-to-specific-target
* | | Added check for negative segment-sizePradeep Kumar Singh2015-06-231-0/+3
| |/ |/| | | | | | | | | Closes-Bug: #1453135 Change-Id: Ia9c2b27d998e6ac1889cc74c12e456c06ecd84d9
* | Fix inconsistent usage of "Positional argument"Andreas Jaeger2015-06-081-1/+1
|/ | | | | | | | | | | All help texts uses "Positional argument" with the exception of tempurl. Update tempurl to use this as well so that the formatting tools work fine and can show this nicely on http://docs.openstack.org/cli-reference/content/swiftclient_commands.html#swiftclient_subcommand_tempurl like it's done for other options. Change-Id: Ib5502c23b236986bea5a4d4a63a46fca411a8494 Closes-Bug: #1463081
* be more explicit in the --version CLI optionJohn Dickinson2015-05-131-1/+1
| | | | Change-Id: I14f33c7ea90ab1fe58aa67f1ff70e527c88cd141
* Mention --segment-size option after 413 response2.4.0Tim Burke2015-03-051-0/+6
| | | | | | | | | | | | Previously, it would only output something like ClientException: Object PUT failed <url> 413 Request Entity Too Large Your request is too large. ... which is misleading as it implies that Swift simply can't handle large objects. Change-Id: Icec9e76b2671490c3a2249ec08992950107c6828
* Merge "Unindent a chunk of st_list"Jenkins2015-03-051-57/+52
|\
| * Unindent a chunk of st_listPete Zaitcev2015-02-261-57/+52
| | | | | | | | | | | | | | | | The chunk looks like nicely containable, but it pushes against the right edge of the screen annoyingly. This patch _only_ prettifies, without any observable changes. Change-Id: I96fcb5848e3e5e0e8a8ebbe398b1d4a0bea5fa24
* | Add improvements to MD5 validation.Daniel Wakefield2015-03-041-2/+6
|/ | | | | | | | | | | | | | | With MD5Sum checking being added, a concern was brought up that It was a change with no possibility of reverting to the old behaviour. This change adds the flag '--ignore-checksum' to the upload subcommand allowing the checks to be turned off. Changed occurrences of the magic string for a null md5 to use a descriptive constant instead. Updated Error messages generated when validation fails. They should now be more descriptive and not output a literal newline sequence. Change-Id: Id1756cbb6700bb7e38f0ee0e75bc535e37f777ed
* Fix crash with -l, -d /, and pseudo foldersPete Zaitcev2015-02-181-3/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | Using -l, -d /, or a pseduo folder by itself works fine, but if you combine all of them, this happens: [zaitcev@guren s]$ swift list -d / testcont amazon_lili.jpg pseudodir/ rin_shibuya_pixiv.jpg swift-c134855-5a.diff [zaitcev@guren s]$ swift list -d / -l testcont 151954 2014-06-20 17:13:52 amazon_lili.jpg Traceback (most recent call last): File "/q/zaitcev/hail/python-swiftclient-tip/bin/swift", line 24, in <module> sys.exit(main()) File "/q/zaitcev/hail/python-swiftclient-tip/swiftclient/shell.py", line 1301, in main globals()['st_%s' % args[0]](parser, argv[1:], output) File "/q/zaitcev/hail/python-swiftclient-tip/swiftclient/shell.py", line 414, in st_list total_bytes += item_bytes TypeError: unsupported operand type(s) for +=: 'int' and 'NoneType' The patch is a straightforward fix that assigns expected variables. Change-Id: If24dcc2491e8b8982455107d2aca30c0491a398a Closes-Bug: 1399678
* Merge "This patch fixes downloading files to stdout."Jenkins2015-02-031-3/+4
|\
| * This patch fixes downloading files to stdout.Joel Wright2015-01-231-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | This patch fixes downloading files to stdout and modifies _SwiftReader to operate as an iterator that performs file checks at the end of iteration as well as a context manager. File verification checks have been removed from __exit__ and added to __iter__. Change-Id: I3250bdeeef8484a9122c4b5b854756a7c8f8731e Closes-Bug: 1395922 Closes-Bug: 1387376
* | Fix cross account upload using --os-storage-urlAlistair Coles2015-01-061-22/+15
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removes an account stat from the object upload path. This stat fails when user is not account admin even though the user may have container ACL permission to write objects. Reduces the severity of the CLI output message when upload fails to create the given container (this is not an error since the container may exist - the user just does not have permission to PUT or POST the container). Changes the 'swift upload' exit return code from 1 to 0 if container PUT fails but object PUT succeeds. For segment uploads, makes the attempt to create the segment container conditional on it not being the same as the manifest container. This avoids an unnecessary container PUT. Fixes another bug that became apparent: with segmented upload a container HEAD may be attempted to determine the policy to be used for the segment container. When this failed the result dict has headers=None which was causing an exception in the shell result handler. Add unit tests for object upload/download and container list with --os-storage-url option. Closes-Bug: #1371650 Change-Id: If1f8a02ee7459ea2158ffa6e958f67d299ec529e
* Print info message about incorrect --totals usageJiří Suchomel2015-01-051-0/+5
| | | | | | | | when neither -l nor --lh is provided. Added test coverage for --totals. Change-Id: I3245e715c26ec28457a21dec07311a58c475c066 Closes-Bug: 1258392
* Merge "Change tests to use new CaptureOutput class."Jenkins2014-12-301-1/+1
|\
| * Change tests to use new CaptureOutput class.Daniel Wakefield2014-12-051-1/+1
| | | | | | | | | | | | | | | | | | | | Added the ability to clear the buffers in the CaptureOutput class so it can be easily used multiple times in the same context manager. Also added a option to suppress the SystemExit associated with printing an error. Change-Id: Ib59bbbe88256f215eed0a8ebc8282e02181d4377
* | Fix misnamed dictionary key.Daniel Wakefield2014-12-081-21/+16
|/ | | | | | | | | | | | | The response dictionary in _delete_segment attach's any error it encounters to the dict key 'exception', all other response dict's use 'error' to store the exception. Changed to make it consistent and added tests Also added a third branch in st_delete as messages created in _delete_container where being silently dropped. Change-Id: Ifbc3b1fae78910fbc6acf4a86cfb0f60bb1aa336
* Merge "Make swift post output an error message when failing"Jenkins2014-11-131-7/+7
|\
| * Make swift post output an error message when failingAlistair Coles2014-10-241-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | swiftclient.shell.st_post was ignoring the result dict returned from SwiftService.post and therefore giving no indication when posts failed e.g. due to invalid auth credentials. This patch ensures that SwiftService always returns a result_dict from post() and then checks the result dict. On failure the shell now outputs the ClientException message and exits with error code 1. Also adds unit tests and cleans up some unnecessary mocked return values in existing tests. Closes-Bug: 1383243 Change-Id: I111150eb3d026c8d01c2cac13d3613ca7304e5b9
* | Replaces Stacktraces with useful error messages.Daniel Wakefield2014-10-211-11/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Changed the message shown when a user doesn't enter project/tenant id/name to be more informative. When attempting to stat a container without supplying project/tenant name or id, an empty response was being returned instead of an error being raised. Changed the error raised in swiftclient.client when no tenant or project is specified to be more specific. Add tests for basic regression checking. Closes-Bug: #1372589 Change-Id: I4eb6964d9f1702db119cc0294edc02841b1ecd5f
* | Fix race in shell when testing for errors to raise SysExitAlistair Coles2014-10-201-3/+1
|/ | | | | | | | | | | | | | | | | SysExit is raised if the OutputManager has non-zero error_count. However, error_count is incremented in a print thread, but tested on the main thread. Currently error_count is tested before the OutputManager print threads have exited, which results in a race. This race means that SysExit is not always raised even when an error has occured. This change moves the error_count test after the OutputManager context manager has exited, which ensures that the OutputManager threads have exited. Change-Id: I5ef7d9f10057fe2b41f48ab95066a7265b86a3ac Closes-Bug: #1379229
View Lorry Controller Status