summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Merge "Remove a debugging print statement"2.3.1Jenkins2014-09-261-1/+0
|\
| * Remove a debugging print statementJoel Wright2014-09-251-1/+0
| | | | | | | | | | | | | | This patch removes a debugging print statement that slipped into service.py in patch https://review.openstack.org/#/c/85453/ Change-Id: I80eee49eaa74b52e9c36f00669206aa7b2cdec9b
* | Merge "Fix unit tests failing when OS_ env vars are set"Jenkins2014-09-261-4/+5
|\ \
| * | Fix unit tests failing when OS_ env vars are setAlistair Coles2014-09-251-4/+5
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tests/unit/test_shell.py:TestParsing tests can fail if there are OS_* variables set in the environment. There is already code in the test setUp to remove ST_* variables, so we should do the same for OS_* variables. This patch also changes the mechanism used to remove and then restore any unwanted variables found in os.environ. The existing setUp() takes a copy of os.environ and then deletes any ST_* variables in the original. In tearDown() it sets os.environ as the copy. However, the environ imported into shell.py remains pointing to the original os.environ object. So after the first call to tearDown, subsequent mocking of os.environ has no effect on shell.environ. This renders some of the tests ineffective e.g. test_insufficient_env_vars_v3 is not actually setting any vars in shell.environ. The issue can be provoked by repeating a test: nosetests -w tests/unit/ test_shell.py:TestParsing.test_args_v3 \ test_shell.py:TestParsing.test_args_v3 The test will pass first time and fail second time. Change-Id: I5d100f81115e74878d510326acb5777e6a3626c8
* | Merge "Add per policy container count to account stat output"Jenkins2014-09-252-0/+34
|\ \
| * | Add per policy container count to account stat outputClay Gerrard2014-09-192-0/+34
| |/ | | | | | | | | | | | | | | | | | | When the account response includes 'X-Storage-Policy-X-Container-Count' the command helpers will stick a helpful extra line in the output. If the swift response doesn't include the information the behavior is unchanged. Change-Id: Iaaa55d207ebe6a9c16e52adb56eb858c57cfb26a
* | Fix bug with some OS options not being passed to clientAlistair Coles2014-09-245-105/+408
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixes a regression that is preventing swiftclient from authenticating using keystone v3 options. swiftclient/shell.py processes the dict of command line options to set up a child dict ('os_options') of keystone auth specific options (line 960). The processing includes stripping --os- prefixes from the command line options before adding keys to os_options. A recent patch https://review.openstack.org/#/c/85453/ introduced a duplication of this option processing in service.py (line 79) which replaces the os_options created in shell.py, but omits keystone v3 related options. Consequently the keystone v3 options are not being passed to the client get_auth() method. This patch adds the keystone v3 related options to the option processing in service.py. For pragmatic reasons (i.e. fixing the bug quickly) the option processing code has not been removed from parse_args in shell.py. It is likely that the code in parse_args is now redundant, but all code paths between parse_args and process_options should be inspected and test coverage added before removing that code. Unit tests have been added in test_shell.py to verify that command line options are correctly passed to the client get_auth method. The MockHttpTest class is re-used in test_shell.py, so it is moved from test_swiftclient.py to tests/unit/utils.py Closes-bug: #1372465 Change-Id: I4fed013cdb8936509609d06093337cc147ade0d6
* Stop creating extraneous directoriesPete Zaitcev2014-09-171-17/+12
| | | | | | | | | | | | | | | | Cannot help streamlining the code a bit, sorry. But the meat is that we should not try to make directories if output path is explicit. Previously we created directories using the URL path, which is obviously wrong if explicit output file is supplied... unless a crafty user supplied the same path with -o that is contained in the URL path. If anyone was doing such tricks, it's not going to work anymore (we are forcing a regression for the sake of theoretical correctness here). Fixes bug: 1369546 Change-Id: Ifce31f2ba233eb55550f3810348bf16bf2447d62
* Work toward Python 3.4 support and testing2.3.0Jeremy Stanley2014-09-031-1/+1
| | | | Change-Id: Ie213bec8d12e277d8c3d18ccc127831004d710a9
* Merge "Adds console script entry point"Jenkins2014-08-281-0/+5
|\
| * Adds console script entry pointAlessandro Pilotti2014-08-211-0/+5
| | | | | | | | | | | | | | | | | | | | | | A Swift client executable needs to be provided on Windows since Python scripts cannot be executed directly. This approach is also consistent with other OpenStack Python clients (e.g. Nova, Neutron, Cinder, etc.). Change-Id: I7f03b7c305dceed4fa6f4fe1c1281659820606e4 Closes-Bug: #1359360
* | Add importable SwiftService incorporating shell.py logicJoel Wright2014-08-269-1496/+2985
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch adds a SwiftService class that incorporates the high level logic from swiftclient/shell.py. It also ports shell.py to use the new class, and updates the code in swiftclient/multithreading.py to allow the SwiftService to be used for multiple operations whilst using only one thread pool. Currently, code that imports swiftclient has to have its own logic for things like creating large objects, parallel uploads, and parallel downloads. This patch adds a SwiftService class that makes that functionality available in Python code as well as through the shell. Change-Id: I08c5796b4c01001d79fd571651c3017c16462ffd Implements: blueprint bin-swift-logic-as-importable-library
* | Merge "fixed unit tests when env vars are set"Jenkins2014-08-211-0/+11
|\ \ | |/ |/|
| * fixed unit tests when env vars are setJohn Dickinson2014-08-181-0/+11
| | | | | | | | Change-Id: Ia73c10c7c81b9434895941ea29282dbcdc667556
* | Do not create an empty directory 'pseudo/'Pete Zaitcev2014-08-181-1/+2
|/ | | | | | | | This is a fix-up for the commit 3d56b65c. The code change was good, but the additional test had a side effect of creating an empty directory "pseudo/". Fix this by mocking-out mkdirs(). Change-Id: Iead55e72d8a75d96bde0a7491aca12bbdcc269cf
* Merge "Fix crash when downloading a pseudo-directory"Jenkins2014-08-172-15/+30
|\
| * Fix crash when downloading a pseudo-directoryPete Zaitcev2014-08-142-15/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If a user creates an object with name ending with a slash, then downloading such container ends in a traceback like this: .............. test5g.file [auth 1.516s, headers 1.560s, total 244.565s, 22.089 MB/s] Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/swiftclient/multithreading.py", lin result = self.func(item, *self.args, **self.kwargs) File "/usr/lib/python2.6/site-packages/swiftclient/shell.py", line 403, in fp = open(path, 'wb') IOError: [Errno 21] Is a directory: 'first-pseudo-folder/' The proposed fix is not to save this object. Note that the contents of the object are available with --output option, as before. Only the crash is fixed. Even though we do not use the contents, we download the object and check its Etag, in case. We also create a corresponding directory, in case the pseudo-directory contains no objects. The format of printout is changed, so user realizes easier when pseudo-directory convention is in effect. Note that this is not a compatibility issue because previously there was crash in such case. Change-Id: I3352f7a4eaf9970961af0cc84c4706fc1eab281d
* | Merge "Add keystone v3 auth support"Jenkins2014-08-156-89/+555
|\ \ | |/ |/|
| * Add keystone v3 auth supportanc2014-07-236-89/+555
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Enables swiftclient to authenticate using the keystone v3 API, allowing user id's, user domains and tenant/project domains to be specified. Since swiftclient imports keystoneclient, the main changes in swiftclient/client.py are to selectively import the correct keystoneclient library version and pass a number of new options to it via the get_auth() function. In addition the get_keystoneclient_2_0 method has been renamed get_auth_keystone to better reflect its purpose since it now deals with both v2 and v3 use cases. In swiftclient/shell.py the new options are added to the parser. To make the default help message shorter, help for all the --os-* options (including the existing v2 options) is only displayed when explicitly requested usng a new --os-help option. A new set of unit tests is added to test_shell.py to verify the parser. A comment in tests/sample.conf explains how to configure the existing functional tests to run using keystone v3 API. Note that to use keystone v3 with swift you will need to set auth_version = v3.0 in the auth_token middleware config section of proxy-server.conf. Change-Id: Ifda0b3263eb919a8c6a1b204ba0a1215ed6f642f
* | Merge "Clean up raw policy stats in account stat"Jenkins2014-08-144-26/+85
|\ \
| * | Clean up raw policy stats in account statYuan Zhou2014-08-064-26/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Storage policy stats was not well parsed in account stat. This patch parses the stats and print out the stats in a format like below: $swift -A http://swift_cluster/auth/v1.0 -U test:tester -K testing stat Account: AUTH_test Containers: 5 Objects: 1 Bytes: 2097152 Objects in policy "golden": 1 Bytess in policy "golden": 2097152 Objects in policy "silver": 0 Bytes in policy "silver": 0 X-Timestamp: 1404697760.88809 X-Trans-Id: txec519e24b44a413abb705-0053da2dcb Content-Type: text/plain; charset=utf-8 Accept-Ranges: bytes Change-Id: I7ad0ee6d88f8393e3a93e90cd52b9b592da7072d
* | | Update theme for docsSteve Martinelli2014-07-243-2/+3
|/ / | | | | | | | | | | | | | | The current developer docs theme used is out of sync with the other openstack projects. This patch will update the docs to provide a more consistent look and feel when using developer docs Change-Id: I2c66f60f6cfdd7b7b50a457c931f11851f668fa6
* | Add a tox job for generating docsSteve Martinelli2014-07-242-0/+5
|/ | | | | | | Add a tox job to make it easier for developers to generate their own docs before submitting a patch. Change-Id: I86dbfc64a9d174697048c254c6d3d042a62350b4
* Fix context sensitive help for info and tempurl2.2.0Alistair Coles2014-07-212-22/+31
| | | | | | | | | | Make it so that swift <cmd> --help will print the info subcommand help for info and tempurl just like all the other subcommands. Also add unit tests to verify subcommand help. Change-Id: Id3666dcf72a9727fbfda2f74c23293ada1c53aa0
* Merge "Allow to specify storage policy when uploading objects"Jenkins2014-07-212-8/+55
|\
| * Allow to specify storage policy when uploading objectsYuan Zhou2014-07-172-8/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Client already supports -H/--header option when creating container or uploading objects. This patch extends this option to support Storage Policy. e.g., swift post con -H 'X-Storage-Policy:p1' This creates one container 'con' with storage policy 'p1'. swift upload con obj -H 'X-Storage-Policy:p2' This creates container 'con' with storage policy 'p2' and uploads object 'obj' into it. Also fixes segmented uploading to non-default storage policy container When uploading large objects with segmentation to container with non-default storage policy, there will be another 'xxx_segments' container created, but with the default storage policy. This results all the segments to be stored with the wrong policy. This patch is for the Storage Policy feature, and also compatible with old versions w/o Storage Policy support. Change-Id: I5c19e90604a0bcf2c85e1732b8a0b97ae6801994
* | Merge "Adding Swift Temporary URL support"Jenkins2014-07-184-2/+145
|\ \ | |/ |/|
| * Adding Swift Temporary URL supportJosh Gachnang2014-07-114-2/+145
| | | | | | | | | | | | | | | | | | | | Temporary URLs allow a user to sign an object URL with a shared secret to so that the object can be downloaded without auth for a specified amount of time. http://docs.openstack.org/trunk/config-reference/content/object-storage-tempurl.html Change-Id: Ife0b6c98c975e074d4dad0a31145573b784747c5
* | Merge "replace string format arguments with function parameters"Jenkins2014-07-111-2/+2
|\ \
| * | replace string format arguments with function parametersChristian Berendt2014-06-031-2/+2
| |/ | | | | | | | | | | | | | | There are files containing string format arguments inside logging messages. Using logging function parameters should be preferred. Change-Id: I90270908d27f0a57c184d5a547bb1af9b1381c95
* | Merge "Add context sensitive help"Jenkins2014-07-091-1/+22
|\ \
| * | Add context sensitive helpMark Seger2014-06-251-1/+22
| |/ | | | | | | Change-Id: I9aaadbaba2c43a12a83f8200eaaa96f48c52028e
* | Merge "Removed now unnecesary workaround for PyPy"Jenkins2014-07-081-5/+0
|\ \
| * | Removed now unnecesary workaround for PyPyAlex Gaynor2014-05-311-5/+0
| |/ | | | | | | Change-Id: I7bf6346edc399ec29b125b50eb7ac2cfbb87cdfa
* | Add CONTRIBUTING.mdliuqing2014-07-031-0/+13
| | | | | | | | | | | | There is no CONTRIBUTING.md file, the patch will add it. Change-Id: Idc0a9ae22f2b315375b660263c7602e3ca02d56b
* | Relax requirement for tenant_name in get_auth()anc2014-06-052-3/+21
|/ | | | | | | | | | | get_auth() in client.py raises an exception if tenant_name is not included in the os_options dict. This is overly constrained since tenant_id is equally sufficient. This patch modifies get_auth to require either tenant_name or tenant_id. Change-Id: Ibbcda1704637eb887efa5895579d260a1e072327
* Merge "change assertEquals to assertEqual"Jenkins2014-05-281-4/+4
|\
| * change assertEquals to assertEqualChristian Berendt2014-05-141-4/+4
| | | | | | | | | | | | | | According to http://docs.python.org/2/library/unittest.html assertEquals is a deprecated alias of assertEqual. Change-Id: Ibf4d548b86c53f30830b7e34b019fa8f67997cd8
* | Use Emacs-friendly coding line.Martin Geisler2014-05-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To signal the file encoding to Emacs, one can add a line with -*- coding: your-favority-encoding -*- Python also understands such a line, infact it also understands the line if it says "encoding" instead of "coding". However, it is only "coding" that has an effect in Emacs. Furthermore, since "encoding" is not recognized, Emacs will prompt the user with a scary looking message when the file is opened: The local variable list in __init__.py contains values that may not be safe. Using the correct variable name fixes this. (The file contain only ASCII characters at the moment, so the line could alternatively be removed completely.) Change-Id: Ie6b4f41043bf97bb59e3de403e4794c302d81783
* | Remove extra double quote from docstringMartin Geisler2014-05-261-1/+1
| | | | | | | | | | | | | | | | The extra " was visible on http://docs.openstack.org/developer/python-swiftclient/swiftclient.html Change-Id: I7d61c8259a4f13464c11ae7e3fa28eb3a58e4baa
* | Merge "fixed several pep8 issues"Jenkins2014-05-243-39/+39
|\ \
| * | fixed several pep8 issuesChristian Berendt2014-05-213-39/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * E111 indentation is not a multiple of four * E128 continuation line under-indented for visual indent * E265 block comment should start with '# ' * E713 test for membership should be 'not in' Tested with pep8 version 1.5.6 (2014-04-14). Change-Id: I7f85c143d463c501a0df20724362ad5c0f2d4dde
* | | Merge "Remove testtools.main() call from tests"Jenkins2014-05-243-13/+0
|\ \ \
| * | | Remove testtools.main() call from testsChristian Schwede2014-05-203-13/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is no testtools.main() at all (this was a unittest.main() before). Let's remove this unused and non-working code. The following code can be used if someone needs to start tests manually: python -m unittest <test_file.py> Change-Id: Id5162ac73825584df6c23dbe68786ff3355fc6ae
* | | | Fix wrong assertions in unit testsanc2014-05-221-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | A couple of assertTrue in the test_swiftclient.py unit tests should be assertEqual. Also, the expected values now need to be bytes literals. Change-Id: I7cc1bd60d9ba82d1a28fbae2e1243d3c799451bd
* | | | Fix Python3 bugs2.1.0Christian Schwede2014-05-213-9/+30
| |/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixes three issues that were found running functional tests with Python 3: 1. python-requests sets 'application/x-www-form-urlencoded' as content-type if the input is not a string object and no content-type is given. 2. Encoding of the headers is now only done if required. This prevents comparisons between unencoded headers and encoded prefixes and avoids unnecessary forloop-iterations. One unittest was extended to ensure it works for unencoded and encoded headers with or without the prefix. 3. Functional tests recently switched to using byte data for testing, thus the comparison needs to be a byte object as well. Change-Id: I035f8b4b9c9ccdc79820b907770a48f86d0343b4
* | | Move test_shell.py under tests/unit/anc2014-05-201-0/+0
|/ / | | | | | | | | | | | | | | | | Looks like test_shell.py got left behind when the unit tests were relocated under tests/unit. As a consequence they weren't included in tox runs. Change-Id: I93435e6141c5e3ad4a9403626f226d7c760cbd7c
* | Merge "Mark swiftclient as being a universal wheel"Jenkins2014-05-161-0/+3
|\ \
| * | Mark swiftclient as being a universal wheelAlex Gaynor2014-05-161-0/+3
| | | | | | | | | | | | Change-Id: I9580ab0f5bc53b472d0b633858090f9e0708ed90
* | | Merge "change assert_ to assertTrue"Jenkins2014-05-161-1/+1
|\ \ \
View Lorry Controller Status