summaryrefslogtreecommitdiff
path: root/glanceclient/common/http.py
Commit message (Collapse)AuthorAgeFilesLines
* Cleanup session objecttrain-em2.17.1stable/trainAlex Schultz2020-03-171-0/+9
| | | | | | | | | | | | | If a session object is not provided to the get_http_client function (as is done via osc), the glance client uses it's own HTTPClient class. This class creates a session but does not properly close it when it is done. This can lead to resource warnings about unclosed sockets. This change adds a __del__() to the HTTPClient class to close out the session correctly. Change-Id: Idccff338fa84c46ca0e429bb533a2a5217205eef Closes-Bug: #1838694 (cherry picked from commit 436f797e8db2fc11b7dace7cb7020e4d44a3d2d7)
* HTTPClient: actually set a timeout for requestsCyril Roelandt2019-04-091-0/+1
| | | | | | | The 'timeout' attribute was previously left unused. Change-Id: If615c390302425fe5a646b4651ec6f56aa08fd22 Closes-Bug: #1822052
* Don't quote colon in HTTP headersimacdonn2018-11-011-1/+5
| | | | | | | | | | | | | | | | Since the introduction of quoting of header content in https://review.openstack.org/568698, the 'x-image-meta-location' header has been broken, because urllib.quote() is really intended to be applied to only the path section of a URL, but in this case, it gets applied to the entire URL, and catches the colon that separates the scheme from the remainder of the URL. This change adds the colon to the list of characters that should not get quoted. Since a colon can be directly represented in ASCII, this should not invalidate the previous change. Change-Id: I76a1c9a361b6c9f6eb95ae766b8c3bcf2267703a Closes-Bug: #1788942
* Skip quote '=' for token headerwangxiyuan2018-07-261-1/+1
| | | | | | | | | | If the token is encoded by base64, it may contain '=' char as well. We should skip quoting it. Change-Id: I1ca63d251fa366f0e8e58128d45b729a2489b65c Partial-Bug: #1783290
* Do not quote '+' for token headerwangxiyuan2018-07-251-10/+15
| | | | | | | | | | | | | | | The token in request header may contain url char, such as '+', if quote it, '+' will change to '%2B' which will lead to 401 error. Our CI doesn't notice this bug because Keystone use fernet token which doesn't contain url char by default. But token format in keystone is plugable, some out-tree token formats may contain url char (for example, PKI/PKIZ token). So we should skip quote token to avoiding information changing. Closes-bug: #1783290 Change-Id: I5aa71b3e2b9b19581e46ccf8a80eda5d637f17d1
* Handle HTTP headers per RFC 8187Brian Rosmaita2018-05-171-2/+21
| | | | | | | | | | | | | | | | | According to RFC 8187, HTTP headers should use 7-bit ASCII encoding. The glanceclient was encoding them as UTF-8, which can leave the 8th bit nonzero when representing unicode, and which presents problems for any recipient following the standard and decoding the headers as ASCII. This change requires keystoneauth1 3.6.2, which has a fix for a bug that made it unable to handle bytes in headers. The dependency is a patch bumping the keystoneauth1 version in upper-constraints. Depends-on: https://review.openstack.org/#/c/569138/ Change-Id: I0d14974126fcb20e23a37347f4f1756c323cf2f5 Closes-bug: #1766235
* Removed the --no-ssl-compression parameter which is deprecatedPranaliD2017-07-251-8/+0
| | | | | | | | --no-ssl-compression is deprecated and no longer used. So, it is removed from the help message. Change-Id: I2b886671a568ed191ee380cf16335ccd9ae85062 Closes-Bug: #1583919
* add explicit dependency on pyopensslDoug Hellmann2017-06-281-0/+5
| | | | | | | | | | | The library is used in glanceclient/common/https.py and the documentation build for the API fails without the dependency. Update the error handling so that when OpenSSL reports an error it is converted to a client communication error. Change-Id: I0c0fb3139bb848d0cbaf88ae6a767a730bea74eb Signed-off-by: Doug Hellmann <doug@doughellmann.com>
* Merge "Replace six.iteritems() with .items()"Jenkins2017-06-281-3/+3
|\
| * Replace six.iteritems() with .items()ji-xuepeng2017-02-081-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | 1.As mentioned in [1], we should avoid usingg six.iteritems to achieve iterators. We can use dict.items instead, as it will return iterators in PY3 as well. And dict.items/keys will more readable. 2.In py2, the performance about list should be negligible, see the link [2]. [1] https://wiki.openstack.org/wiki/Python3 [2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html Change-Id: I71c13040318eca6e5ed993e8aa03f8003986a71c
* | Allow global_request_id in Client constructorSean Dague2017-06-011-0/+9
| | | | | | | | | | | | | | | | | | | | This allows us to pass in a global_request_id in the client constructor so that subsequent calls with this client pass that to the servers. This enables cross project request_id tracking. oslo spec I65de8261746b25d45e105394f4eeb95b9cb3bd42 Change-Id: Iea1e754a263a01dae5ed598fdda134394aff54b0
* | v2: Content-Type: application/octet-stream header always addedckonstanski2017-05-191-7/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The bug: any existing Content-Type header cannot be found because the call to headers.get() fails. Therefore we end up with two Content-Type headers because a new one (applicaion/octet-stream) gets added unconditionally. The cause: the strings (keys and values) in the headers dict are converted from unicode sequences of type <str> to utf-8 sequences of type <bytes>. This happens in safe_encode() (oslo_utils/encodeutils.py:66). <str> != <bytes> even if they appear to have the same characters. Hence, for python 3.x, _set_common_request_kwargs() adds content-type to header even if custom content-type is set in the request. This results in unsupported media type exception when glance client is used with keystoneauth and python 3.x The fix: follow the directions in encode_headers(). It says to do this just before sending the request. Honor this principle; do not encode headers and then perform more business logic on them. Change-Id: Idf6079b32f70bc171f5016467048e917d42f296d Closes-bug: #1641239 Co-Authored-By: Pushkar Umaranikar <pushkar.umaranikar@intel.com>
* | Convert IOError from requestsEric Fried2017-05-191-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This requests commit [1] changed the behavior when a nonexistent cacert file is passed in: now it raises IOError. This is getting through glanceclient.common.http.HTTPClient._request, which used to raise CommunicationError in this scenario. Even though there is arguably a better exception than CommunicationError to represent this condition (like maybe IOError), for backward compatibility this change set converts IOError to CommunicationError. We also improve the unit test to raise the original exception if the expected conditions aren't met; this improves debugability. [1] https://github.com/kennethreitz/requests/commit/7d8b87c37f3a5fb993fd83eda6888ac217cd108e Change-Id: I6a2cf4c6d041b67d3509153b4cef18b459263648 Closes-Bug: #1692085
* | Merge "Replace functions 'dict.get' and 'del' with 'dict.pop'"Jenkins2017-03-061-3/+2
|\ \
| * | Replace functions 'dict.get' and 'del' with 'dict.pop'bhagyashris2017-01-311-3/+2
| |/ | | | | | | | | | | | | | | | | | | Refactoring code: Making dict to use single instruction: pop() rather than two instructions: get() and del, giving the codes a format that carries through. TrivialRefactoring Change-Id: Idb21df37c287fdff24c29153676f82544f735297
* | x-openstack-request-id logged twice in logsAbhishek Kekane2017-02-221-10/+10
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | In the recent release of keystoneauth1 2.18.0 provision is made to log x-openstack-request-id for session client. Once this new library is synced in openstack projects, the x-openstack-request-id will be logged twice on the console if session client is used. For example, $ glance --debug image-list DEBUG:keystoneauth.session:GET call to image for http://10.232.48.204:9292/v2/schemas/image used request id req-96a3f203-c605-4c96-a31d-f1199d41705c DEBUG:glanceclient.common.http:GET call to glance-api for http://10.232.48.204:9292/v2/schemas/image used request id req-96a3f203-c605-4c96-a31d-f1199d41705c Above log will be logged twice on the console. Removed logging of x-openstack-request-id in case of SessionClient as it is already logged in keystoneauth1. x-openstack-request-id will only be logged once on console if HTTPClient is used. Depends-On: I492b331ff3da8d0b91178bf0d5fe1d3702f15bd7 Closes-Bug: #1657351 Change-Id: I64258f997dc060113f53682aee74bdd40a346e54
* Merge "Revert "Add last_request_id member to HTTPClient and SessionClient""Jenkins2016-12-141-4/+0
|\
| * Revert "Add last_request_id member to HTTPClient and SessionClient"Abhishek Kekane2016-05-231-4/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit 9e532db8b0f0ba537edef143a6f5380a2aaa1e4b. If glanceclient is used in multi-threaded environment, then there is a possibility of getting invalid/wrong last request-id. To avoid this, need to use thread local storage to store last-request-id and add public method to return this request-id to caller. http://specs.openstack.org/openstack/openstack-specs/specs/return-request-id.html#alternatives Change-Id: I08d8d87fc0cc291f1b930b2c0cfc110ec8394131
* | switch from keystoneclient to keystoneauthItisha Dewan2016-09-031-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | move glanceclient to keystoneauth as keystoneclient's auth session, plugins and adapter code has been deprecated. refer to [1] for more information. 1: https://github.com/openstack/python-keystoneclient/commit/1a84e24fa4ce6d3169b59e385f35b2a63f2257f0 implements bp: use-keystoneauth Co-Authored-By: Itisha <ishadewan07@gmail.com> Change-Id: I88fb327628e1bec48dc391f50d66b3deab4a8ab9
* | Merge "Fix string interpolation to delayed to be handled by the logging code"Jenkins2016-07-301-1/+1
|\ \
| * | Fix string interpolation to delayed to be handled by the logging codehaobing12016-07-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | String interpolation should be delayed to be handled by the logging code, rather than being done at the point of the logging call. See the oslo i18n guideline. * http://docs.openstack.org/developer/oslo.i18n/guidelines.html Change-Id: If06663076e4081c6268ba88c157513723b734b31 Closes-Bug: #1596829
* | | Merge "Log request-id before exceptions raised"Jenkins2016-07-301-10/+10
|\ \ \
| * | | Log request-id before exceptions raisedAbhishek Kekane2016-07-181-10/+10
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As of now request-id is not logged if an excpetion is raised. Rearranged code so that request-id is logged even in case of an exception. Change-Id: Iee0398404ee752c0d880edf3054207c35862e71a Closes-Bug: #1603863
* | | py3: Fix encoding and use sys.stdin.bufferSirushti Murugesan2016-07-251-1/+1
|/ / | | | | | | | | | | | | | | | | * exc.py: Encode body in response before calling replace over it. * http.py: prepend the bytes literal to the empty string or else we hit bug 1342080 again in python 3. * utils.py: Use sys.stdin.buffer in python 3. Change-Id: Ieefb8c633658e507486438e5518c5d53e819027d
* | Log request-id for each api callAbhishek Kekane2016-06-271-0/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Added support to log 'X-Openstack-Request-Id' for each api call. If glanceclient is used from command line then following log will be logged on console if --debug flag is used. DEBUG:glanceclient.common.http:GET call to glance-api for http://172.26.88.20:9292/v2/schemas/image used request id req-e0c7c97a-8fc0-4ce3-a669-d0b1eb5d7aae If python-glanceclient is used in applications (e.g. Nova) then following log message will be logged in service logs. DEBUG glanceclient.common.http [req-be074f1e-1c17-4786-b703-2a221751c8f4 demo demo] GET call to glance-api for http://172.26.88.20:9292/v1/images/detail?is_public=none&limit=20 used request id req-9b1dd929-df30-46b2-a8f2-dfd6ffbad3fc DocImpact: To use this feature user need to set 'default_log_levels' in third party application. For example nova uses glance then in nova.conf 'default_log_levels' should be set as below: default_log_levels = glanceclient=DEBUG Implements: blueprint log-request-id Change-Id: Ib04a07bac41ad2a5e997348f3b0bccc640169dc9
* | Merge "Fix "Codec can't encode characters""Jenkins2016-05-231-16/+16
|\ \
| * | Fix "Codec can't encode characters"Darja Shakhray2016-05-191-16/+16
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Headers were encoded in HTTPClient, but when glance client started to use SessionClient this functionality was lost. This commit replaces static method "encode_headers" from HTTPClient and makes it a common function, that SessionClient can use when converting image meta to headers. Change-Id: If9f8020220d2a0431b4241b38b9c83c09c0d75cb Closes-bug: #1574587
* | | Merge "Add last_request_id member to HTTPClient and SessionClient"Jenkins2016-05-181-0/+4
|\ \ \ | |/ / |/| / | |/
| * Add last_request_id member to HTTPClient and SessionClientCao ShuFeng2016-03-301-0/+4
| | | | | | | | | | | | | | | | | | apiclient.base.Resource.get method requires manager.client to have last_request_id member. Otherwise get operation fails with AttributeError exception. Change-Id: I0ece85e3f61f2a7f176520ddf3ebee7792e51993 Closes-bug: 1552533
* | Update auth_token before sending requestkairat_kushaev2016-03-301-4/+4
|/ | | | | | | | | | | | | | Previously auth_token was initialized once in __init__ method. After that we stored token in session headers. So to refresh token users need to instantiate a new session inside http client or re-create client itself. In order to provide possibility to refresh token we need to add token header before sending the request. So users can just update auth_token attribute in the HTTPClient to refresh user token. Change-Id: Ifebe9011870bbddc46fc6d6a26563641d5559e97 Closes-Bug: #1563495
* Remove code needed for python2.5kairat_kushaev2016-02-011-6/+0
| | | | | | | | glance has a code specific for python2.5. We need to delete this code cause glanceclient doesn't support neither python2.5 or python2.6. Change-Id: I17e4905b6e02fcfff033a6cde03324e2a47bfce2
* Remove broken try/except workaround for old requestsMonty Taylor2015-12-151-5/+1
| | | | | | | | | Not only is this code broken on the requests we require on distro-provided requests, it's not needed anymore. Remove it. Closes-bug: 1526254 Change-Id: I47a07bf9910f118392785fc20e015f036a2e8a7c
* Ensure that identity token in header is not an unicode stringVincent Untz2015-11-171-1/+2
| | | | | | | | | | | | | We need all the headers to be safe strings so they can be joined together and not become an unicode string in doing so. This fixes a bug when creating an image with non-ascii characters in the name. This is required for python 2.6 compatibility. Change-Id: I66ebc27edf4ccd8f903399da58705711c372536d Closes-Bug: 1448080
* Merge "Add support for setting Accept-Language header"Jenkins2015-10-141-0/+4
|\
| * Add support for setting Accept-Language headerFrode Nordahl2015-10-141-0/+4
| | | | | | | | | | | | DocImpact Closes-Bug: 1480529 Change-Id: I35a37d55edb700a5993bd5cc352335a87a15e47a
* | Check if v2 is available and fallbackFlavio Percoco2015-09-041-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We have a basic implementation for a fallback mechanism that will use v1 rather than v2 when downloading schema files from glance-api fails. However, this is not sound. If the schemas are cached already, we won't check if v2 is available and fail to fallback. This patch fixes the aforementioned issue by getting the list of available versions from the server only when the API versions was not explicitly specified through the CLI. That is, for all commands that don't pass `--os-image-api-version 2`, we'll check v2's availability and we'll fallback to v1 if it isn't available. This patch also changes how we handle `/versions` calls in the client. The server has been, incorrectly, replying to requests to `/version` with a 300 error, which ended up in the client re-raising such exception. While I think 300 shouldn't raise an exception, I think we should handle that in a spearate patch. Therefore, this patch just avoids raising such exception when `/version` is explicitly called. This fallback behaviour and the check on `/versions` will be removed in future versions of the client. The later depends on this bug[0] being fixed. [0] https://bugs.launchpad.net/glance/+bug/1491350 Closes-bug: #1489381 Change-Id: Ibeba6bc86db2a97b8a2b4bd042248464cd792e5e
* | Remove custom SSL compression handlingStuart McLaren2015-08-261-13/+10
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Custom SSL handling was introduced because disabling SSL layer compression provided an approximately five fold performance increase in some cases. Without SSL layer compression disabled the image transfer would be CPU bound -- with the CPU performing the DEFLATE algorithm. This would typically limit image transfers to < 20 MB/s. When --no-ssl-compression was specified the client would not negotiate any compression algorithm during the SSL handshake with the server which would remove the CPU bottleneck and transfers could approach wire speed. In order to support '--no-ssl-compression' two totally separate code paths exist depending on whether this is True or False. When SSL compression is disabled, rather than using the standard 'requests' library, we enter some custom code based on pyopenssl and httplib in order to disable compression. This patch/spec proposes removing the custom code because: * It is a burden to maintain Eg adding new code such as keystone session support is more complicated * It can introduce additional failure modes We have seen some bugs related to the 'custom' certificate checking * Newer Operating Systems disable SSL for us. Eg. While Debian 7 defaulted to compression 'on', Debian 8 has compression 'off'. This makes both servers and client less likely to have compression enabled. * Newer combinations of 'requests' and 'python' do this for us Requests disables compression when backed by a version of python which supports it (>= 2.7.9). This makes clients more likely to disable compression out-of-the-box. * It is (in principle) possible to do this on older versions too If pyopenssl, ndg-httpsclient and pyasn1 are installed on older operating system/python combinations, the requests library should disable SSL compression on the client side. * Systems that have SSL compression enabled may be vulnerable to the CRIME (https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-4929) attack. Installations which are security conscious should be running the Glance server with SSL disabled. Full Spec: https://review.openstack.org/#/c/187674 Blueprint: remove-custom-client-ssl-handling Change-Id: I7e7761fc91b0d6da03939374eeedd809534f6edf
* Enable flake8 checksDarja Shakhray2015-07-211-0/+1
| | | | | | | | | | | | This commit enables new flake8 checks: * E265 block comment should start with '# ' * H405 multi line docstring summary not separated with an empty line * E123 closing bracket does not match indentation of opening bracket's line * H238 old style class declaration, use new style (inherit from `object`) * E128 continuation line under-indented for visual indent and makes related changes in the code. Change-Id: Ie993afc930f6b74d7a990bcaa9fc0e9f5ba1585c
* Fix exception message in Http.pywangxiyuan2015-07-141-2/+4
| | | | | | | In common.http.py, the exception RequestTimeout has wrong message.This patch fixed it. Change-Id: Ie8ff188b9c82ce424cb8177278f36e4d1275b306
* Import only modules and update tox.iniCindy Pallares2015-06-171-4/+4
| | | | | | | | | As stated in the OpenStack Hacking Guidelines, it is prefered that only modules should be imported. Also updated tox.ini to ignore opestack/common among others. Change-Id: I2f0a603c31052eadee581c11880c0ec6bd392829
* Make glanceclient accept a session objectJamie Lennox2015-06-111-66/+125
| | | | | | | | | | | To make this work we create a different HTTPClient that extends the basic keystoneclient Adapter. The Adapter is a standard set of parameters that all clients should know how to use like region_name and user_agent. We extend this with the glance specific response manipulation like loading and sending iterables. Implements: bp session-objects Change-Id: Ie8eb4bbf7d1a037099a6d4b272cab70525fbfc85
* Merge "Change oslo.utils to oslo_utils"Jenkins2015-02-061-3/+3
|\
| * Change oslo.utils to oslo_utilsLouis Taylor2015-02-051-3/+3
| | | | | | | | | | | | | | | | | | | | The oslo.utils libraries are moving away from namespace packages. This requires oslo.utils>=1.2.0 bp drop-namespace-packages Change-Id: I803df61e91eabb96329d859aef6bea03530fb84f
* | Merge "Close streamed requests explicitly"Jenkins2015-02-061-1/+12
|\ \ | |/ |/|
| * Close streamed requests explicitlyIan Cordasco2015-01-151-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | If we don't explicitly close a response after streaming its download, then we can run into HTTPConnectionPool full warnings. It also will hurt performance if we have to continuously create new sockets for new responses. Calling close will return the connection to the pool so it can be reused. Note this is only necessary when streaming a response. If we don't stream it, then requests will return the connection to the pool for us. Change-Id: I803bd4dd0e769c233501d5e5ff07a19705fbe233 Closes-bug: 1341777
* | Ignore NoneType when encoding headersIan Cordasco2015-02-031-1/+1
| | | | | | | | | | | | | | | | | | Some generated header values may in fact be None. Trying to encode None causes the client to fail with an exception and cannot be worked around by the user. Change-Id: I638b1fba0ef9a07d726445d8c2cdd774140f5b83 Closes-bug: 1415935
* | Remove openstack.common.importutilsLouis Taylor2015-01-281-1/+1
| | | | | | | | | | | | This module now lives in oslo.utils, so import it from there. Change-Id: I41fa4897fc820596fb010336044ff4c493017d5a
* | Remove openstack.common.strutilsLouis Taylor2015-01-271-5/+6
| | | | | | | | | | | | | | This module now lives in oslo.utils, so import it from there instead. Co-Authored-By: Ian Cordasco <ian.cordasco@rackspace.com> Change-Id: Ib35dc840992433542490670781badd9529ec8947
* | Handle HTTP byte returns in python 3Jamie Lennox2015-01-141-2/+2
|/ | | | | | | | | | The returns from requests' response.content is a bytes type. Under python 3 this fails in error handling and string conversion. The response.text variable should be used to treat a response body as a string. Closes-Bug: #1407531 Change-Id: Ifd588b5f6820ef21beb186d88d0b3f1a267695aa
* Merge "Curl statements to include globoff for IPv6 URLs"Jenkins2015-01-081-1/+1
|\
View Lorry Controller Status