diff options
author | Jordan Cook <jordan.cook@pioneer.com> | 2021-03-25 09:44:31 -0500 |
---|---|---|
committer | Jordan Cook <jordan.cook@pioneer.com> | 2021-03-28 10:39:40 -0500 |
commit | 2aebfe37e27a4e85c1640a0fe4d1632ba19a6fbf (patch) | |
tree | 65236e4bf5ea564188f9e43c6683354c5f97ac0e | |
parent | 111e03a4a72bc4f20868fa32b63726de84bd6b72 (diff) | |
download | requests-cache-2aebfe37e27a4e85c1640a0fe4d1632ba19a6fbf.tar.gz |
Move 'Security' section to a separate page, link from Readme, and add itsdangerous to default package dependencies
-rw-r--r-- | README.md | 47 | ||||
-rw-r--r-- | docs/advanced_usage.rst | 46 | ||||
-rw-r--r-- | docs/conf.py | 2 | ||||
-rw-r--r-- | docs/index.rst | 1 | ||||
-rw-r--r-- | docs/security.rst | 48 | ||||
-rw-r--r-- | examples/README.md | 2 | ||||
-rw-r--r-- | setup.py | 5 |
7 files changed, 86 insertions, 65 deletions
@@ -1,4 +1,4 @@ -# requests-cache +# Requests-Cache [![Build](https://github.com/reclosedev/requests-cache/actions/workflows/build.yml/badge.svg)](https://github.com/reclosedev/requests-cache/actions/workflows/build.yml) [![Coverage](https://coveralls.io/repos/github/reclosedev/requests-cache/badge.svg?branch=master)](https://coveralls.io/github/reclosedev/requests-cache?branch=master) [![Documentation](https://img.shields.io/readthedocs/requests-cache/latest)](https://requests-cache.readthedocs.io/en/latest/) @@ -23,26 +23,27 @@ Install with pip: pip install requests-cache ``` -Requirements: -Requires python 3.6+. -You may need additional dependencies depending on which backend you want to use. -To install with extra dependencies for all supported backends: -```bash -pip install requests-cache[backends] -``` +**Requirements:** +* Requires python 3.6+. +* You may need additional dependencies depending on which backend you want to use. To install with + extra dependencies for all supported backends: -See [Contributing Guide](https://github.com/reclosedev/requests-cache/blob/master/CONTRIBUTING.md) -for setup info for local development. + ```bash + pip install requests-cache[backends] + ``` -## Full Examples -* You can find a working example at Real Python: -[Caching External API Requests](https://realpython.com/blog/python/caching-external-api-requests) -* There are some additional examples in the [examples/](https://github.com/reclosedev/requests-cache/tree/master/examples) folder +**Optional Setup Steps:** +* See [Security](https://requests-cache.readthedocs.io/en/latest/security.html) for recommended + setup for more secure cache serialization. +* See [Contributing Guide](https://requests-cache.readthedocs.io/en/latest/contributing.html) + for setup info for local development. ## General Usage There are two main ways of using `requests-cache`: -* Using [CachedSession](https://requests-cache.readthedocs.io/en/latest/api.html#requests_cache.core.CachedSession) (recommended) -* Globally patching `requests` using [install_cache](https://requests-cache.readthedocs.io/en/latest/api.html#requests_cache.core.install_cache) +* **Sessions:** Use [requests_cache.CachedSession](https://requests-cache.readthedocs.io/en/latest/api.html#requests_cache.core.CachedSession) + in place of [requests.Session](https://requests.readthedocs.io/en/master/user/advanced/#session-objects) (recommended) +* **Patching:** Globally patch `requests` using + [requests_cache.install_cache](https://requests-cache.readthedocs.io/en/latest/api.html#requests_cache.core.install_cache) ### Sessions `CachedSession` wraps `requests.Session` with caching features, and otherwise behaves the same as a @@ -60,9 +61,6 @@ The URL in this example adds a delay of 1 second, but all 100 requests will comp second. The response will be fetched once, saved to `demo_cache.sqlite`, and subsequent requests will return the cached response near-instantly. -There are many ways to customize caching behavior; see -[Advanced Usage](https://requests-cache.readthedocs.io/en/latest/advanced_usage.html) for details. - ### Patching Patching with `requests_cache.install_cache()` will add caching to all `requests` functions: ```python @@ -126,7 +124,7 @@ You can also set expiration on a per-request basis, which will override any sess session.get('http://httpbin.org/get', expire_after=360) ``` -If a per-session expiration is set but you want to temporarily disable it, use ```-1```: +If a per-session expiration is set but you want to temporarily disable it, use `-1`: ```python # Never expire session.get('http://httpbin.org/get', expire_after=-1) @@ -147,9 +145,16 @@ Or, to revalidate the cache with a new expiration: session.remove_expired_responses(expire_after=360) ``` +## More Features & Examples +* You can find a working example at Real Python: + [Caching External API Requests](https://realpython.com/blog/python/caching-external-api-requests) +* There are some additional examples in the [examples/](https://github.com/reclosedev/requests-cache/tree/master/examples) folder +* See [Advanced Usage](https://requests-cache.readthedocs.io/en/latest/advanced_usage.html) for + details on customizing cache behavior and other features beyond the basics. + ## Related Projects If `requests-cache` isn't quite what you need, you can help make it better! See the -[Contributing Guide](https://github.com/reclosedev/requests-cache/blob/master/CONTRIBUTING.md) +[Contributing Guide](https://requests-cache.readthedocs.io/en/latest/contributing.html) for details. You can also check out these other python cache projects: diff --git a/docs/advanced_usage.rst b/docs/advanced_usage.rst index cf55778..4c0f687 100644 --- a/docs/advanced_usage.rst +++ b/docs/advanced_usage.rst @@ -170,42 +170,6 @@ You can then use your custom backend in a ``CachedSession`` with the ``backend`` >>> session = CachedSession(backend=MyCache()) -Security --------- -The python ``pickle`` module has some -`known security vulnerabilities <https://docs.python.org/3/library/pickle.html>`_, -meaning it should only be used to serialize and deserialize data you trust. Since this isn't always -possible, requests-cache can optionally use `itsdangerous <https://itsdangerous.palletsprojects.com>`_ -to add a layer of security around these operations. - -It works by signing serialized data with a secret key that you control. Then, if the data is tampered -with, the signature check fails and raises an error. To enable this behavior, first install the extra package:: - - $ pip install itsdangerous - -Then create your key and set it in your application. A common pattern for this (or any other -application credentials) is to store it wherever you store the rest of your credentials -(system keyring, password database, etc.) and set it in an environment variable. - -Then, initialize your cache with your key: - - >>> import os - >>> from requests_cache import CachedSession - >>> session = CachedSession(secret_key=os.environ['SECRET_KEY']) - >>> session.get('https://httpbin.org/get') - -You can verify that it's working by modifying the cached item (*without* your key): - - >>> session_2 = CachedSession(secret_key='a different key') - >>> cache_key = list(session_2.cache.responses.keys())[0] - >>> session_2.cache.responses[cache_key] = 'exploit!' - -Then, if you try to get that cached response again (*with* your key), you will get an error: - - >>> session.get('https://httpbin.org/get') - BadSignature: Signature b'iFNmzdUOSw5vqrR9Cb_wfI1EoZ8' does not match - - Usage with other requests features ---------------------------------- @@ -340,10 +304,10 @@ Usage is the same as other libraries that subclass `requests.Session`:: class CachedArchiveSession(CacheMixin, ArchiveSession): """Session with features from both CachedSession and ArchiveSession""" - Potential Issues ---------------- - -.. warning:: Version updates of ``requests``, ``urllib3`` or ``requests_cache`` itself may not be - compatible with previously cached data (see `issue #56 <https://github.com/reclosedev/requests-cache/issues/56>`_). - To prevent this, you can use a virtualenv and pin your dependency versions. +* Version updates of ``requests``, ``urllib3`` or ``requests-cache`` itself may not be compatible with + previously cached data (see issues `#56 <https://github.com/reclosedev/requests-cache/issues/56>`_ + and `#102 <https://github.com/reclosedev/requests-cache/issues/102>`_). + The best way to prevent this is to use a virtualenv and pin your dependency versions. +* See :ref:`security` for notes on serialization security diff --git a/docs/conf.py b/docs/conf.py index f2c2ae8..930c430 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -5,7 +5,7 @@ import sys # Add project path sys.path.insert(0, os.path.abspath('..')) -from requests_cache import __version__ +from requests_cache import __version__ # noqa: E402 # General information about the project. project = 'requests-cache' diff --git a/docs/index.rst b/docs/index.rst index 78d1d14..6eec557 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -13,6 +13,7 @@ Requests-cache documentation :maxdepth: 2 advanced_usage + security api contributing history diff --git a/docs/security.rst b/docs/security.rst new file mode 100644 index 0000000..24aa203 --- /dev/null +++ b/docs/security.rst @@ -0,0 +1,48 @@ +Security +======== +.. warning:: The python ``pickle`` module has `known security vulnerabilities <https://docs.python.org/3/library/pickle.html>`_, + potentially leading to code execution when deserialzing data. + +This means it should only be used to deserialize data that you trust hasn't been tampered with. +Since this isn't always possible, requests-cache can optionally use +`itsdangerous <https://itsdangerous.palletsprojects.com>`_ to add a layer of security around these operations. +It works by signing serialized data with a secret key that you control. Then, if the data is tampered +with, the signature check fails and raises an error. + +Creating and Storing a Secret Key +--------------------------------- +To enable this behavior, first create a secret key, which can be any ``str`` or ``bytes`` object. + +One common pattern for handling this is to store it wherever you store the rest of your credentials +(`Linux keyring <https://itsfoss.com/ubuntu-keyring>`_, +`macOS keychain <https://support.apple.com/guide/mac-help/use-keychains-to-store-passwords-mchlf375f392/mac>`_, +`password database <https://keepassxc.org>`_, etc.), +set it in an environment variable, and then read it in your application: + + >>> import os + >>> secret_key = os.environ['SECRET_KEY'] + +Alternatively, you can use the `keyring <https://keyring.readthedocs.io>`_ package to read the key +directly: + + >>> import keyring + >>> secret_key = keyring.get_password('requests-cache-example', 'secret_key') + +Signing Cached Responses +------------------------ +Once you have your key, just pass it to :py:class:`.CachedSession` or :py:func:`.install_cache` to start using it: + + >>> from requests_cache import CachedSession + >>> session = CachedSession(secret_key=secret_key) + >>> session.get('https://httpbin.org/get') + +You can verify that it's working by modifying the cached item (*without* your key): + + >>> session_2 = CachedSession(secret_key='a different key') + >>> cache_key = list(session_2.cache.responses.keys())[0] + >>> session_2.cache.responses[cache_key] = 'exploit!' + +Then, if you try to get that cached response again (*with* your key), you will get an error: + + >>> session.get('https://httpbin.org/get') + BadSignature: Signature b'iFNmzdUOSw5vqrR9Cb_wfI1EoZ8' does not match diff --git a/examples/README.md b/examples/README.md new file mode 100644 index 0000000..6645a4e --- /dev/null +++ b/examples/README.md @@ -0,0 +1,2 @@ +# Requests-Cache Examples +This folder contains some complete examples for using the main features of requests-cache. @@ -8,7 +8,7 @@ extras_require = { # Packages used for CI jobs 'build': ['coveralls', 'twine', 'wheel'], # Packages for all supported backends + features - 'backends': ['boto3', 'pymongo', 'redis', 'itsdangerous'], + 'backends': ['boto3', 'pymongo', 'redis'], # Packages used for documentation builds 'docs': [ 'm2r2', @@ -32,6 +32,7 @@ extras_require = { # All development/testing packages combined extras_require['dev'] = list(chain.from_iterable(extras_require.values())) + setup( name='requests-cache', packages=find_packages(), @@ -39,7 +40,7 @@ setup( author='Roman Haritonov', author_email='reclosedev@gmail.com', url='https://github.com/reclosedev/requests-cache', - install_requires=['requests>=2.0.0'], + install_requires=['requests>=2.0.0', 'itsdangerous'], extras_require=extras_require, include_package_data=True, ) |