summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorJordan Cook <jordan.cook@pioneer.com>2021-03-20 15:19:58 -0500
committerJordan Cook <jordan.cook@pioneer.com>2021-03-24 16:46:27 -0500
commitab18e06217a8a1902d1859aae6b033f469b2232b (patch)
tree98136dffa4d16ed3d645e2bdee1ac60421001ff7
parent78819897213ec3bff57d4c7094fd585dbb8d48d4 (diff)
downloadrequests-cache-ab18e06217a8a1902d1859aae6b033f469b2232b.tar.gz
More usage examples, formatting, and editing for Readme + Sphinx docs
* Closes #135, #165 * Add a 'Summary' section at the top of the Readme explaining the scope of requests-cache and why you would want to use it * Add some more info explaining the difference between using `CachedSession` directly vs. patching with `install_cache()` * Move basic examples from 'User Guide' section into Readme * Include Readme in Sphinx docs (using `.. mdinclude::`) and remove duplicate sections * Include Contributing guide in Sphinx docs * Convert History doc to markdown and include in Sphinx docs * Use `automod` options to move main cache documentation from `CacheMixin` back to to `CachedSession`, since that's probably where a user will look first * Add more detailed usage examples to an 'Advanced Usage' section for `CachedSession` options (`filter_fn`, `ignore_parameters`, etc.) * Update example scripts and move them to `examples/` folder
-rw-r--r--CONTRIBUTING.md4
-rw-r--r--HISTORY.md123
-rw-r--r--HISTORY.rst184
-rw-r--r--README.md143
-rw-r--r--docs/advanced_usage.rst349
-rw-r--r--docs/api.rst45
-rw-r--r--docs/conf.py4
-rw-r--r--docs/contributing.rst1
-rw-r--r--docs/history.rst1
-rw-r--r--docs/index.rst29
-rw-r--r--docs/user_guide.rst229
-rw-r--r--example.py30
-rw-r--r--example_request.py32
-rw-r--r--examples/basic_usage.py31
-rw-r--r--examples/expiration.py42
-rw-r--r--examples/session_patch.py36
-rw-r--r--pyproject.toml2
-rw-r--r--setup.cfg6
18 files changed, 738 insertions, 553 deletions
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 9a217e9..64d0cd2 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -15,7 +15,7 @@ If you have an issue or PR that hasn't received a response in a timely manner, o
discuss ideas about the project in general, please reach out on the Code Shelter chat server, under
[projects/requests-cache](https://codeshelter.zulipchat.com/#narrow/stream/186993-projects/topic/requests-cache).
-## Installation
+## Dev Installation
To set up for local development:
```bash
@@ -24,7 +24,7 @@ $ cd requests-cache
$ pip install -Ue ".[dev]"
```
-## Pre-commit hooks
+## Pre-commit Hooks
[Pre-commit](https://github.com/pre-commit/pre-commit) config is included to run the same checks
locally that are run in CI jobs by GitHub Actions. This is optional but recommended.
```bash
diff --git a/HISTORY.md b/HISTORY.md
new file mode 100644
index 0000000..ce3e6dd
--- /dev/null
+++ b/HISTORY.md
@@ -0,0 +1,123 @@
+# History
+
+## 0.6.0 (TBD)
+[See all included issues and PRs](https://github.com/reclosedev/requests-cache/milestone/1?closed=1)
+
+* Drop support for python <= 3.5
+* Add support for setting expiration for individual requests
+* Add support for setting expiration based on URL glob patterns
+* Add support for overriding original expiration (i.e., revalidating) in `CachedSession.remove_expired_responses()`
+* Add `CacheMixin` class to be make the features of `CachedSession` usable as a mixin class,
+ for compatibility with other `requests`-based libraries
+* Add `CachedResponse` class to wrapped cached `requests.Response` objects, make additional cache
+ information available to client code
+* Add property `BaseCache.urls` to get all URLs persisted in the cache
+* Update usage of deprecated MongoClient `save()` method
+* Fix TypeError with `DbPickleDict` initialization
+* Fix usage of `CachedSession.cache_disabled` if used within another contextmanager
+* Fix non-thread-safe iteration in `BaseCache`
+* Fix `get_cache()`, `clear()`, and `remove_expired_responses()` so they will do nothing if
+ requests-cache is not installed
+* Add `HEAD` to default `allowable_methods`
+
+## 0.5.2 (2019-08-14)
+* Fix DeprecationWarning from collections #140
+
+## 0.5.1 (2019-08-13)
+* Remove Python 2.6 Testing from travis #133
+* Fix DeprecationWarning from collections #131
+* vacuum the sqlite database after clearing a table #134
+* Fix handling of unpickle errors #128
+
+## 0.5.0 (2019-04-18)
+Project is now added to [Code Shelter](https://www.codeshelter.co)
+
+* Add gridfs support, thanks to @chengguangnan
+* Add dynamodb support, thanks to @ar90n
+* Add response filter #104, thanks to @christopher-dG
+* Fix bulk_commit #78
+* Fix remove_expired_responses missed in __init__.py #93
+* Fix deprecation warnings #122, thanks to mbarkhau
+
+## 0.4.13 (2016-12-23)
+* Support PyMongo3, thanks to @craigls #72
+* Fix streaming releate issue #68
+
+## 0.4.12 (2016-03-19)
+* Fix ability to pass backend instance in `install_cache` #61
+
+
+## 0.4.11 (2016-03-07)
+* `ignore_parameters` feature, thanks to @themiurgo and @YetAnotherNerd (#52, #55)
+* More informative message for missing backend dependencies, thanks to @Garrett-R (#60)
+
+## 0.4.10 (2015-04-28)
+* Better transactional handling in sqlite #50, thanks to @rgant
+* Compatibility with streaming in requests >= 2.6.x
+
+## 0.4.9 (2015-01-17)
+* `expire_after` now also accepts `timedelta`, thanks to @femtotrader
+* Added Ability to include headers to cache key (`include_get_headers` option)
+* Added string representation for `CachedSession`
+
+## 0.4.8 (2014-12-13)
+* Fix bug in reading cached streaming response
+
+## 0.4.7 (2014-12-06)
+* Fix compatibility with Requests > 2.4.1 (json arg, response history)
+
+## 0.4.6 (2014-10-13)
+* Monkey patch now uses class instead lambda (compatibility with rauth)
+* Normalize (sort) parameters passed as builtin dict
+
+## 0.4.5 (2014-08-22)
+* Requests==2.3.0 compatibility, thanks to @gwillem
+
+## 0.4.4 (2013-10-31)
+* Check for backend availability in install_cache(), not at the first request
+* Default storage fallbacks to memory if `sqlite` is not available
+
+## 0.4.3 (2013-09-12)
+* Fix `response.from_cache` not set in hooks
+
+## 0.4.2 (2013-08-25)
+* Fix `UnpickleableError` for gzip responses
+
+
+## 0.4.1 (2013-08-19)
+* `requests_cache.enabled()` context manager
+* Compatibility with Requests 1.2.3 cookies handling
+
+## 0.4.0 (2013-04-25)
+* Redis backend. Thanks to @michaelbeaumont
+* Fix for changes in Requests 1.2.0 hooks dispatching
+
+
+## 0.3.0 (2013-02-24)
+* Support for `Requests` 1.x.x
+* `CachedSession`
+* Many backward incompatible changes
+
+## 0.2.1 (2013-01-13)
+* Fix broken PyPi package
+
+## 0.2.0 (2013-01-12)
+* Last backward compatible version for `Requests` 0.14.2
+
+## 0.1.3 (2012-05-04)
+* Thread safety for default `sqlite` backend
+* Take into account the POST parameters when cache is configured
+ with 'POST' in `allowable_methods`
+
+## 0.1.2 (2012-05-02)
+* Reduce number of `sqlite` database write operations
+* `fast_save` option for `sqlite` backend
+
+## 0.1.1 (2012-04-11)
+* Fix: restore responses from response.history
+* Internal refactoring (`MemoryCache` -> `BaseCache`, `reduce_response`
+ and `restore_response` moved to `BaseCache`)
+* `connection` option for `MongoCache`
+
+## 0.1.0 (2012-04-10)
+* initial PyPI release
diff --git a/HISTORY.rst b/HISTORY.rst
deleted file mode 100644
index 6be510b..0000000
--- a/HISTORY.rst
+++ /dev/null
@@ -1,184 +0,0 @@
-.. :changelog:
-
-History
--------
-
-0.6.0 (TBD)
-++++++++++++++++++
-`See all included issues and PRs <https://github.com/reclosedev/requests-cache/milestone/1?closed=1>`_
-
-* Drop support for python <= 3.5
-* Fix TypeError with ``DbPickleDict`` initialization
-* Fix usage of `CachedSession.cache_disabled` if used within another contextmanager
-* Update usage of deprecated MongoClient ``save()`` method
-* Fix non-thread-safe iteration in ``BaseCache``
-* Add support for setting ``expire_after`` per request
-* Refactor CachedSession to be usable as a mixin class, for compatibility with other ``requests.Session`` extensions
-* Fix get_cache, clear, and remove_expired_responses to do nothing if requests-cache is not installed
-* Add ``HEAD`` to default ``allowable_methods``
-
-
-0.5.2 (2019-08-14)
-++++++++++++++++++
-
-* Fix DeprecationWarning from collections #140
-
-
-0.5.1 (2019-08-13)
-++++++++++++++++++
-
-* Remove Python 2.6 Testing from travis #133
-* Fix DeprecationWarning from collections #131
-* vacuum the sqlite database after clearing a table #134
-* Fix handling of unpickle errors #128
-
-
-0.5.0 (2019-04-18)
-++++++++++++++++++
-Project is now added to https://www.codeshelter.co/
-
-* Add gridfs support, thanks to @chengguangnan
-* Add dynamodb support, thanks to @ar90n
-* Add response filter #104, thanks to @christopher-dG
-* Fix bulk_commit #78
-* Fix remove_expired_responses missed in __init__.py #93
-* Fix deprecation warnings #122, thanks to mbarkhau
-
-
-0.4.13 (2016-12-23)
-+++++++++++++++++++
-* Support PyMongo3, thanks to @craigls #72
-* Fix streaming releate issue #68
-
-
-0.4.12 (2016-03-19)
-+++++++++++++++++++
-* Fix ability to pass backend instance in ``install_cache`` #61
-
-
-0.4.11 (2016-03-07)
-+++++++++++++++++++
-* ``ignore_parameters`` feature, thanks to @themiurgo and @YetAnotherNerd (#52, #55)
-* More informative message for missing backend dependencies, thanks to @Garrett-R (#60)
-
-
-0.4.10 (2015-04-28)
-+++++++++++++++++++
-* Better transactional handling in sqlite #50, thanks to @rgant
-* Compatibility with streaming in requests >= 2.6.x
-
-
-0.4.9 (2015-01-17)
-++++++++++++++++++
-
-* ``expire_after`` now also accepts ``timedelta``, thanks to @femtotrader
-* Added Ability to include headers to cache key (``include_get_headers`` option)
-* Added string representation for ``CachedSession``
-
-
-0.4.8 (2014-12-13)
-++++++++++++++++++
-
-* Fix bug in reading cached streaming response
-
-
-0.4.7 (2014-12-06)
-++++++++++++++++++
-
-* Fix compatibility with Requests > 2.4.1 (json arg, response history)
-
-
-0.4.6 (2014-10-13)
-++++++++++++++++++
-
-* Monkey patch now uses class instead lambda (compatibility with rauth)
-* Normalize (sort) parameters passed as builtin dict
-
-
-0.4.5 (2014-08-22)
-++++++++++++++++++
-
-* Requests==2.3.0 compatibility, thanks to @gwillem
-
-
-0.4.4 (2013-10-31)
-++++++++++++++++++
-
-* Check for backend availability in install_cache(), not at the first request
-* Default storage fallbacks to memory if ``sqlite`` is not available
-
-
-0.4.3 (2013-09-12)
-++++++++++++++++++
-
-* Fix ``response.from_cache`` not set in hooks
-
-
-
-0.4.2 (2013-08-25)
-++++++++++++++++++
-
-* Fix ``UnpickleableError`` for gzip responses
-
-
-
-0.4.1 (2013-08-19)
-++++++++++++++++++
-
-* ``requests_cache.enabled()`` context manager
-* Compatibility with Requests 1.2.3 cookies handling
-
-
-0.4.0 (2013-04-25)
-++++++++++++++++++
-
-* Redis backend. Thanks to @michaelbeaumont
-* Fix for changes in Requests 1.2.0 hooks dispatching
-
-
-0.3.0 (2013-02-24)
-++++++++++++++++++
-
-* Support for ``Requests`` 1.x.x
-* ``CachedSession``
-* Many backward incompatible changes
-
-0.2.1 (2013-01-13)
-++++++++++++++++++
-
-* Fix broken PyPi package
-
-0.2.0 (2013-01-12)
-++++++++++++++++++
-
-* Last backward compatible version for ``Requests`` 0.14.2
-
-
-0.1.3 (2012-05-04)
-++++++++++++++++++
-
-* Thread safety for default ``sqlite`` backend
-* Take into account the POST parameters when cache is configured
- with 'POST' in ``allowable_methods``
-
-
-0.1.2 (2012-05-02)
-++++++++++++++++++
-
-* Reduce number of ``sqlite`` database write operations
-* ``fast_save`` option for ``sqlite`` backend
-
-
-0.1.1 (2012-04-11)
-++++++++++++++++++
-
-* Fix: restore responses from response.history
-* Internal refactoring (``MemoryCache`` -> ``BaseCache``, ``reduce_response``
- and ``restore_response`` moved to ``BaseCache``)
-* ``connection`` option for ``MongoCache``
-
-
-0.1.0 (2012-04-10)
-++++++++++++++++++
-
-* initial PyPI release
diff --git a/README.md b/README.md
index 417eeff..07dfbca 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,4 @@
# requests-cache
-Requests-cache is a transparent persistent cache for the [requests](http://python-requests.org) library (version 2+).
-
[![Build](https://github.com/reclosedev/requests-cache/actions/workflows/build.yml/badge.svg)](https://github.com/reclosedev/requests-cache/actions/workflows/build.yml)
[![Coverage](https://coveralls.io/repos/github/reclosedev/requests-cache/badge.svg?branch=master)](https://coveralls.io/github/reclosedev/requests-cache?branch=master)
[![Documentation](https://img.shields.io/readthedocs/requests-cache/latest)](https://requests-cache.readthedocs.io/en/latest/)
@@ -9,51 +7,145 @@ Requests-cache is a transparent persistent cache for the [requests](http://pytho
[![PyPI - Format](https://img.shields.io/pypi/format/requests-cache?color=blue)](https://pypi.org/project/requests-cache)
[![Code Shelter](https://www.codeshelter.co/static/badges/badge-flat.svg)](https://www.codeshelter.co/)
+## Summary
+**Requests-cache** is a transparent persistent HTTP cache for the python [requests](http://python-requests.org)
+library. It is especially useful for web scraping, consuming REST APIs, slow or rate-limited
+sites, or any other scenario in which you're making lots of requests that are likely to be sent
+more than once.
+
+Several storage backends are included: **SQLite**, **Redis**, **MongoDB**, and **DynamoDB**.
+
+See full project documentation at: https://requests-cache.readthedocs.io
+
## Installation
Install with pip:
-```
+```bash
pip install requests-cache
```
-## Usage example
+Requirements:
+Requires python 3.6+.
+You may need additional dependencies depending on which backend you want to use.
+To install with extra dependencies for all supported backends:
+```bash
+pip install requests-cache[backends]
+```
+
+See [Contributing Guide](https://github.com/reclosedev/requests-cache/blob/master/CONTRIBUTING.md)
+for setup info for local development.
+
+## Full Examples
+* You can find a working example at Real Python:
+[Caching External API Requests](https://realpython.com/blog/python/caching-external-api-requests)
+* There are some additional examples in the [examples/](https://github.com/reclosedev/requests-cache/tree/master/examples) folder
+
+## General Usage
+There are two main ways of using `requests-cache`:
+* Using [CachedSession](https://requests-cache.readthedocs.io/en/latest/api.html#requests_cache.core.CachedSession) (recommended)
+* Globally patching `requests` using [install_cache](https://requests-cache.readthedocs.io/en/latest/api.html#requests_cache.core.install_cache)
+
+### Sessions
+`CachedSession` wraps `requests.Session` with caching features, and otherwise behaves the same as a
+normal session.
+
+Basic example:
```python
-import requests_cache
+from requests_cache import CachedSession
-requests_cache.install_cache('demo_cache')
+session = CachedSession('demo_cache', backend='sqlite')
+for i in range(100):
+ session.get('http://httpbin.org/delay/1')
```
+The URL in this example adds a delay of 1 second, but all 100 requests will complete in just over 1
+second. The response will be fetched once, saved to `demo_cache.sqlite`, and subsequent requests
+will return the cached response near-instantly.
-And all responses with headers and cookies will be transparently cached to `demo_cache.sqlite`.
-For example, following the code will take only 1-2 seconds instead of 10, and will run instantly on next launch:
+There are many ways to customize caching behavior; see
+[Advanced Usage](https://requests-cache.readthedocs.io/en/latest/advanced_usage.html) for details.
+### Patching
+Patching with `requests_cache.install_cache()` will add caching to all `requests` functions:
```python
import requests
+import requests_cache
-for i in range(10):
- requests.get('http://httpbin.org/delay/1')
+requests_cache.install_cache()
+requests.get('http://httpbin.org/get')
+session = requests.Session()
+session.get('http://httpbin.org/get')
```
-It can be useful when you are creating some simple data scraper with constantly
-changing parsing logic or data format, and don't want to redownload pages or
-write complex error handling and persistence.
+`install_cache()` takes all the same parameters as `CachedSession`. It can be temporarily disabled
+with `disabled()`, and completely removed with `uninstall_cache()`:
+```python
+# Neither of these requests will use the cache
+with requests_cache.disabled():
+ requests.get('http://httpbin.org/get')
-For more complex workflows, it is possible to cache different requests with different expiration times, or disable caching for a specific request completely:
+requests_cache.uninstall_cache()
+requests.get('http://httpbin.org/get')
+```
+
+**Limitations:**
+Like any other utility that uses global patching, there are some scenarios where you won't want to
+use this:
+* In a multi-threaded or multiprocess applications
+* In an application that uses other packages that extend or modify `requests.Session`
+* In a package that will be used by other packages or applications
-For more complex workflows, it is possible to cache different requests with different expiry times, or disable caching for a specific request completely:
+### Cache Backends
+Several cache backends are included, which can be selected with the `backend` parameter to
+`CachedSession` or `install_cache()`:
+* `'memory'` : Not persistent, just stores responses with an in-memory dict
+* `'sqlite'` : [SQLite](https://www.sqlite.org) database (**default**)
+* `'redis'` : [Redis](https://redis.io/) cache (requires `redis`)
+* `'mongodb'` : [MongoDB](https://www.mongodb.com/) database (requires `pymongo`)
+* `'dynamodb'` : [Amazon DynamoDB](https://aws.amazon.com/dynamodb/) database (requires `boto3`)
+
+### Cache Expiration
+By default, cached responses will be stored indefinitely. There are a number of ways you can handle
+cache expiration. The simplest is using the `expire_after` param with a value in seconds:
```python
-import time
-import requests
-import requests_cache
+# Expire after 30 seconds
+session = CachedSession(expire_after=30)
+```
-requests_cache.install_cache('demo_cache')
+Or a `timedelta`:
+```python
+from datetime import timedelta
-# Hits the URL only 2 times
-for i in range(10):
- requests.get('http://httpbin.org/delay/1', expire_after=1)
- time.sleep(0.2)
+# Expire after 30 days
+session = CachedSession(expire_after=timedelta(days=30))
```
+You can also set expiration on a per-request basis, which will override any session settings:
+```python
+# Expire after 6 minutes
+session.get('http://httpbin.org/get', expire_after=360)
+```
+
+If a per-session expiration is set but you want to temporarily disable it, use ```-1```:
+```python
+# Never expire
+session.get('http://httpbin.org/get', expire_after=-1)
+```
+
+For better performance, expired responses won't be removed immediately, but will be removed
+(or replaced) the next time they are accessed. To manually clear all expired responses:
+```python
+session.remove_expired_responses()
+```
+Or, when using patching:
+```python
+requests_cache.remove_expired_responses()
+```
+
+Or, to revalidate the cache with a new expiration:
+```python
+session.remove_expired_responses(expire_after=360)
+```
## Related Projects
If `requests-cache` isn't quite what you need, you can help make it better! See the
@@ -72,8 +164,3 @@ You can also check out these other python cache projects:
file-based cache built on SQLite
* [aiocache](https://github.com/aio-libs/aiocache): General-purpose (not HTTP-specific) async cache
backends
-
-## Links
-- **Documentation** at [readthedocs](https://requests-cache.readthedocs.io)
-- **Source code and issue tracking** at [GitHub](https://github.com/reclosedev/requests-cache)
-- **Working example** at [Real Python](https://realpython.com/blog/python/caching-external-api-requests)
diff --git a/docs/advanced_usage.rst b/docs/advanced_usage.rst
new file mode 100644
index 0000000..cf55778
--- /dev/null
+++ b/docs/advanced_usage.rst
@@ -0,0 +1,349 @@
+.. _advanced-usage:
+
+Advanced Usage
+==============
+.. contents::
+ :local:
+
+CachedSession Options
+---------------------
+See :py:class:`requests_cache.CachedSession` for a full list of parameters.
+
+Cache Name
+~~~~~~~~~~
+The ``cache_name`` parameter will be used as follows depending on the backend:
+
+* ``sqlite``: Cache filename, e.g ``my_cache.sqlite``
+* ``mongodb``: Database name
+* ``redis``: Namespace, meaning all keys will be prefixed with ``'cache_name:'``
+
+Cache Keys
+~~~~~~~~~~
+The cache key is a hash created from request information, and is used as an index for cached
+responses. There are a couple ways you can customize what information is used to create this key:
+
+* Use ``include_get_headers`` if you want headers to be included in the cache key. In other
+ words, this will create separate cache items for responses with different headers.
+* Use ``ignored_parameters`` to exclude specific request params from the cache key. This is
+ useful, for example, if you request the same resource with different credentials or access
+ tokens.
+
+HTTP methods and status codes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+You can choose which request HTTP methods and response status codes you want to cache using the
+parameters ``allowable_methods`` and ``allowable_codes``, respectively. By default, only GET and HEAD
+requests and 200 responses are cached. Example:
+
+ >>> from requests_cache import CachedSession
+ >>>
+ >>> session = CachedSession(
+ >>> allowable_methods=('GET', 'POST'),
+ >>> allowable_codes=(200, 418),
+ >>> )
+
+Custom response filter
+~~~~~~~~~~~~~~~~~~~~~~
+If you need more advanced behaviour for determining what to cache, you can provide a custom filtering
+function via the ``filter_fn`` param. This function that takes a :py:class:`requests.Response` object
+and returns a boolean indicating whether or not that response should be cached. It will be applied to
+both new responses (on write) and previously cached responses (on read). Example:
+
+ >>> from sys import getsizeof
+ >>> from requests_cache import CachedSession
+ >>>
+ >>> def filter_by_size(response):
+ >>> """Don't cache responses with a body over 1 MB"""
+ >>> return getsizeof(response.content) <= 1024 * 1024
+ >>>
+ >>> session = CachedSession(filter_fn=filter_by_size)
+
+Cache Expiration
+~~~~~~~~~~~~~~~~
+Use ``expire_after`` to specify how long responses will be cached. This can be:
+
+* A positive number (in seconds)
+* ``-1`` (to never expire)
+* A :py:class:`.timedelta`
+* A :py:class:`datetime`
+
+This will only apply to responses cached in the current session; to apply a different expiration
+to previously cached responses, see :py:meth:`remove_expired_responses`.
+
+Expiration can also be set on a per-URL or per request basis. The following order of precedence
+is used:
+
+1. Per-request expiration (``expire_after`` argument for :py:meth:`.request`)
+2. Per-URL expiration (``urls_expire_after`` argument for ``CachedSession``)
+3. Per-session expiration (``expire_after`` argument for ``CachedSession``)
+
+URL Patterns
+~~~~~~~~~~~~
+You can use ``expire_after_urls`` to set different expiration times for different requests, based on
+URL glob patterns. This allows you to customize caching based on what you know about the resources
+you're requesting. For example, you might request one resource that gets updated frequently, another
+that changes infrequently, and another that never changes. Example:
+
+ >>> urls_expire_after = {
+ >>> '*.site_1.com': 30,
+ >>> 'site_2.com/resource_1': 60 * 2,
+ >>> 'site_2.com/resource_2': 60 * 60 * 24,
+ >>> 'site_2.com/static': -1,
+ >>> }
+
+**Notes:**
+
+* ``urls_expire_after`` should be a dict in the format ``{'pattern': expire_after}``
+* ``expire_after`` accepts the same types as ``CachedSession.expire_after``
+* Patterns will match request **base URLs**, so the pattern ``site.com/resource/`` is equivalent to
+ ``http*://site.com/resource/**``
+* If there is more than one match, the first match will be used in the order they are defined
+* If no patterns match a request, ``expire_after`` will be used as a default.
+
+Cache Inspection
+----------------
+Here are some ways to get additional information out of the cache session, backend, and responses:
+
+Response Attributes
+~~~~~~~~~~~~~~~~~~~
+The following attributes are available on responses:
+* ``from_cache``: indicates if the response came from the cache
+* ``created_at``: ``datetime`` of when the cached response was created or last updated
+* ``expires``: ``datetime`` after which the cached response will expire
+* ``is_expired``: indicates if the cached response is expired (if an old response was returned due to a request error)
+
+Examples:
+
+ >>> from requests_cache import CachedSession
+ >>> session = CachedSession(expire_after=timedelta(days=1))
+
+ >>> # Placeholders are added for non-cached responses
+ >>> r = session.get('http://httpbin.org/get')
+ >>> print(r.from_cache, r.created_at, r.expires, r.is_expired)
+ False None None None
+
+ >>> # Values will be populated for cached responses
+ >>> r = session.get('http://httpbin.org/get')
+ >>> print(r.from_cache, r.created_at, r.expires, r.is_expired)
+ True 2021-01-01 18:00:00 2021-01-02 18:00:00 False
+
+Cache Contents
+~~~~~~~~~~~~~~
+You can use :py:meth:`.CachedSession.cache.urls` to see all URLs currently in the cache:
+
+ >>> session = CachedSession()
+ >>> print(session.urls)
+ ['https://httpbin.org/get', 'https://httpbin.org/stream/100']
+
+If needed, you can get more details on cached responses via ``CachedSession.cache.responses``, which
+is a dict-like interface to the cache backend. See :py:class:`requests.Response` and
+:py:class:`.CachedResponse` for a full list of attributes available.
+
+For example, if you wanted to to see all URLs requested with a specific method:
+
+ >>> post_urls = [
+ >>> response.url for response in session.cache.responses.values()
+ >>> if response.request.method == 'POST'
+ >>> ]
+
+You can also inspect ``CachedSession.cache.redirects``, which maps redirect URLs to keys of the
+responses they redirect to.
+
+Custom Backends
+---------------
+If the built-in :ref:`cache_backends` don't suit your needs and you want to create your own, you can create
+subclasses of :py:class:`.BaseCache` and :py:class:`.BaseStorage`:
+
+ >>> from requests_cache import CachedSession
+ >>> from requests_cache.backends import BaseCache, BaseStorage
+ >>>
+ >>> class MyCache(BaseCache):
+ >>> """Wrapper for higher-level cache operations"""
+ >>> def __init__(self, **kwargs):
+ >>> super().__init__(**kwargs)
+ >>> self.redirects = MyStorage(**kwargs)
+ >>> self.responses = MyStorage(**kwargs)
+ >>>
+ >>> class MyStorage(BaseStorage):
+ >>> """Lower-level backend storage operations"""
+
+You can then use your custom backend in a ``CachedSession`` with the ``backend`` parameter:
+
+ >>> session = CachedSession(backend=MyCache())
+
+Security
+--------
+The python ``pickle`` module has some
+`known security vulnerabilities <https://docs.python.org/3/library/pickle.html>`_,
+meaning it should only be used to serialize and deserialize data you trust. Since this isn't always
+possible, requests-cache can optionally use `itsdangerous <https://itsdangerous.palletsprojects.com>`_
+to add a layer of security around these operations.
+
+It works by signing serialized data with a secret key that you control. Then, if the data is tampered
+with, the signature check fails and raises an error. To enable this behavior, first install the extra package::
+
+ $ pip install itsdangerous
+
+Then create your key and set it in your application. A common pattern for this (or any other
+application credentials) is to store it wherever you store the rest of your credentials
+(system keyring, password database, etc.) and set it in an environment variable.
+
+Then, initialize your cache with your key:
+
+ >>> import os
+ >>> from requests_cache import CachedSession
+ >>> session = CachedSession(secret_key=os.environ['SECRET_KEY'])
+ >>> session.get('https://httpbin.org/get')
+
+You can verify that it's working by modifying the cached item (*without* your key):
+
+ >>> session_2 = CachedSession(secret_key='a different key')
+ >>> cache_key = list(session_2.cache.responses.keys())[0]
+ >>> session_2.cache.responses[cache_key] = 'exploit!'
+
+Then, if you try to get that cached response again (*with* your key), you will get an error:
+
+ >>> session.get('https://httpbin.org/get')
+ BadSignature: Signature b'iFNmzdUOSw5vqrR9Cb_wfI1EoZ8' does not match
+
+
+Usage with other requests features
+----------------------------------
+
+Request Hooks
+~~~~~~~~~~~~~
+Requests has an `Event Hook <https://requests.readthedocs.io/en/master/user/advanced/#event-hooks>`_
+system that can be used to add custom behavior into different parts of the request process.
+It can be used, for example, for request throttling:
+
+ >>> import time
+ >>> import requests
+ >>> from requests_cache import CachedSession
+ >>>
+ >>> def make_throttle_hook(timeout=1.0):
+ >>> """Make a request hook function that adds a custom delay for non-cached requests"""
+ >>> def hook(response, *args, **kwargs):
+ >>> if not getattr(response, 'from_cache', False):
+ >>> print('sleeping')
+ >>> time.sleep(timeout)
+ >>> return response
+ >>> return hook
+ >>>
+ >>> session = CachedSession()
+ >>> session.hooks['response'].append(make_throttle_hook(0.1))
+ >>> # The first (real) request will have an added delay
+ >>> session.get('http://httpbin.org/get')
+ >>> session.get('http://httpbin.org/get')
+
+Streaming Requests
+~~~~~~~~~~~~~~~~~~
+If you use `streaming requests <https://2.python-requests.org/en/master/user/advanced/#id9>`_, you
+can use the same code to iterate over both cached and non-cached requests. A cached request will,
+of course, have already been read, but will use a file-like object containing the content.
+Example::
+
+ from requests_cache import CachedSession
+
+ session = CachedSession()
+ for i in range(2):
+ r = session.get('https://httpbin.org/stream/20', stream=True)
+ for chunk in r.iter_lines():
+ print(chunk.decode('utf-8'))
+
+
+.. _library_compatibility:
+
+Usage with other requests-based libraries
+-----------------------------------------
+This library works by patching and/or extending ``requests.Session``. Many other libraries out there
+do the same thing, making it potentially difficult to combine them. For that scenario, a mixin class
+is provided, so you can create a custom class with behavior from multiple Session-modifying libraries::
+
+ from requests import Session
+ from requests_cache import CacheMixin
+ from some_other_lib import SomeOtherMixin
+
+ class CustomSession(CacheMixin, SomeOtherMixin ClientSession):
+ """Session class with features from both requests-html and requests-cache"""
+
+Requests-HTML
+~~~~~~~~~~~~~
+Example with `requests-html <https://github.com/psf/requests-html>`_::
+
+ import requests
+ from requests_cache import CacheMixin, install_cache
+ from requests_html import HTMLSession
+
+ class CachedHTMLSession(CacheMixin, HTMLSession):
+ """Session with features from both CachedSession and HTMLSession"""
+
+ session = CachedHTMLSession()
+ r = session.get("https://github.com/")
+ print(r.from_cache, r.html.links)
+
+Or, using the monkey-patch method::
+
+ install_cache(session_factory=CachedHTMLSession)
+ r = requests.get("https://github.com/")
+ print(r.from_cache, r.html.links)
+
+The same approach can be used with other libraries that subclass ``requests.Session``.
+
+Requests-futures
+~~~~~~~~~~~~~~~~
+Example with `requests-futures <https://github.com/ross/requests-futures>`_:
+
+Some libraries, including `requests-futures`, support wrapping an existing session object.
+
+ session = FutureSession(session=CachedSession())
+
+In this case, ``FutureSession`` must wrap ``CachedSession`` rather than the other way around, since
+``FutureSession`` returns (as you might expect) futures rather than response objects.
+See `issue #135 <https://github.com/reclosedev/requests-cache/issues/135>`_ for more notes on this.
+
+Requests-mock
+~~~~~~~~~~~~~
+Example with `requests-mock <https://github.com/jamielennox/requests-mock>`_:
+
+Requests-mock works a bit differently. It has multiple methods of mocking requests, and the
+method most compatible with requests-cache is attaching its
+`adapter <https://requests-mock.readthedocs.io/en/latest/adapter.html>`_ to a CachedSession::
+
+ import requests
+ from requests_mock import Adapter
+ from requests_cache import CachedSession
+
+ # Set up a CachedSession that will make mock requests where it would normally make real requests
+ adapter = Adapter()
+ adapter.register_uri(
+ 'GET',
+ 'mock://some_test_url',
+ headers={'Content-Type': 'text/plain'},
+ text='mock response',
+ status_code=200,
+ )
+ session = CachedSession()
+ session.mount('mock://', adapter)
+
+ session.get('mock://some_test_url', text='mock_response')
+ response = session.get('mock://some_test_url')
+ print(response.text)
+
+Internet Archive
+~~~~~~~~~~~~~~~~
+Example with `internetarchive <https://github.com/jjjake/internetarchive>`_:
+
+Usage is the same as other libraries that subclass `requests.Session`::
+
+ from requests_cache import CacheMixin
+ from internetarchive.session import ArchiveSession
+
+ class CachedArchiveSession(CacheMixin, ArchiveSession):
+ """Session with features from both CachedSession and ArchiveSession"""
+
+
+Potential Issues
+----------------
+
+.. warning:: Version updates of ``requests``, ``urllib3`` or ``requests_cache`` itself may not be
+ compatible with previously cached data (see `issue #56 <https://github.com/reclosedev/requests-cache/issues/56>`_).
+ To prevent this, you can use a virtualenv and pin your dependency versions.
diff --git a/docs/api.rst b/docs/api.rst
index ad13a49..cae87cf 100644
--- a/docs/api.rst
+++ b/docs/api.rst
@@ -1,52 +1,41 @@
API
===
+This section covers all the public interfaces of ``requests-cache``
-This part of the documentation covers all the interfaces of `requests-cache`
-
-
-Public api
+Public API
----------
+.. Explicitly show inherited method docs on CachedSession instead of CachedMixin
+.. autoclass:: requests_cache.core.CachedSession
+ :members: send, request, cache_disabled, remove_expired_responses
+ :show-inheritance:
+
+.. autoclass:: requests_cache.core.CacheMixin
.. automodule:: requests_cache.core
- :members:
+ :members:
+ :exclude-members: CachedSession, CacheMixin
-----------------------------------------------
+.. automodule:: requests_cache.response
+ :members:
-.. _cache_backends:
+----------------------------------------------
-Cache backends
+Cache Backends
--------------
-
.. automodule:: requests_cache.backends.base
:members:
-.. _backends_sqlite:
-
.. automodule:: requests_cache.backends.sqlite
:members:
-.. _backends_mongo:
-
.. automodule:: requests_cache.backends.mongo
:members:
-.. _backends_redis:
-
-.. automodule:: requests_cache.backends.redis
+.. automodule:: requests_cache.backends.gridfs
:members:
-----------------------------------------------
-
-Internal modules which can be used outside
-------------------------------------------
-
-.. _backends_dbdict:
-
-.. automodule:: requests_cache.backends.storage.dbdict
- :members:
-
-.. automodule:: requests_cache.backends.storage.mongodict
+.. automodule:: requests_cache.backends.redis
:members:
-.. automodule:: requests_cache.backends.storage.redisdict
+.. automodule:: requests_cache.backends.dynamodb
:members:
diff --git a/docs/conf.py b/docs/conf.py
index d152386..f2c2ae8 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -21,19 +21,21 @@ templates_path = ['_templates']
# Sphinx extension modules
extensions = [
'sphinx.ext.autodoc',
+ 'sphinx.ext.autosectionlabel',
'sphinx.ext.intersphinx',
'sphinx.ext.napoleon',
'sphinx.ext.viewcode',
'sphinx_autodoc_typehints',
'sphinx_copybutton',
# 'sphinxcontrib.apidoc',
- # 'm2r2',
+ 'm2r2',
]
# Enable automatic links to other projects' Sphinx docs
intersphinx_mapping = {
'python': ('https://docs.python.org/3', None),
'requests': ('https://requests.readthedocs.io/en/master/', None),
+ 'urllib3': ('http://urllib3.readthedocs.org/en/latest', None),
}
# Enable Google-style docstrings
diff --git a/docs/contributing.rst b/docs/contributing.rst
new file mode 100644
index 0000000..4fc5016
--- /dev/null
+++ b/docs/contributing.rst
@@ -0,0 +1 @@
+.. mdinclude:: ../CONTRIBUTING.md
diff --git a/docs/history.rst b/docs/history.rst
new file mode 100644
index 0000000..d26e5be
--- /dev/null
+++ b/docs/history.rst
@@ -0,0 +1 @@
+.. mdinclude:: ../HISTORY.md
diff --git a/docs/index.rst b/docs/index.rst
index ad390d1..78d1d14 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,26 +1,21 @@
-.. requests-cache documentation master file, created by
- sphinx-quickstart on Sun Apr 08 20:51:24 2012.
- You can adapt this file completely to your liking, but it should at least
- contain the root `toctree` directive.
-
Requests-cache documentation
============================
-`Requests-cache <http://pypi.python.org/pypi/requests-cache>`_
-is a transparent persistent cache for requests_ (version >= 1.1.0) library.
-
-Source code and issue tracking can be found at
-`GitHub <https://github.com/reclosedev/requests-cache>`_.
-
+.. Include Readme contents (except for the link to readthedocs, since we're already on readthedocs!)
+.. mdinclude:: ../README.md
+ :end-line: 17
-**Contents:**
+.. mdinclude:: ../README.md
+ :start-line: 19
.. toctree::
- :maxdepth: 2
-
- user_guide
- api
+ :caption: Contents
+ :maxdepth: 2
+ advanced_usage
+ api
+ contributing
+ history
Indices and tables
==================
@@ -28,5 +23,3 @@ Indices and tables
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
-
-.. _requests: http://docs.python-requests.org/ \ No newline at end of file
diff --git a/docs/user_guide.rst b/docs/user_guide.rst
index 23d4d45..7ab8737 100644
--- a/docs/user_guide.rst
+++ b/docs/user_guide.rst
@@ -1,228 +1,5 @@
-.. _user_guide:
+.. This is just a placeholder in case anyone had this page bookmarked
-User guide
+User Guide
==========
-
-
-Installation
-------------
-
-Install with pip_ or easy_install_::
-
- pip install --upgrade requests-cache
-
-or download latest version from version control::
-
- git clone git://github.com/reclosedev/requests-cache.git
- cd requests-cache
- python setup.py install
-
-
-.. warning:: Version updates of ``requests``, ``urllib3`` or ``requests_cache`` itself can break existing
- cache database (see https://github.com/reclosedev/requests-cache/issues/56 ).
- So if your code relies on cache, or is expensive in terms of time and traffic, please be sure to use
- something like ``virtualenv`` and pin your requirements.
-
-.. _pip: http://pypi.python.org/pypi/pip/
-.. _easy_install: http://pypi.python.org/pypi/setuptools
-
-Usage
------
-
-.. currentmodule:: requests_cache.core
-
-There is two ways of using :mod:`requests_cache`:
-
- - Using :class:`CachedSession` instead ``requests.Session``
- - Monkey patching ``requests`` to use :class:`CachedSession` by default
-
-Monkey-patching allows to add caching to existent program by adding just two lines:
-
-Import :mod:`requests_cache` and call :func:`install_cache`
-::
-
- import requests
- import requests_cache
-
- requests_cache.install_cache()
-
-And you can use ``requests``, all responses will be cached transparently!
-
-For example, following code will take only 1-2 seconds instead 10::
-
- for i in range(10):
- requests.get('http://httpbin.org/delay/1')
-
-Cache can be configured with some options, such as cache filename, backend
-(sqlite, mongodb, redis, memory), expiration time, etc. E.g. cache stored in sqlite
-database (default format) named ``'test_cache.sqlite'`` with expiration
-set to 300 seconds can be configured as::
-
- requests_cache.install_cache('test_cache', backend='sqlite', expire_after=300)
-
-.. seealso::
- Full list of options can be found in
- :func:`requests_cache.install_cache() <requests_cache.core.install_cache>` reference
-
-
-Transparent caching is achieved by monkey-patching ``requests`` library
-It is possible to uninstall this patch with :func:`requests_cache.uninstall_cache() <requests_cache.core.uninstall_cache>`.
-
-Also, you can use :func:`requests_cache.disabled() <requests_cache.core.disabled>`
-context manager for temporary disabling caching::
-
- with requests_cache.disabled():
- print(requests.get('http://httpbin.org/ip').text)
-
-
-If ``Response`` is taken from cache, ``from_cache`` attribute will be ``True``:
-::
-
- >>> import requests
- >>> import requests_cache
- >>> requests_cache.install_cache()
- >>> requests_cache.clear()
- >>> r = requests.get('http://httpbin.org/get')
- >>> r.from_cache
- False
- >>> r = requests.get('http://httpbin.org/get')
- >>> r.from_cache
- True
-
-It can be used, for example, for request throttling with help of ``requests`` hook system::
-
- import time
- import requests
- import requests_cache
-
- def make_throttle_hook(timeout=1.0):
- """
- Returns a response hook function which sleeps for `timeout` seconds if
- response is not cached
- """
- def hook(response, *args, **kwargs):
- if not getattr(response, 'from_cache', False):
- print('sleeping')
- time.sleep(timeout)
- return response
- return hook
-
- if __name__ == '__main__':
- requests_cache.install_cache('wait_test')
- requests_cache.clear()
-
- s = requests_cache.CachedSession()
- s.hooks['response'].append(make_throttle_hook(0.1))
- s.get('http://httpbin.org/delay/get')
- s.get('http://httpbin.org/delay/get')
-
-.. seealso:: `example.py <https://github.com/reclosedev/requests-cache/blob/master/example.py>`_
-
-.. note:: requests_cache prefetchs response content, be aware if your code uses streaming requests.
-
-
-.. _request_caching:
-
-Request caching
----------------
-
-If you want to cache on a per request basis, all requests support the ``expire_after`` keyword argument, which takes an expiration time in seconds::
-
- requests.get('https://httpbin.org/get', expire_after=10)
-
-Alternatively, ``expire_after`` accepts :class:`None`, ``'default'``, or a :class:`~datetime.timedelta`. :class:`None` enables infinite caching, ``'default'`` uses the expiry defined in :func:`install_cache`, and the timedelta is used as is.
-
-.. note:: Internally, numbers are converted to :class:`~datetime.timedelta` instances as well.
-
-.. seealso:: `example_request.py <https://github.com/reclosedev/requests-cache/blob/master/example_request.py>`_
-
-
-.. _persistence:
-
-Persistence
------------
-
-:mod:`requests_cache` designed to support different backends for persistent storage.
-By default it uses ``sqlite`` database. Type of storage can be selected with ``backend`` argument of :func:`install_cache`.
-
-List of available backends:
-
-- ``'sqlite'`` - sqlite database (**default**)
-- ``'memory'`` - not persistent, stores all data in Python ``dict`` in memory
-- ``'mongodb'`` - (**experimental**) MongoDB database (``pymongo < 3.0`` required)
-- ``'redis'`` - stores all data on a redis data store (``redis`` required)
-
-You can write your own and pass instance to :func:`install_cache` or :class:`CachedSession` constructor.
-See :ref:`cache_backends` API documentation and sources.
-
-.. _expiration:
-
-Expiration
-----------
-
-If you are using cache with ``expire_after`` parameter set, responses are removed from the storage only when the same
-request is made. Since the store sizes can get out of control pretty quickly with expired items
-you can remove them using :func:`remove_expired_responses`
-or :meth:`BaseCache.remove_old_entries(created_before) <requests_cache.backends.base.BaseCache.remove_old_entries>`.
-::
-
- expire_after = timedelta(hours=1)
- requests_cache.install_cache(expire_after=expire_after)
- ...
- requests_cache.core.remove_expired_responses()
- # or
- remove_old_entries.get_cache().remove_old_entries(datetime.utcnow() - expire_after)
- # when used as session
- session = CachedSession(..., expire_after=expire_after)
- ...
- session.cache.remove_old_entries(datetime.utcnow() - expire_after)
-
-
-For more information see :doc:`API reference <api>`.
-
-Usage with other libraries that modify requests.Session
--------------------------------------------------------
-This library works by patching ``requests.Session``. Many other libraries out there do the same
-thing, making it difficult to combine them. For that case, a mixin class is provided, so you can
-create a custom class with behavior from multiple Session-modifying libraries::
-
- from requests import Session
- from requests_cache import CacheMixin
- from some_other_lib import SomeOtherMixin
-
- class CustomSession(CacheMixin, SomeOtherMixin ClientSession):
- """Session class with features from both requests-html and requests-cache"""
-
-
-Example with `requests-html <https://github.com/psf/requests-html>`_::
-
- import requests
- from requests_cache import CacheMixin, install_cache
- from requests_html import HTMLSession
-
- class CachedHTMLSession(CacheMixin, HTMLSession):
- """Session with features from both CachedSession and HTMLSession"""
-
- session = CachedHTMLSession()
- r = session.get("https://github.com/")
- print(r.from_cache, r.html.links)
-
-Or, using the monkey-patch method::
-
- install_cache(session_factory=CachedHTMLSession)
- r = requests.get("https://github.com/")
- print(r.from_cache, r.html.links)
-
-The same approach can be used with other libraries that subclass ``requests.Session``.
-
-Example with `requests-mock <https://github.com/jamielennox/requests-mock>`_::
-
- import requests
- from requests_mock import Mocker
- from requests_cache import CachedSession
-
- session = CachedSession()
- with Mocker(session=session) as m:
- m.get('http://test.com', text='mock_response')
- response = session.get('http://test.com')
- print(response.text)
+This page has moved. Please see :ref:`usage` and :ref:`advanced-usage` sections.
diff --git a/example.py b/example.py
deleted file mode 100644
index f1e8211..0000000
--- a/example.py
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/usr/bin/env python
-import time
-
-import requests
-
-import requests_cache
-
-requests_cache.install_cache('example_cache')
-
-
-def main():
- # Once cached, delayed page will be taken from cache
- # redirects also handled
- for i in range(5):
- requests.get('http://httpbin.org/delay/2')
- r = requests.get('http://httpbin.org/redirect/5')
- print(r.text)
-
- # And if we need to get fresh page or don't want to cache it?
- with requests_cache.disabled():
- print(requests.get('http://httpbin.org/ip').text)
-
- # Debugging info about cache
- print(requests_cache.get_cache())
-
-
-if __name__ == "__main__":
- t = time.time()
- main()
- print('Elapsed: %.3f seconds' % (time.time() - t))
diff --git a/example_request.py b/example_request.py
deleted file mode 100644
index ead525d..0000000
--- a/example_request.py
+++ /dev/null
@@ -1,32 +0,0 @@
-#!/usr/bin/env python
-import time
-
-import requests
-
-import requests_cache
-
-requests_cache.install_cache('example_cache', backend='sqlite')
-
-
-def main():
- # Normal caching forever
- response = requests.get('https://httpbin.org/get')
- assert not response.from_cache
- response = requests.get('https://httpbin.org/get')
- assert response.from_cache
-
- # Caching with expiration
- requests_cache.clear()
- response = requests.get('https://httpbin.org/get', expire_after=1)
- response = requests.get('https://httpbin.org/get')
- assert response.from_cache
- # After > 1 second, it's cached value is expired
- time.sleep(1.2)
- response = requests.get('https://httpbin.org/get')
- assert not response.from_cache
-
-
-if __name__ == "__main__":
- t = time.perf_counter()
- main()
- print('Elapsed: %.3f seconds' % (time.perf_counter() - t))
diff --git a/examples/basic_usage.py b/examples/basic_usage.py
new file mode 100644
index 0000000..3888681
--- /dev/null
+++ b/examples/basic_usage.py
@@ -0,0 +1,31 @@
+#!/usr/bin/env python
+"""A Simple example using requests-cache with httpbin"""
+import time
+
+from requests_cache import CachedSession
+
+
+def main():
+ session = CachedSession('example_cache', backend='sqlite')
+
+ # The real request will only be made once; afterward, the cached response is used
+ for i in range(5):
+ response = session.get('http://httpbin.org/get')
+
+ # This is more obvious when calling a slow endpoint
+ for i in range(5):
+ response = session.get('http://httpbin.org/delay/2')
+
+ # Caching can be disabled if we want to get a fresh page and not cache it
+ with session.cache_disabled():
+ print(session.get('http://httpbin.org/ip').text)
+
+ # Get some debugging info about the cache
+ print(session.cache)
+ print('Cached URLS:', session.cache.urls)
+
+
+if __name__ == "__main__":
+ t = time.time()
+ main()
+ print('Elapsed: %.3f seconds' % (time.time() - t))
diff --git a/examples/expiration.py b/examples/expiration.py
new file mode 100644
index 0000000..4afa5fa
--- /dev/null
+++ b/examples/expiration.py
@@ -0,0 +1,42 @@
+#!/usr/bin/env python
+"""An example of setting expiration for individual requests"""
+import time
+
+from requests_cache import CachedSession
+
+
+def main():
+ session = CachedSession('example_cache', backend='sqlite')
+
+ # By default, cached responses never expire
+ response = session.get('https://httpbin.org/get')
+ assert not response.from_cache
+ response = session.get('https://httpbin.org/get')
+ assert response.from_cache
+ assert not response.expires
+
+ # We can set default expiration for the session using expire_after
+ session = CachedSession('example_cache', backend='sqlite', expire_after=60)
+ session.cache.clear()
+ response = session.get('https://httpbin.org/get')
+ response = session.get('https://httpbin.org/get')
+ print('Expiration time:', response.expires)
+
+ # This can also be overridden for individual requests
+ session.cache.clear()
+ response = session.get('https://httpbin.org/get', expire_after=1)
+ response = session.get('https://httpbin.org/get')
+ assert response.from_cache
+ print('Expiration time:', response.expires)
+
+ # After 1 second, the cached value will expired
+ time.sleep(1.2)
+ assert response.is_expired
+ response = session.get('https://httpbin.org/get')
+ assert not response.from_cache
+
+
+if __name__ == "__main__":
+ t = time.perf_counter()
+ main()
+ print('Elapsed: %.3f seconds' % (time.perf_counter() - t))
diff --git a/examples/session_patch.py b/examples/session_patch.py
new file mode 100644
index 0000000..8ae900b
--- /dev/null
+++ b/examples/session_patch.py
@@ -0,0 +1,36 @@
+#!/usr/bin/env python
+"""The same as `basic_usage.py`, but using global session patching"""
+import time
+
+import requests
+import requests_cache
+
+# After installation, all requests functions and Session methods will be cached
+requests_cache.install_cache('example_cache', backend='sqlite')
+
+
+def main():
+ # The real request will only be made once; afterward, the cached response is used
+ for i in range(5):
+ response = requests.get('http://httpbin.org/get')
+
+ # This is more obvious when calling a slow endpoint
+ for i in range(5):
+ response = requests.get('http://httpbin.org/delay/2')
+
+ # Caching can be disabled if we want to get a fresh page and not cache it
+ with requests_cache.disabled():
+ print(requests.get('http://httpbin.org/ip').text)
+
+ # Get some debugging info about the cache
+ print(requests_cache.get_cache())
+ print('Cached URLS:', requests_cache.get_cache().urls)
+
+ # Uninstall to remove caching from all requests functions
+ requests_cache.uninstall_cache()
+
+
+if __name__ == "__main__":
+ t = time.time()
+ main()
+ print('Elapsed: %.3f seconds' % (time.time() - t))
diff --git a/pyproject.toml b/pyproject.toml
index b2bafe1..aa18291 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -16,7 +16,7 @@ source = ['requests_cache']
profile = "black"
line_length = 100
skip_gitignore = true
-skip = ['requests_cache/__init__.py']
+skip = ['examples/', 'requests_cache/__init__.py']
known_first_party = ['test']
# Things that are common enough they may as well be grouped with stdlib imports
extra_standard_library = ['pytest', 'setuptools']
diff --git a/setup.cfg b/setup.cfg
index 37718f7..216ff4c 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -3,9 +3,9 @@ universal = 1
[metadata]
description = Persistent cache for requests library
-long_description = file: README.rst, HISTORY.rst
-long_description_content_type = text/x-rst
-keywords = requests, cache, persistence
+long_description = file: README.md, HISTORY.md
+long_description_content_type = text/markdown
+keywords = requests, http, cache, persistence, sqlite, redis, mongodb, gridfs, dynamodb
license = BSD License
license_files = LICENSE
python_requires = >=3.6