diff options
| author | Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> | 2019-06-16 00:10:06 -0700 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2019-06-16 00:10:06 -0700 |
| commit | 45d6547acfb9ae1639adbe03dd14f38cd0642ca2 (patch) | |
| tree | 28e30a63d8ff5b43b22152c3a93153f97e15ae51 /Misc | |
| parent | 159ae24895272dce5fd53dd8e54809743e4f394f (diff) | |
| download | cpython-git-45d6547acfb9ae1639adbe03dd14f38cd0642ca2.tar.gz | |
bpo-35922: Fix RobotFileParser when robots.txt has no relevant crawl delay or request rate (GH-11791)
Co-Authored-By: Tal Einat <taleinat+github@gmail.com>
(cherry picked from commit 8047e0e1c620f69cc21f9ca48b24bf2cdd5c3668)
Co-authored-by: Rémi Lapeyre <remi.lapeyre@henki.fr>
Diffstat (limited to 'Misc')
| -rw-r--r-- | Misc/NEWS.d/next/Library/2019-06-11-19-34-29.bpo-35922.rxpzWr.rst | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/Misc/NEWS.d/next/Library/2019-06-11-19-34-29.bpo-35922.rxpzWr.rst b/Misc/NEWS.d/next/Library/2019-06-11-19-34-29.bpo-35922.rxpzWr.rst new file mode 100644 index 0000000000..5271a49562 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2019-06-11-19-34-29.bpo-35922.rxpzWr.rst @@ -0,0 +1,4 @@ +Fix :meth:`RobotFileParser.crawl_delay` and +:meth:`RobotFileParser.request_rate` to return ``None`` rather than +raise :exc:`AttributeError` when no relevant rule is defined in the +robots.txt file. Patch by Rémi Lapeyre. |