summaryrefslogtreecommitdiff
path: root/Lib/test/test_robotparser.py
Commit message (Collapse)AuthorAgeFilesLines
* [2.7] bpo-32861: robotparser fix incomplete __str__ methods. (GH-5711) ↵Serhiy Storchaka2018-05-151-0/+26
| | | | | | | | | (GH-6795) (GH-6817) The robotparser's __str__ representation now includes wildcard entries. (cherry picked from commit c3fa1f2b93fa4bf96a8aadc74ee196384cefa31e) Co-authored-by: Michael Lazar <lazar.michael22@gmail.com>.
* bpo-30759: Copy test_robotparser from master (#2546)Victor Stinner2017-07-051-239/+177
| | | | | | | | | | | | | | | | | Copy Lib/test/test_robotparser.py from master to 2.7 and adapt it for Python 2.7: * Replace urllib.robotparser with robotparser * Adjust HTTPServer import * Replace io.StringIO with StringIO.StringIO * Remove tests on crawl_delay() and request_rate() since these methods were added to Python 3 * Remove subTest() * Add test_main() which explicitly lists all test cases Patch based on the commit 4da0fd06ce49132d5c2e0525c4a31f807690a9ed written by Berker Peksag. Co-Authored-By: Berker Peksag <berker.peksag@gmail.com>
* don't fail tests when www.python.org can't be validated by the systemBenjamin Peterson2014-11-251-2/+3
|
* Issue #20939: Use www.example.com instead of www.python.org to avoid testNed Deily2014-03-261-0/+7
| | | | failures when ssl is not present.
* #17403: urllib.parse.robotparser normalizes the urls before adding to ruleline.Senthil Kumaran2013-05-291-0/+12
| | | | This helps in handling certain types invalid urls in a conservative manner.
* Avoid failing in test_robotparser when mueblesmoraleda.com is flaky andAntoine Pitrou2011-07-081-1/+16
| | | | | an overzealous DNS service (e.g. OpenDNS) redirects to a placeholder Web site.
* Merged revisions 84597-84599 via svnmerge fromAntoine Pitrou2010-09-071-14/+15
| | | | | | | | | | | | | | | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/branches/py3k ........ r84597 | antoine.pitrou | 2010-09-07 22:42:19 +0200 (mar., 07 sept. 2010) | 5 lines Issue #8574: better implementation of test.support.transient_internet(). Original patch by Victor. ........ r84598 | antoine.pitrou | 2010-09-07 23:05:49 +0200 (mar., 07 sept. 2010) | 6 lines Issue #9792: In case of connection failure, socket.create_connection() would swallow the exception and raise a new one, making it impossible to fetch the original errno, or to filter timeout errors. Now the original error is re-raised. ........ r84599 | antoine.pitrou | 2010-09-07 23:09:09 +0200 (mar., 07 sept. 2010) | 4 lines Improve transient_internet() again to detect more network errors, and use it in test_robotparser. Fixes #8574. ........
* Merged revisions 83238 via svnmerge fromGeorg Brandl2010-08-011-0/+14
| | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/branches/py3k ........ r83238 | georg.brandl | 2010-07-29 19:55:01 +0200 (Do, 29 Jul 2010) | 1 line #4108: the first default entry (User-agent: *) wins. ........
* Merged revisions 83209 via svnmerge fromSenthil Kumaran2010-07-281-0/+11
| | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/branches/py3k ........ r83209 | senthil.kumaran | 2010-07-28 21:57:56 +0530 (Wed, 28 Jul 2010) | 3 lines Fix Issue6325 - robotparse to honor urls with query strings. ........
* Backport some robotparser test and skip the test if the external resource is ↵Florent Xicluna2010-04-021-5/+18
| | | | not available.
* convert usage of fail* to assert*Benjamin Peterson2009-06-301-2/+2
|
* Close issue 3437 - missing state change when Allow lines are processed.Skip Montanaro2008-07-271-0/+69
| | | | Adds test cases which use Allow: as well.
* Correct attribute name.Georg Brandl2008-07-181-1/+1
|
* fixes 813986Skip Montanaro2007-08-281-0/+11
|
* Standardize on test.test_support.run_unittest() (as opposed to a mix of ↵Collin Winter2007-04-251-2/+2
| | | | run_unittest() and run_suite()). Also, add functionality to run_unittest() that admits usage of unittest.TestLoader.loadTestsFromModule().
* Get rid of relative imports in all unittests. Now anything thatBarry Warsaw2002-07-231-1/+2
| | | | | | | | | | | imports e.g. test_support must do so using an absolute package name such as "import test.test_support" or "from test import test_support". This also updates the README in Lib/test, and gets rid of the duplicate data dirctory in Lib/test/data (replaced by Lib/email/test/data). Now Tim and Jack can have at it. :)
* Whitespace normalization.Tim Peters2002-04-161-9/+9
|
* Correct various errors:Martin v. Löwis2002-02-281-0/+141
- Use substring search, not re search for user-agent and paths. - Consider * entry last. Unquote, then requote URLs. - Treat empty Disallow as "allow everything". Add test cases. Fixes #523041
View Lorry Controller Status