diff options
| author | Benjamin Peterson <benjamin@python.org> | 2018-01-30 10:23:17 -0800 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2018-01-30 10:23:17 -0800 |
| commit | a23a2c555c4187f349276fe2f2ceffa953d0afe9 (patch) | |
| tree | ea9859894bb7872ee1bbae352f499c71e7e7fb52 /Lib/lib2to3/tests/support.py | |
| parent | 6b2bbcc4cca414f35f67caa4674f59f41ff638ea (diff) | |
| download | cpython-git-a23a2c555c4187f349276fe2f2ceffa953d0afe9.tar.gz | |
[3.6] closes bpo-30117: fix lib2to3 ParserIdempotency test (GH-1242) (GH-5443)
Fix two (in my opinion) spurious failure conditions in the lib2to3.tests.test_parser.TestParserIdempotency test_parser test.
Use the same encoding found in the initial file to write a temp file for a diff. This retains the BOM if the encoding was initially utf-8-sig.
If the file cannot be parsed using the normal grammar, try again with no print statement which should succeed for valid files using future print_function
For case (1), the driver was correctly handling a BOM in a utf-8 file, but then the test was not writing a comparison file using 'utf-8-sig' to diff against, so the BOM got removed. I don't think that is the fault of the parser, and lib2to3 will retain the BOM.
For case (2), lib2to3 pre-detects the use of from __future__ import print_function or allows the user to force this interpretation with a -p flag, and then selects a different grammar with the print statement removed. That makes the test cases unfair to this test as the driver itself doesn't know which grammar to use. As a minimal fix, the test will try using a grammar with the print statement, and if that fails fall back on a grammar without it. A more thorough handling of the idempotency test would to be to parse all files using both grammars and ignore if one of the two failed but otherwise check both. I didn't think this was necessary but can change..
(cherry picked from commit 14e976e00e65bf343ba0fca016c3c9132a843daf)
Diffstat (limited to 'Lib/lib2to3/tests/support.py')
| -rw-r--r-- | Lib/lib2to3/tests/support.py | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/Lib/lib2to3/tests/support.py b/Lib/lib2to3/tests/support.py index ae7cfe8ee2..fe084e8903 100644 --- a/Lib/lib2to3/tests/support.py +++ b/Lib/lib2to3/tests/support.py @@ -15,7 +15,13 @@ test_dir = os.path.dirname(__file__) proj_dir = os.path.normpath(os.path.join(test_dir, "..")) grammar_path = os.path.join(test_dir, "..", "Grammar.txt") grammar = pgen2_driver.load_grammar(grammar_path) +grammar_no_print_statement = pgen2_driver.load_grammar(grammar_path) +del grammar_no_print_statement.keywords["print"] driver = pgen2_driver.Driver(grammar, convert=pytree.convert) +driver_no_print_statement = pgen2_driver.Driver( + grammar_no_print_statement, + convert=pytree.convert +) def parse_string(string): return driver.parse_string(reformat(string), debug=True) |