summaryrefslogtreecommitdiff
path: root/Parser
Commit message (Collapse)AuthorAgeFilesLines
* Merge remote-tracking branch 'origin/master' into refactor-lambda-parametersrefactor-lambda-parametersGuido van Rossum2020-04-304-794/+1323
|\
| * bpo-40334: Add support for feature_version in new PEG parser (GH-19827)Lysandros Nikolaou2020-04-304-799/+1328
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `ast.parse` and `compile` support a `feature_version` parameter that tells the parser to parse the input string, as if it were written in an older Python version. The `feature_version` is propagated to the tokenizer, which uses it to handle the three different stages of support for `async` and `await`. Additionally, it disallows the following at parser level: - The '@' operator in < 3.5 - Async functions in < 3.5 - Async comprehensions in < 3.6 - Underscores in numeric literals in < 3.6 - Await expression in < 3.5 - Variable annotations in < 3.6 - Async for-loops in < 3.5 - Async with-statements in < 3.5 - F-strings in < 3.6 Closes we-like-parsers/cpython#124.
* | Fix an oversight (kwds -> lambda_kwds)Guido van Rossum2020-04-301-6/+6
| |
* | Refactor lambda_parameters similar to parametersGuido van Rossum2020-04-301-1030/+1118
|/
* bpo-40334: Support type comments (GH-19780)Guido van Rossum2020-04-303-1410/+2483
| | | | | | | | | | This implements full support for # type: <type> comments, # type: ignore <stuff> comments, and the func_type parsing mode for ast.parse() and compile(). Closes https://github.com/we-like-parsers/cpython/issues/95. (For now, you need to use the master branch of mypy, since another issue unique to 3.9 had to be fixed there, and there's no mypy release yet.) The only thing missing is `feature_version=N`, which is being tracked in https://github.com/we-like-parsers/cpython/issues/124.
* bpo-40334: refactor and cleanup for the PEG generators (GH-19775)Pablo Galindo2020-04-293-70/+23
|
* bpo-40334: Disallow invalid single statements in the new parser (GH-19774)Lysandros Nikolaou2020-04-291-0/+51
| | | | | After parsing is done in single statement mode, the tokenizer buffer has to be checked for additional lines and a `SyntaxError` must be raised, in case there are any. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40334: Explicitly cast to int in pegen.c to fix a compiler warning ↵Pablo Galindo2020-04-291-4/+4
| | | | (GH-19779)
* bpo-40334: Fix shifting of nested f-strings in the new parser (GH-19771)Lysandros Nikolaou2020-04-291-0/+9
| | | `JoinedStr`s and `FormattedValue also needs to be shifted, in order to correctly compute the location information of nested f-strings.
* bpo-40334: Catch E_EOF error, when the tokenizer returns ERRORTOKEN (GH-19743)Lysandros Nikolaou2020-04-281-3/+6
| | | An E_EOF error was only being caught after the parser exited before this commit. There are some cases though, where the tokenizer returns ERRORTOKEN *and* has set an E_EOF error (like when EOF directly follows a line continuation character) which weren't correctly handled before.
* bpo-40334: Support PyPARSE_DONT_IMPLY_DEDENT in the new parser (GH-19736)Pablo Galindo2020-04-271-1/+1
|
* bpo-40334: Support CO_FUTURE_BARRY_AS_BDFL in the new parser (GH-19721)Pablo Galindo2020-04-275-309/+401
| | | This commit also allows to pass flags to the new parser in all interfaces and fixes a bug in the parser generator that was causing to inline rules with actions, making them disappear.
* Use Py_ssize_t instead of ssize_t (GH-19685)Pablo Galindo2020-04-241-1/+1
|
* bpo-40334: Improve various PEG-Parser related stuff (GH-19669)Lysandros Nikolaou2020-04-232-28/+50
| | | The changes in this commit are all related to @vstinner's original review comments of the initial PEP 617 implementation PR.
* bpo-40334: Fix build errors and warnings in test_peg_generator (GH-19672)Pablo Galindo2020-04-232-1/+12
|
* bpo-40334: Don't downcast from Py_ssize_t to int (GH-19671)Pablo Galindo2020-04-231-21/+21
|
* Compile extensions in test_peg_generator with C99 (GH-19668)Pablo Galindo2020-04-231-14/+10
|
* bpo-40334: Fix errors in parse_string.c with old compilers (GH-19666)Pablo Galindo2020-04-231-10/+14
|
* bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)Pablo Galindo2020-04-226-0/+19002
| | | | Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* bpo-40335: Correctly handle multi-line strings in tokenize error scenarios ↵Pablo Galindo2020-04-212-22/+26
| | | | | (GH-19619) Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
* bpo-40267: Fix message when last input character produces a SyntaxError ↵Lysandros Nikolaou2020-04-151-0/+3
| | | | | | | | (GH-19521) When there is a SyntaxError after reading the last input character from the tokenizer and if no newline follows it, the error message used to be `unexpected EOF while parsing`, which is wrong.
* bpo-40268: Remove unused structmember.h includes (GH-19530)Victor Stinner2020-04-151-1/+1
| | | | | | If only offsetof() is needed: include stddef.h instead. When structmember.h is used, add a comment explaining that PyMemberDef is used.
* bpo-40268: Remove explicit pythread.h includes (#19529)Victor Stinner2020-04-151-1/+0
| | | | Remove explicit pythread.h includes: it is always included by Python.h.
* bpo-40268: Remove a few pycore_pystate.h includes (GH-19510)Victor Stinner2020-04-142-5/+5
|
* bpo-40268: Rename _PyInterpreterState_GET_UNSAFE() (GH-19509)Victor Stinner2020-04-141-3/+3
| | | | | | | Rename _PyInterpreterState_GET_UNSAFE() to _PyInterpreterState_GET() for consistency with _PyThreadState_GET() and to have a shorter name (help to fit into 80 columns). Add also "assert(tstate != NULL);" to the function.
* bpo-40268: Include explicitly pycore_interp.h (GH-19505)Victor Stinner2020-04-141-1/+2
| | | | pycore_pystate.h no longer includes pycore_interp.h: it's now included explicitly in files accessing PyInterpreterState.
* bpo-40246: Report a better error message for invalid string prefixes (GH-19476)Lysandros Nikolaou2020-04-121-0/+4
|
* bpo-40141: Add line and column information to ast.keyword nodes (GH-19283)Pablo Galindo2020-04-021-0/+1
|
* bpo-40020: Fix realloc leak on failure in growable_comment_array_add (GH-19083)Alexander Riccio2020-03-301-3/+5
| | | | | Fix a leak and subsequent crash in parsetok.c caused by realloc misuse on a rare codepath. Realloc returns a null pointer on failure, and then growable_comment_array_deallocate crashes later when it dereferences it.
* bpo-39882: Add _Py_FatalErrorFormat() function (GH-19157)Victor Stinner2020-03-252-2/+2
|
* bpo-39999: Improve compatibility of the ast module. (GH-19056)Serhiy Storchaka2020-03-221-3/+3
| | | | | | | * Re-add removed classes Suite, slice, Param, AugLoad and AugStore. * Add docstrings for dummy classes. * Add docstrings for attribute aliases. * Set __module__ to "ast" instead of "_ast".
* bpo-39988: Remove ast.AugLoad and ast.AugStore node classes. (GH-19038)Serhiy Storchaka2020-03-171-1/+1
|
* bpo-39638: Keep ASDL signatures in the AST nodes (GH-18515)Batuhan Taşkaya2020-03-162-8/+47
|
* bpo-39969: Remove ast.Param node class as is no longer used (GH-19020)Batuhan Taşkaya2020-03-151-1/+1
|
* bpo-34822: Simplify AST for subscription. (GH-9605)Serhiy Storchaka2020-03-101-5/+4
| | | | | | | | | * Remove the slice type. * Make Slice a kind of the expr type instead of the slice type. * Replace ExtSlice(slices) with Tuple(slices, Load()). * Replace Index(value) with a value itself. All non-terminal nodes in AST for expressions are now of the expr type.
* bpo-36287: Make ast.dump() not output optional fields and attributes with ↵Serhiy Storchaka2020-03-101-0/+11
| | | | | | | default values. (GH-18843) The default values for optional fields and attributes of AST nodes are now set as class attributes (e.g. Constant.kind is set to None).
* Fix typo in the parser generator (GH-18603)xatier2020-03-091-1/+1
|
* bpo-39882: Py_FatalError() logs the function name (GH-18819)Victor Stinner2020-03-072-5/+8
| | | | | | | | | | | | The Py_FatalError() function is replaced with a macro which logs automatically the name of the current function, unless the Py_LIMITED_API macro is defined. Changes: * Add _Py_FatalErrorFunc() function. * Remove the function name from the message of Py_FatalError() calls which included the function name. * Update tests.
* bpo-39639: Remove the AST "Suite" node and associated code (GH-18513)Batuhan Taşkaya2020-03-041-5/+1
| | | | | | | | The AST "Suite" node is no longer used and it can be removed from the ASDL definition and related structures (compiler, visitors, ...). Co-Authored-By: Victor Stinner <vstinner@python.org> Co-authored-by: Brett Cannon <54418+brettcannon@users.noreply.github.com> Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* closes bpo-39721: Fix constness of members of tok_state struct. (GH-18600)Andy Lester2020-02-273-27/+37
| | | | | | | | | | | | | | | | | | | | | The function PyTokenizer_FromUTF8 from Parser/tokenizer.c had a comment: /* XXX: constify members. */ This patch addresses that. In the tok_state struct: * end and start were non-const but could be made const * str and input were const but should have been non-const Changes to support this include: * decode_str() now returns a char * since it is allocated. * PyTokenizer_FromString() and PyTokenizer_FromUTF8() each creates a new char * for an allocate string instead of reusing the input const char *. * PyTokenizer_Get() and tok_get() now take const char ** arguments. * Various local vars are const or non-const accordingly. I was able to remove five casts that cast away constness.
* bpo-39219: Fix SyntaxError attributes in the tokenizer. (GH-17828)Serhiy Storchaka2020-02-121-4/+32
| | | | * Always set the text attribute. * Correct the offset attribute for non-ascii sources.
* bpo-39500: Document PyUnicode_IsIdentifier() function (GH-18397)Victor Stinner2020-02-111-1/+2
| | | | PyUnicode_IsIdentifier() does not call Py_FatalError() anymore if the string is not ready.
* bpo-38823: Fix refleaks in _ast initialization error path (GH-17276)Brandt Bucher2020-02-061-8/+18
|
* Allow pgen to produce a DOT format dump of the grammar (GH-18005)Pablo Galindo2020-01-143-2/+46
| | | Originally suggested by Anthony Shaw.
* Remove unused functions in Parser/parsetok.c (GH-17365)Emmanuel Arias2020-01-131-18/+0
|
* bpo-39307: Fix memory leak on error path in parsetok (GH-17953)Alex Henrie2020-01-131-0/+1
|
* bpo-39209: Manage correctly multi-line tokens in interactive mode (GH-17860)Pablo Galindo2020-01-061-0/+2
|
* bpo-36500: Simplify PCbuild/build.bat and prevent path separator changing in ↵Steve Dower2019-12-171-1/+2
| | | | comments (GH-17644)
* bpo-38673: dont switch to ps2 if the line starts with comment or whitespace ↵Batuhan Taşkaya2019-12-081-0/+6
| | | | | (GH-17421) https://bugs.python.org/issue38673
* bpo-36876: Moved Parser/listnode.c statics to interpreter state. (GH-16328)Vinay Sajip2019-11-071-10/+14
|
View Lorry Controller Status