<feed xmlns='http://www.w3.org/2005/Atom'>
<title>delta/cpython-git.git/Modules/parsermodule.c, branch v3.8.9</title>
<subtitle>github.com: python/cpython.git
</subtitle>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/'/>
<entry>
<title>bpo-37253: Add _PyCompilerFlags_INIT macro (GH-14018)</title>
<updated>2019-06-13T00:36:03+00:00</updated>
<author>
<name>Miss Islington (bot)</name>
<email>31488909+miss-islington@users.noreply.github.com</email>
</author>
<published>2019-06-13T00:36:03+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=92e836c7dcaf74f7b8617250414224d24d1eb1f2'/>
<id>92e836c7dcaf74f7b8617250414224d24d1eb1f2</id>
<content type='text'>
Add a new _PyCompilerFlags_INIT macro to initialize PyCompilerFlags
variables, rather than initializing cf_flags and cf_feature_version
explicitly in each variable.
(cherry picked from commit 37d66d7d4bc7dbac9809d69966a774ebb32563be)

Co-authored-by: Victor Stinner &lt;vstinner@redhat.com&gt;</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Add a new _PyCompilerFlags_INIT macro to initialize PyCompilerFlags
variables, rather than initializing cf_flags and cf_feature_version
explicitly in each variable.
(cherry picked from commit 37d66d7d4bc7dbac9809d69966a774ebb32563be)

Co-authored-by: Victor Stinner &lt;vstinner@redhat.com&gt;</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-36974: tp_print -&gt; tp_vectorcall_offset and tp_reserved -&gt; tp_as_async (GH-13464)</title>
<updated>2019-05-31T02:13:39+00:00</updated>
<author>
<name>Jeroen Demeyer</name>
<email>J.Demeyer@UGent.be</email>
</author>
<published>2019-05-31T02:13:39+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=530f506ac91338b55cf2be71b1cdf50cb077512f'/>
<id>530f506ac91338b55cf2be71b1cdf50cb077512f</id>
<content type='text'>
Automatically replace
tp_print -&gt; tp_vectorcall_offset
tp_compare -&gt; tp_as_async
tp_reserved -&gt; tp_as_async
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Automatically replace
tp_print -&gt; tp_vectorcall_offset
tp_compare -&gt; tp_as_async
tp_reserved -&gt; tp_as_async
</pre>
</div>
</content>
</entry>
<entry>
<title>fix warnings by adding more const (GH-12924)</title>
<updated>2019-04-23T11:39:37+00:00</updated>
<author>
<name>Inada Naoki</name>
<email>songofacandy@gmail.com</email>
</author>
<published>2019-04-23T11:39:37+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=09415ff0ebbbe97c8cd08ac1f94673d7a49c8018'/>
<id>09415ff0ebbbe97c8cd08ac1f94673d7a49c8018</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-36440: include node names in ParserError messages, instead of numeric IDs (GH-12565)</title>
<updated>2019-04-03T05:12:07+00:00</updated>
<author>
<name>tyomitch</name>
<email>tyomitch@gmail.com</email>
</author>
<published>2019-04-03T05:12:07+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=cb0748d3939c31168ab5d3b80e3677494497d5e3'/>
<id>cb0748d3939c31168ab5d3b80e3677494497d5e3</id>
<content type='text'>
The error messages in the parser module are referring to numeric IDs for the nodes. To improve readability, use the node names when reporting errors.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The error messages in the parser module are referring to numeric IDs for the nodes. To improve readability, use the node names when reporting errors.</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-36256: Fix bug in parsermodule when parsing if statements (GH-12477)</title>
<updated>2019-03-21T23:33:02+00:00</updated>
<author>
<name>Pablo Galindo</name>
<email>Pablogsal@gmail.com</email>
</author>
<published>2019-03-21T23:33:02+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=9a0000d15d27361eaa47b77600c7c00a9787a894'/>
<id>9a0000d15d27361eaa47b77600c7c00a9787a894</id>
<content type='text'>
bpo-36256: Fix bug in parsermodule when parsing if statements

In the parser module, when validating nodes before starting the parsing with to create a ST in "parser_newstobject" there is a problem that appears when two arcs in the same DFA state has transitions with labels with the same type. For example, the DFA for if_stmt has a state with
two labels with the same type: "elif" and "else" (type NAME). The algorithm tries one by one the arcs until the label that starts the arc transition has a label with the same type of the current child label we are trying to accept. In this case, the arc for "elif" comes before the arc for "else"and passes this test (because the current child label is "else" and has the same type as "elif"). This lead to expecting a namedexpr_test (305) instead of a colon (11). The solution is to compare also the string representation (in case there is one) of the labels to see if the transition that we have is the correct one.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
bpo-36256: Fix bug in parsermodule when parsing if statements

In the parser module, when validating nodes before starting the parsing with to create a ST in "parser_newstobject" there is a problem that appears when two arcs in the same DFA state has transitions with labels with the same type. For example, the DFA for if_stmt has a state with
two labels with the same type: "elif" and "else" (type NAME). The algorithm tries one by one the arcs until the label that starts the arc transition has a label with the same type of the current child label we are trying to accept. In this case, the arc for "elif" comes before the arc for "else"and passes this test (because the current child label is "else" and has the same type as "elif"). This lead to expecting a namedexpr_test (305) instead of a colon (11). The solution is to compare also the string representation (in case there is one) of the labels to see if the transition that we have is the correct one.</pre>
</div>
</content>
</entry>
<entry>
<title>Remove d_initial from the parser as it is unused (GH-12212)</title>
<updated>2019-03-09T15:35:50+00:00</updated>
<author>
<name>tyomitch</name>
<email>tyomitch@gmail.com</email>
</author>
<published>2019-03-09T15:35:50+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=1b304f992ddfc1cc40758dd633bc6a2595399189'/>
<id>1b304f992ddfc1cc40758dd633bc6a2595399189</id>
<content type='text'>
d_initial, the first state of a particular DFA in the parser has always been initialized to 0 in the old pgen as well as the new pgen. As this value is not used and the first state of each DFA is assumed to be the first element in the array representing it, remove d_initial from the parser to reduce complexity.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
d_initial, the first state of a particular DFA in the parser has always been initialized to 0 in the old pgen as well as the new pgen. As this value is not used and the first state of each DFA is assumed to be the first element in the array representing it, remove d_initial from the parser to reduce complexity.</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)</title>
<updated>2019-03-07T20:38:08+00:00</updated>
<author>
<name>Guido van Rossum</name>
<email>guido@python.org</email>
</author>
<published>2019-03-07T20:38:08+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=495da292255b92dd73758fdd0e4c7d27d82b1e57'/>
<id>495da292255b92dd73758fdd0e4c7d27d82b1e57</id>
<content type='text'>
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)



https://bugs.python.org/issue35975</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)



https://bugs.python.org/issue35975</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-35766: Merge typed_ast back into CPython (GH-11645)</title>
<updated>2019-01-31T11:40:27+00:00</updated>
<author>
<name>Guido van Rossum</name>
<email>guido@python.org</email>
</author>
<published>2019-01-31T11:40:27+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=dcfcd146f8e6fc5c2fc16a4c192a0c5f5ca8c53c'/>
<id>dcfcd146f8e6fc5c2fc16a4c192a0c5f5ca8c53c</id>
<content type='text'>
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-33416: Add end positions to Python AST (GH-11605)</title>
<updated>2019-01-22T11:18:22+00:00</updated>
<author>
<name>Ivan Levkivskyi</name>
<email>levkivskyi@gmail.com</email>
</author>
<published>2019-01-22T11:18:22+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=9932a22897ef9905161dac7476e6976370e13515'/>
<id>9932a22897ef9905161dac7476e6976370e13515</id>
<content type='text'>
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST.  Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
  ```python
  class C:
      pass

  pass
  ```
  the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.

An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points:
* It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`.
* I add end position information to both CST and AST.  Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient.
* Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear.
* For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in
  ```python
  class C:
      pass

  pass
  ```
  the end line and end column for the class definition is (2, 8).
* For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node.
* I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing.

An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.</pre>
</div>
</content>
</entry>
<entry>
<title>bpo-33012: Fix invalid function cast warnings with gcc 8. (GH-6749)</title>
<updated>2018-11-27T11:27:31+00:00</updated>
<author>
<name>Serhiy Storchaka</name>
<email>storchaka@gmail.com</email>
</author>
<published>2018-11-27T11:27:31+00:00</published>
<link rel='alternate' type='text/html' href='http://git.baserock.org/cgit/delta/cpython-git.git/commit/?id=62be74290aca26d16f3f55ece7ff6dad14e60e8d'/>
<id>62be74290aca26d16f3f55ece7ff6dad14e60e8d</id>
<content type='text'>
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
</content>
<content type='xhtml'>
<div xmlns='http://www.w3.org/1999/xhtml'>
<pre>
Fix invalid function cast warnings with gcc 8
for method conventions different from METH_NOARGS, METH_O and
METH_VARARGS excluding Argument Clinic generated code.
</pre>
</div>
</content>
</entry>
</feed>
