index
:
delta/cpython-git.git
2.7
3.10
3.11
3.4
3.5
3.6
3.7
3.7-readme-update
3.8
3.9
Mariatta-patch-1
Mariatta-patch-2
alex-patch-1
ast-new-type-comment-null-check
backport-0613c1e-3.7
backport-10654c1-3.7
backport-109d966-3.9
backport-24bcefc-3.6
backport-24bcefc-3.7
backport-24bcefc-3.8
backport-24bd50b-2.7
backport-24bd50b-3.6
backport-283f9a2-3.6
backport-51c3e28-3.10
backport-51c3e28-3.8
backport-52d1b86-3.6
backport-52f9842-3.9
backport-5a88d50-3.8
backport-5a8d121-3.5
backport-5bdab64-3.7
backport-6ec0dec-3.9
backport-704e2d3-2.7
backport-731ff68-3.6
backport-73d6002-3.7
backport-74ae50e-2.7
backport-74ae50e-3.7
backport-76b5d71-3.9
backport-8acd0e0-3.9
backport-8ed6503-3.8
backport-916610e-3.6
backport-916610e-3.7
backport-95da310-3.8
backport-adf24bd-3.8
backport-adf24bd-3.9
backport-cd16661-2.7
backport-eeab510-2.7
backport-eeab510-3.6
backport-eeab510-3.7
backport-f7955a8-3.10
backport-f7955a8-3.9
backport-ff023ed-3.8
benjamin-eval-loop-edit
benjamin-expat-2.2.6
benjamin-external-zip
benjamin-interp-initialize
benjamin-keyword-restriction
benjamin-musl-z-workaround
benjamin-pickle-overflow
benjamin-prlimit-permission-test
benjamin-pyapi
benjamin-tanh-zero
benjamin-traceback-obo
berkerpeksag-update-codeowners
bind-socket
bpo-37741-importlib-metadata-docs-visible
bpo-39107
bpo-39235-genexp-endloc
bpo-41855-backport-fix
bpo-42057
bpo-43428/importlib-metadata-3.7
bpo-43569/man-as-machine
bpo-43780/importlib_metadata-3.10
bpo-43856/importlib-metadata-new
bpo-43926/importlib-metadata-4
bpo-45419
bpo-45514/deprecate-importlib-resources-legacy
bpo-45765/metadata-distribution-empty-path
bpo-46118/resources-package
bpo-46124/zoneinfo-resources
bpo-46125/importlib-resources-test-traversable
bpo-46126/bad-error-message
bpo-46425/restore-simple-importlib-tests
bpo-46474/entrypoint-regex
bpo-46474/entrypoint-regex-fix
branch-v3.10.0
branch-v3.9.0
bugfix/16396-linux-wintypes-importable
buildbot-custom
cjw296-patch-1
constant-kind
datetime_backport_test
dependabot/github_actions/actions/cache-2.1.7
dependabot/github_actions/actions/cache-v2.1.3
dependabot/github_actions/actions/cache-v2.1.4
dependabot/github_actions/actions/cache-v2.1.5
dependabot/github_actions/actions/upload-artifact-2.3.1
dependabot/github_actions/actions/upload-artifact-v2.2.1
dependabot/github_actions/actions/upload-artifact-v2.2.2
dependabot/github_actions/actions/upload-artifact-v2.2.3
descr-howto-feedback
enum-lost-fixes
enum-private-310
ethanfurman-acks
faster-bytes-iter
feature-version
feature/42129-resources-namespace-packages
feature/putrequest-hooks
fedora-2.7
find_module-deprecation-windows
fix-cf-feature-versiob
fix-cf-feature-version
fix-cf-feature-version-38
fix-cf-feature-version-39
fix-misc-acks
fix-misc-acks-38
fix-misc-acks-sort
fix-namedexpr-comment
fix-req-type
fix-segfaults
fix-time_ns-link
fix-tok-type-comments-flag
fix-traceback-syntax-error
fixup-bpo-34003
importlib_metadata-3.3
main
master
misc-acks-comment
mp-manager-tests
ncoghlan-bpo-33499-whats-new
ncoghlan-clarify-U-mode-deprecation
ncoghlan-fix-versionchanged-note
ncoghlan-whatsnew-38-placeholder-notes
orphaned-comment
pep572
pipelines-badge-3.6
pipelines-readme-3.7
pr_18370
pr_27789
pr_4170
print-statement
readme-update
refactor-lambda-parameters
revert-10245-msix
revert-10341-patch-7
revert-11293-bpo35565
revert-11318-fast-enum
revert-11341-bpo35603
revert-11477-backport-cbb1645-3.6
revert-11482-backport-cbb1645-2.7
revert-13569-undo-ea2b76bdc5f97f49701213d105b8ec2387ea2fa5
revert-13599-codeowner
revert-13893-fix-issue-37193
revert-14522-remove_unused_imports37
revert-15687-backport-87bd207-3.7
revert-15895-named_tuple_concept
revert-16839-backport-5a88d50-3.8
revert-17659-utf8-buffer
revert-17774-issue27973-win-fix
revert-19108-remove-cElementTree
revert-22049-bpo-1635741-signalmodule
revert-22513-update_test_codecs
revert-22997-tempfile-overhead
revert-23107-revert-13893-fix-issue-37193
revert-23398-bpo-1635741-struct
revert-23827-backport-dcc997c-3.9
revert-23828-backport-dcc997c-3.8
revert-23917-enum-prepare_kwds
revert-24821-fix/bigint_freelist
revert-25103-binary-locale
revert-25134-regen_stable_abi
revert-28650-revert-28615-test-datetime-discover
revert-28660-revert-gh28645
revert-29124-bpo-44019
revert-30472-issue-46301
revert-6121-is_integer
revert-7960-backport-2cc9d21-3.7
revert-9703-master
soft-keywords
suspcious
sync-39-whatsnew
travis-3.5
travis-3.6
travis-macOS
type-comments
unixdeepfreeze
v3.8.4
whatsnew-617
windeepfreeze
zhangyangyu-patch-1
zooba-patch-1
github.com: python/cpython.git
summary
refs
log
tree
commit
diff
log msg
author
committer
range
path:
root
/
Parser
/
tokenizer.h
Commit message (
Expand
)
Author
Age
Files
Lines
*
gh-92651: Remove the Include/token.h header file (#92652)
Victor Stinner
2022-05-11
1
-1
/
+1
*
Ensure the str member of the tokenizer is always initialised (GH-29681)
Pablo Galindo Salgado
2021-11-21
1
-1
/
+1
*
bpo-45434: Mark the PyTokenizer C API as private (GH-28924)
Victor Stinner
2021-10-13
1
-5
/
+5
*
Fix typos in the Objects directory (GH-28766)
Christian Clauss
2021-10-06
1
-1
/
+1
*
bpo-44854: Remove trailing whitespaces (GH-27689)
Serhiy Storchaka
2021-08-09
1
-1
/
+1
*
bpo-44201: Avoid side effects of "invalid_*" rules in the REPL (GH-26298)
Pablo Galindo
2021-05-22
1
-0
/
+10
*
bpo-25643: Refactor the C tokenizer into smaller, logical units (GH-25050)
Pablo Galindo
2021-03-28
1
-3
/
+2
*
bpo-43410: Fix crash in the parser when producing syntax errors when reading ...
Pablo Galindo
2021-03-14
1
-1
/
+3
*
bpo-42864: Improve error messages regarding unclosed parentheses (GH-24161)
Pablo Galindo
2021-01-19
1
-0
/
+1
*
bpo-42827: Fix crash on SyntaxError in multiline expressions (GH-24140)
Lysandros Nikolaou
2021-01-14
1
-0
/
+1
*
closes bpo-39721: Fix constness of members of tok_state struct. (GH-18600)
Andy Lester
2020-02-27
1
-5
/
+5
*
bpo-36623: Clean parser headers and include files (GH-12253)
Pablo Galindo
2019-04-13
1
-0
/
+2
*
bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)
Guido van Rossum
2019-03-07
1
-0
/
+7
*
bpo-35808: Retire pgen and use pgen2 to generate the parser (GH-11814)
Pablo Galindo
2019-03-01
1
-8
/
+0
*
bpo-35766: Merge typed_ast back into CPython (GH-11645)
Guido van Rossum
2019-01-31
1
-0
/
+2
*
bpo-16806: Fix `lineno` and `col_offset` for multi-line string tokens (GH-10021)
Anthony Sottile
2019-01-13
1
-0
/
+5
*
bpo-33306: Improve SyntaxError messages for unbalanced parentheses. (GH-6516)
Serhiy Storchaka
2018-12-17
1
-1
/
+4
*
tokenizer: Remove unused tabs options (#4422)
Victor Stinner
2017-11-17
1
-3
/
+0
*
bpo-30406: Make async and await proper keywords (#1669)
Jelle Zijlstra
2017-10-05
1
-7
/
+0
*
Remove obsolete declaration in tokenizer.h (#962)
Jim Fasarakis-Hilliard
2017-04-03
1
-2
/
+0
*
Issue #24619: Simplify async/await tokenization.
Yury Selivanov
2015-07-23
1
-15
/
+6
*
Issue #24619: New approach for tokenizing async/await.
Yury Selivanov
2015-07-22
1
-6
/
+15
*
PEP 0492 -- Coroutines with async and await syntax. Issue #24017.
Yury Selivanov
2015-05-11
1
-0
/
+7
*
Issue #1772673: The type of `char*` arguments now changed to `const char*`.
Serhiy Storchaka
2013-10-19
1
-3
/
+3
*
Issue #9319: Include the filename in "Non-UTF8 code ..." syntax error.
Victor Stinner
2011-04-05
1
-1
/
+0
*
Issue #10785: Store the filename as Unicode in the Python parser.
Victor Stinner
2011-04-05
1
-1
/
+7
*
#10222: fix for overzealous AIX compiler.
Georg Brandl
2010-10-29
1
-1
/
+1
*
Issue #9713, #10114: Parser functions (eg. PyParser_ASTFromFile) expects
Victor Stinner
2010-10-16
1
-1
/
+1
*
Issue #10095: fp_setreadl() doesn't reopen the file, reuse instead the file
Victor Stinner
2010-10-14
1
-1
/
+1
*
Recorded merge of revisions 81029 via svnmerge from
Antoine Pitrou
2010-05-09
1
-45
/
+45
*
Merged revisions 76230 via svnmerge from
Benjamin Peterson
2009-11-13
1
-2
/
+3
*
ignore the coding cookie in compile(), exec(), and eval() if the source is a ...
Benjamin Peterson
2009-03-02
1
-0
/
+1
*
Latin-1 source code was not being properly decoded when passed through
Brett Cannon
2008-10-17
1
-2
/
+2
*
Issue 1267, continued.
Guido van Rossum
2007-10-22
1
-1
/
+1
*
Patch 1267 by Christian Heimes.
Guido van Rossum
2007-10-19
1
-0
/
+1
*
Use an enum for decoding_state. It makes the code a little more
Neil Schemenauer
2007-09-21
1
-1
/
+7
*
Merged revisions 57778-58052 via svnmerge from
Thomas Wouters
2007-09-08
1
-0
/
+2
*
Convert code from sys.stdin.encoding to UTF-8 in
Martin v. Löwis
2007-09-04
1
-1
/
+2
*
Patch #1440601: Add col_offset attribute to AST nodes.
Martin v. Löwis
2006-03-01
1
-0
/
+1
*
Remove unused field.
Martin v. Löwis
2006-03-01
1
-1
/
+0
*
Constify filenames and scripts. Fixes #651362.
Martin v. Löwis
2002-12-11
1
-2
/
+2
*
Ignore encoding declarations inside strings. Fixes #603509.
Martin v. Löwis
2002-09-03
1
-0
/
+1
*
Make pgen compile with pydebug. Duplicate normalized names, as it may
Martin v. Löwis
2002-08-04
1
-0
/
+2
*
Patch #534304: Implement phase 1 of PEP 263.
Martin v. Löwis
2002-08-04
1
-0
/
+11
*
REMOVED all CWI, CNRI and BeOpen copyright markings.
Guido van Rossum
2000-09-01
1
-9
/
+0
*
Nuke all remaining occurrences of Py_PROTO and Py_FPROTO.
Tim Peters
2000-07-09
1
-5
/
+4
*
Change copyright notice - 2nd try.
Guido van Rossum
2000-06-30
1
-6
/
+0
*
Change copyright notice.
Guido van Rossum
2000-06-30
1
-21
/
+6
*
Add checking for inconsistent tab usage
Guido van Rossum
1998-04-09
1
-0
/
+6
*
Another directory quickly renamed.
Guido van Rossum
1997-04-29
1
-4
/
+5
[next]