| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove unused variable
This variable is unused since the first commit to this file
in the commit history, so it's probably safe to remove it.
* Lex identifier as label only if it's at line start
* Stop matching identifiers that begin with a digit
This is so we don't match numbers as labels with the new labels rule.
* Add label tests
* Fix existing tests
|
| |
|
|
| |
The CFamilyLexer was matching whitespace as Text instead of Whitespace.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
CFamilyLexer failed to tokenize preprocessor macros when they were
preceded by line break surrounded by spaces. This was the case because
prerpocessor regex rule expected to start at the beginning of the line,
but the space regex rule matched also the whitespace after the line
break. Now the space rule has been refined not to match the line break.
Because of this, the preprocessor regex rule correctly matches
prerpocessor tokens even when they are preceded by white spaces, at the
cost of adding some more tokens in the token stream in some cases. This
change preserves the behavior of invalid preprocessor usage failing to
tokenize.
|
|
|
Co-authored-by: Georg Brandl <georg@python.org>
|
