| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(#1555)
* MySQL: Tokenize quoted schema object names, and escape characters, uniquely
Changes in this patch:
* Name.Quoted and Name.Quoted.Escape are introduced as non-standard tokens
* HTML and LaTeX formatters were confirmed to provide default formatting
if they encounter these two non-standard tokens. They also add style
classes based on the token name, like "n-Quoted" (HTML) or "nQuoted"
(LaTeX) so that users can add custom styles for these.
* Removed "\`" and "\\" as schema object name escapes. These are relics
of the previous regular expression for backtick-quoted names and are
not treated as escape sequences. The behavior was confirmed in the
MySQL documentation as well as by running queries in MySQL Workbench.
* Prevent "123abc" from being treated as an integer followed by a schema
object name. MySQL allows leading numbers in schema object names as long
as 0-9 are not the only characters in the schema object name.
* Add ~10 more unit tests to validate behavior.
Closes #1551
* Remove an end-of-line regex match that triggered a lint warning
Also, add tests that confirm correct behavior. No tests failed before
or after removing the '$' match in the regex, but now regexlint isn't
complaining.
Removing the '$' matching probably depends on the fact that Pygments
adds a newline at the end of the input text, so there is always something
after a bare integer literal.
|
|
|
* Overhaul the MySQL lexer
Fixes #975, #1063, #1453
Changes include:
Documentation
-------------
* Note in the lexer docstring that Oracle MySQL is the target syntax.
MariaDB syntax is not a target (though there is significant overlap).
Unit tests
----------
* Add 140 unit tests for MySQL.
Literals
--------
* Hexadecimal/binary/date/time/timestamp literals are supported.
* Integer mantissas are supported for scientific notation.
* In-string escapes are now tokenized properly.
* Support the "unknown" constant.
Comments
--------
* Optimizer hints are now supported, and keywords are
recognized and tokenized as preprocessor instructions.
* Remove nested multi-line comment support, which is no
longer supported in MySQL.
Variables
---------
* Support the '@' prefix for variable names.
* Lift restrictions on characters in unquoted variable names.
(MySQL does not impose a restriction on lead characters.)
* Support single/double/backtick-quoted variable names, including escapes.
* Support the '@@' prefix for system variable names.
* Support '?' as a variable so people can demonstrate prepared statements.
Keywords
--------
* Keyword / data type / function are now in a separate, auto-updating file.
* Support 25 additional data types (including spatial and JSON types).
* Support 460 additional MySQL keywords.
* Support 372 MySQL functions.
Explicit function support resolves a bug that causes non-function
items to be treated as functions simply because they have a trailing
opening parenthesis.
* Support exceptions for the 'SET' keyword, which is both a datatype and
a keyword depending on context.
Schema object names
-------------------
* Support Unicode in MySQL schema object names.
* Support parsing of backtick-quoted schema object name escapes.
(Escapes do not produce a distinct token type at this time.)
Operators
---------
* Remove non-operator characters from the list of operators.
* Remove non-punctuation characters from the list of punctuation.
* Cleanup items based on feedback
* Remove an unnecessary optional newline lookahead for single-line comments
|
