diff options
author | Akim Demaille <akim.demaille@gmail.com> | 2020-04-17 08:05:40 +0200 |
---|---|---|
committer | Akim Demaille <akim.demaille@gmail.com> | 2020-04-17 08:53:37 +0200 |
commit | e86b14069dc00ee4a8c234b976ce8c68c8f7081a (patch) | |
tree | 95dd71ca5195006afce530854566c71af640415b | |
parent | 5d983253f7106fe835953c7bbda15ca90247577b (diff) | |
download | bison-e86b14069dc00ee4a8c234b976ce8c68c8f7081a.tar.gz |
doc: token_kind_type in C++
* data/skeletons/c++.m4: Define the old names in terms on the new
ones, instead of the converse.
* doc/bison.texi (C++ Parser Interface): Be more extensive about
token_kind_type.
-rw-r--r-- | TODO | 7 | ||||
-rw-r--r-- | data/skeletons/c++.m4 | 4 | ||||
-rw-r--r-- | doc/bison.texi | 75 |
3 files changed, 58 insertions, 28 deletions
@@ -17,7 +17,7 @@ ** Documentation - YYERRCODE, YYUNDEF, YYEOF - symbol.type_get should be kind_get, and it's not documented. -- YYERRCODE and "end of file" and translation +- YYERRCODE and translation ** Java *** Examples @@ -35,9 +35,8 @@ breaks. token vs terminal, variable vs non terminal. ** api.token.raw -Maybe we should exhibit the YYUNDEFTOK token. It could also be assigned a -semantic value so that yyerror could be used to report invalid lexemes. -See also the item "$undefined" below. +The YYUNDEFTOK could be assigned a semantic value so that yyerror could be +used to report invalid lexemes. See also the item "$undefined" below. ** push parsers Consider deprecating impure push parsers. They add a lot of complexity, for diff --git a/data/skeletons/c++.m4 b/data/skeletons/c++.m4 index 6617ec3a..d5f94d15 100644 --- a/data/skeletons/c++.m4 +++ b/data/skeletons/c++.m4 @@ -171,7 +171,7 @@ m4_bpatsubst(m4_dquote(m4_bpatsubst(m4_dquote(b4_namespace_ref[ ]), # -------------- # Output the definition of the token kinds. m4_define([b4_token_enums], -[[enum yytokentype +[[enum token_kind_type { ]b4_symbol([-2], [id])[ = -2, ]b4_symbol_foreach([b4_token_enum])dnl @@ -259,6 +259,8 @@ m4_define([b4_public_types_declare], struct token { ]b4_token_enums[ + /// Backward compatibility alias. + typedef token_kind_type yytokentype; }; /// Token kind, as returned by yylex. diff --git a/doc/bison.texi b/doc/bison.texi index 467799b4..f9794ecf 100644 --- a/doc/bison.texi +++ b/doc/bison.texi @@ -6234,8 +6234,8 @@ introduced in Bison 3.0. @c ================================================== api.token.prefix +@anchor{api-token-prefix} @deffn Directive {%define api.token.prefix} @{@var{prefix}@} - @itemize @item Languages(s): all @@ -7463,7 +7463,7 @@ An opaque type that captures the circumstances of the syntax error. @end deffn @deffn {Type} yysymbol_kind_t -An enum that includes all the grammar symbols, tokens and nonterminals. Its +An enum of all the grammar symbols, tokens and nonterminals. Its enumerators are forged from the symbol names: @example @@ -11523,6 +11523,43 @@ extended using the @code{%parse-param} feature: its semantics is slightly changed since it describes an additional member of the parser class, and an additional argument for its constructor. + +@defcv {Type} {parser} {token} +A structure that contains (only) the @code{token_kind_type} enumeration, +which defines the tokens. To refer to the token @code{FOO}, use +@code{yy::parser::token::FOO}. The scanner can use @samp{typedef +yy::parser::token token;} to ``import'' the token enumeration (@pxref{Calc++ +Scanner}). +@end defcv + +@defcv {Type} {parser} {token_kind_type} +An enumeration of the token kinds. Its enumerators are forged from the +token names, with a possible token prefix +(@pxref{api-token-prefix,,@code{api.token.prefix}}): + +@example +/// Token kinds. +struct token +@{ + enum token_kind_type + @{ + YYEMPTY = -2, // No token. + YYEOF = 0, // "end of file" + YYERRCODE = 256, // error + YYUNDEF = 257, // "invalid token" + PLUS = 258, // "+" + MINUS = 259, // "-" + [...] + VAR = 271, // "variable" + NEG = 272 // NEG + @}; +@}; + +/// Token kind, as returned by yylex. +typedef token::token_kind_type token_kind_type; +@end example +@end defcv + @defcv {Type} {parser} {semantic_type} The types for semantic values. @xref{C++ Semantic Values}. @end defcv @@ -11532,14 +11569,6 @@ The type of locations, if location tracking is enabled. @xref{C++ Location Values}. @end defcv -@defcv {Type} {parser} {token} -A structure that contains (only) the @code{yytoken_kind_t} enumeration, -which defines the tokens. To refer to the token @code{FOO}, use -@code{yy::parser::token::FOO}. The scanner can use @samp{typedef -yy::parser::token token;} to ``import'' the token enumeration (@pxref{Calc++ -Scanner}). -@end defcv - @defcv {Type} {parser} {syntax_error} This class derives from @code{std::runtime_error}. Throw instances of it from the scanner or from the actions to raise parse errors. This is @@ -11988,7 +12017,7 @@ A type that captures the circumstances of the syntax error. @end defcv @defcv {Type} {parser} {symbol_kind_type} -An enum that includes all the grammar symbols, tokens and nonterminals. Its +An enum of all the grammar symbols, tokens and nonterminals. Its enumerators are forged from the symbol names: @example @@ -13145,7 +13174,7 @@ The parser context provides information to build error reports when you invoke @samp{%define parse.error custom}. @defcv {Type} {YYParser} {SymbolKind} -An enum that includes all the grammar symbols, tokens and nonterminals. Its +An enum of all the grammar symbols, tokens and nonterminals. Its enumerators are forged from the symbol names: @example @@ -14713,18 +14742,18 @@ Data type of semantic values; @code{int} by default. @end deffn @deffn {Type} yysymbol_kind_t -An enum that includes all the symbols, tokens and nonterminals, of the -grammar. @xref{Syntax Error Reporting Function}. The symbol kinds are used +An enum of all the symbols, tokens and nonterminals, of the grammar. +@xref{Syntax Error Reporting Function}. The symbol kinds are used internally by the parser, and should not be confused with the token kinds: the symbol kind of a terminal symbol is not equal to its token kind! (Unless -@samp{%define api.token.raw} was used). +@samp{%define api.token.raw} was used.) @end deffn @deffn {Type} yytoken_kind_t -An enum that includes all the declared @dfn{token kinds} declared with -@code{%token} (@pxref{Token Decl}). These are the return values for -@code{yylex}. They should not be confused with the @emph{symbol kinds}, -used internally by the parser. +An enum of all the @dfn{token kinds} declared with @code{%token} +(@pxref{Token Decl}). These are the return values for @code{yylex}. They +should not be confused with the @emph{symbol kinds}, used internally by the +parser. @end deffn @@ -14925,8 +14954,8 @@ first nonterminal symbol in a language specification. @xref{Start Decl}. @item Symbol kind -A finite enumeration of all the possible grammar symbols, as processed by -the parser. @xref{Symbols}. +A (finite) enumeration of the grammar symbols, as processed by the parser. +@xref{Symbols}. @item Symbol table A data structure where symbol names and associated data are stored during @@ -14949,8 +14978,8 @@ Bison parser is a stream of tokens which comes from the lexical analyzer. @xref{Symbols}. @item Token kind -A finite enumeration of all the possible grammar terminals, as discriminated -by the scanner. @xref{Symbols}. +A (finite) enumeration of the grammar terminals, as discriminated by the +scanner. @xref{Symbols}. @item Unreachable state A parser state to which there does not exist a sequence of transitions from |