summaryrefslogtreecommitdiff
path: root/parse.y
Commit message (Collapse)AuthorAgeFilesLines
* Introduce anddot_multiple_assignment_check functionS-H-GAMELINKS2023-05-141-6/+10
|
* [Bug #19619] Preserve numbered parameters contextNobuyoshi Nakada2023-05-021-3/+23
| | | | Preserve numbered parameters context across method definitions
* Handle private AREF call in compile.cNobuyoshi Nakada2023-04-301-4/+1
|
* Get rid of a breakpoint left in parse.yTakashi Kokubun2023-04-101-1/+0
|
* [Bug #19563] Yield words separators per linesNobuyoshi Nakada2023-04-071-10/+15
| | | | | | So that newlines across a here-doc terminator will be separated tokens. Cf. https://github.com/ruby/irb/pull/558
* `*` in an array pattern should not be parsed as nil in ripperKazuki Tsujimoto2023-04-011-1/+1
| | | | | | | | | | | | | | | After 6c0925ba7017efde6091e2ec4f1a6be268166696, it was impossible to distinguish between the presence or absence of `*`. # Before the commit Ripper.sexp('0 in []')[1][0][2][1] #=> [:aryptn, nil, nil, nil, nil] Ripper.sexp('0 in [*]')[1][0][2][1] #=> [:aryptn, nil, nil, [:var_field, nil], nil] # After the commit Ripper.sexp('0 in []')[1][0][2][1] #=> [:aryptn, nil, nil, nil, nil] Ripper.sexp('0 in [*]')[1][0][2][1] #=> [:aryptn, nil, nil, nil, nil] This commit reverts it.
* Fix gc_verify_internal_consistency error for pattern_matching in ripperyui-knk2023-03-311-21/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `gc_verify_internal_consistency` reports "found internal inconsistency" for "test_pattern_matching.rb". http://ci.rvm.jp/results/trunk-gc-asserts@ruby-sp2-docker/4501173 Ruby's parser manages objects by two different ways. 1. For parser * markable node holds objects * call `RB_OBJ_WRITTEN` with `p->ast` as parent * `mark_ast_value` marks objects 2. For ripper * unmarkable node, NODE_RIPPER/NODE_CDECL, holds objects * call `rb_ast_add_mark_object`. This function calls `rb_hash_aset` then `RB_OBJ_WRITTEN` is called with `mark_hash` as parent * `mark_hash` marks objects However in current pattern_matching implementation * markable node holds objects * call `rb_ast_add_mark_object` This commit fix it to be #2. This was inconsistency however always `mark_hash` is made young by `rb_ast_add_mark_object` call then objects are not collected.
* [Bug #19547] Add token for unescaped backslashNobuyoshi Nakada2023-03-301-0/+1
| | | | | This token is exposed only when `RubyVM::AbstractSyntaxTree` with `keep_tokens` option.
* [Bug #19175] p_kw without a sub pattern should be `assignable'Kazuki Tsujimoto2023-03-261-1/+1
|
* [Bug #19175] p_rest should be `assignable'Kazuki Tsujimoto2023-03-261-33/+26
| | | | It should also check for duplicate names.
* [Bug #19539] Match heredoc identifier from end of lineNobuyoshi Nakada2023-03-191-10/+10
| | | | Not to ignore leading spaces in indented heredoc identifier.
* Expand tabs in parse.yTakashi Kokubun2023-03-091-7361/+7361
| | | | | I used the same script as https://github.com/ruby/ruby/pull/6094 but for a .y file.
* Suppress -Wunused-but-set-variable warningNobuyoshi Nakada2023-02-141-0/+1
|
* Extract body rules from endless method definitionsNobuyoshi Nakada2023-02-011-64/+36
|
* Remove not used argument from tokenize_identyui-knk2023-01-251-4/+4
| | | | This has not been used since 5e59be3eddd2ce28610ec8d1b42845cde0080833
* Fix format specifiers for pointer differencesNobuyoshi Nakada2023-01-071-3/+16
|
* [Bug #19312] Return end-of-input at `__END__`Nobuyoshi Nakada2023-01-061-7/+5
|
* [Bug #19291] Rewind to the previous lineNobuyoshi Nakada2023-01-021-4/+13
| | | | | | When rewinding looking ahead after newline token, also reset the last line string, the pointers to it, and the location, not only the line number.
* EXPR_DOT is set when next token is tANDDOT ("&.") [ci skip]yui-knk2022-12-261-1/+1
|
* Disallow mixed usage of ... and */**Shugo Maeda2022-12-151-6/+26
| | | | [Feature #19134]
* [Bug #19195] Allow optional newlines before closing parenthesisNobuyoshi Nakada2022-12-131-1/+1
|
* Use loc to fix a failure of test_ast.rbShugo Maeda2022-12-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | ``` 1) Failure: TestAst#test_ranges:test/fiber/scheduler.rb [/home/runner/work/ruby/ruby/src/test/ruby/test_ast.rb:122]: <[]> expected but was <[{:type=>:max_validation_error, :max=> #<RubyVM::AbstractSyntaxTree::Node::CodePosition:0x00007f80d630b598 @column=20, @lineno=203>, :end_pos=> #<RubyVM::AbstractSyntaxTree::Node::CodePosition:0x00007f80d630b778 @column=19, @lineno=203>, :node=> (BLOCK_PASS@203:15-203:19 (ARGSPUSH@203:15-203:20 (SPLAT@203:16-203:19 (LVAR@203:16-203:19 :*)) (HASH@203:16-203:19 (LIST@203:16-203:19 nil (LVAR@203:16-203:19 :**) nil))) (LVAR@203:16-203:19 :&))}]>. ```
* Should use argsloc for the last argument for arg_append()Shugo Maeda2022-12-051-2/+2
|
* Introduce encoding check macroS-H-GAMELINKS2022-12-021-3/+4
|
* Remove ruby2_keywords related to args forwardingyui-knk2022-11-291-2/+0
| | | | | | | This was introduced by b609bdeb5307e280137b4b2838af0fe4e4b46f1c to suppress warnings. However these warngins were deleted by beae6cbf0fd8b6619e5212552de98022d4c4d4d4. Therefore these codes are not needed anymore.
* Use idFWD_* instead of ANON_*_IDShugo Maeda2022-11-291-18/+14
|
* Allow ** in def foo(...)Shugo Maeda2022-11-291-16/+4
| | | | [Feature #19134]
* Fix the position of rescue clause without exc_listJeremy Evans2022-11-241-1/+10
| | | | | | | | | | | If the rescue clause has only exc_var and not exc_list, use the exc_var position instead of the rescue body position. This issue appears to have been introduced in 688169fd83b24564b653c03977c168cea50ccd35 when "opt_list" was split into "exc_list exc_var". Fixes [Bug #18974]
* Refactor to use has_delayed_token macroyui-knk2022-11-211-1/+1
|
* Enhance keep_tokens option for RubyVM::AbstractSyntaxTree parsing methodsyui-knk2022-11-211-61/+419
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implementation for Language Server Protocol (LSP) sometimes needs token information. For example both `m(1)` and `m(1, )` has same AST structure other than node locations then it's impossible to check the existence of `,` from AST. However in later case, it might be better to suggest variables list for the second argument. Token information is important for such case. This commit adds these methods. * Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of` * Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes. * Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node. [Feature #19070] Impacts on memory usage and performance are below: Memory usage: ``` $ cat test.rb root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true) $ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] 11408kb # keep_tokens :false $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 17508kb # keep_tokens :true $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 30960kb ``` Performance: ``` $ cat ../ast_keep_tokens.yml prelude: | src = <<~SRC module M class C def m1(a, b) 1 + a + b end end end SRC benchmark: without_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false) with_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true) $ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml /home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \ --executables="compare-ruby::./ruby -I.ext/common --disable-gem" \ --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \ --output=markdown --output-compare -v ../ast_keep_tokens.yml compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] warming up.. | |compare-ruby|built-ruby| |:--------------------|-----------:|---------:| |without_keep_tokens | 21.659k| 21.303k| | | 1.02x| -| |with_keep_tokens | 6.220k| 5.691k| | | 1.09x| -| ```
* Make anonymous rest arg (*) and block arg (&) accessible from ARGS nodeyui-knk2022-11-181-4/+7
|
* Allow anonymous keyword rest parameter with other keyword parametersShugo Maeda2022-11-181-1/+1
| | | | Fixes [Bug #19132]
* Using UNDEF_P macroS-H-GAMELINKS2022-11-161-6/+6
|
* Now bison 3.0 or later is requiredNobuyoshi Nakada2022-11-091-0/+2
|
* Set default %printer for NODE ntermsyui-knk2022-11-081-0/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Before: ``` Reducing stack by rule 639 (line 5062): $1 = token "integer literal" (1.0-1.1: 1) -> $$ = nterm simple_numeric (1.0-1.1: ) ``` After: ``` Reducing stack by rule 641 (line 5078): $1 = token "integer literal" (1.0-1.1: 1) -> $$ = nterm simple_numeric (1.0-1.1: NODE_LIT) ``` `"<*>"` is supported by Bison 2.3b (2008-05-27) or later. https://git.savannah.gnu.org/cgit/bison.git/commit/?id=12e3584054c16ab255672c07af0ffc7bb220e8bc Therefore developers need to install Bison 2.3b+ to build ruby from source codes if their Bison is older. Minimum version requirement for Bison is changed to 3.0. See: https://bugs.ruby-lang.org/issues/19068 [Feature #19068]
* Do not set `$!` to `SyntaxError` when error tolerantNobuyoshi Nakada2022-10-091-1/+1
|
* Fix typosyui-knk2022-10-081-37/+37
|
* Fix SEGV of dump parsetreeyui-knk2022-10-081-0/+3
| | | | | | | | | Assign internal_id to semantic value so that dump parsetree option can render the tree for these codes without SEGV. * `def m(&); end` * `def m(*); end` * `def m(**); end`
* "expr_value" can be erroryui-knk2022-10-081-0/+6
| | | | | | | | | | | | So that "IF" node is kept in the case below ``` def m if end ``` [Feature #19013]
* Move `error` from top_stmts and top_stmt to stmtyui-knk2022-10-081-8/+6
| | | | | | | | | | | | | | | | | | | By this change, syntax error is recovered smaller units. In the case below, "DEFN :bar" is same level with "CLASS :Foo" now. ``` module Z class Foo foo. end def bar end end ``` [Feature #19013]
* Treat "end" as reserved word with consideration of indentyui-knk2022-10-081-1/+36
| | | | | | | | | | | | | | | | | | | | "end" after "." or "::" is treated as local variable or method, see `EXPR_DOT_bit` for detail. However this "changes" where `bar` method is defined. In the example below it is not module Z but class Foo. ``` module Z class Foo foo. end def bar end end ``` [Feature #19013]
* Generates "end" tokens if parser hits end of inputyui-knk2022-10-081-6/+105
| | | | | | but "end" tokens are needed for correct language. [Feature #19013]
* Add error_tolerant option to RubyVM::ASTyui-knk2022-10-081-1/+14
| | | | | | | If this option is enabled, SyntaxError is not raised and Node is returned even if passed script is broken. [Feature #19013]
* Suppress a warning on clangShugo Maeda2022-09-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | The following warning appears without this fix: ``` parse.y:78:1: warning: unknown warning group '-Wpsabi', ignored [-Wunknown-warning-option] RBIMPL_WARNING_IGNORED(-Wpsabi) ^ ./include/ruby/internal/warning_push.h:103:39: note: expanded from macro 'RBIMPL_WARNING_IGNORED' ^ ./include/ruby/internal/warning_push.h:99:39: note: expanded from macro 'RBIMPL_WARNING_PRAGMA2' ^ ./include/ruby/internal/warning_push.h:98:39: note: expanded from macro 'RBIMPL_WARNING_PRAGMA1' ^ ./include/ruby/internal/warning_push.h:97:39: note: expanded from macro 'RBIMPL_WARNING_PRAGMA0' ^ <scratch space>:49:27: note: expanded from here clang diagnostic ignored "-Wpsabi" ^ 1 warning generated. ```
* Reuse `opt_arg_append` functionS.H2022-09-141-7/+1
|
* Fix unexpected "duplicated key name" error in paren-less one line pattern ↵Kazuki Tsujimoto2022-09-091-4/+12
| | | | | | matching [Bug #18990]
* [Bug #18963] Separate string contents by here document terminatorNobuyoshi Nakada2022-08-281-0/+13
|
* Reuse `nonlocal_var` patternsS.H2022-08-221-6/+2
|
* Reuse opt_nl ruleS-H-GAMELINKS2022-08-191-2/+1
|
* Repalce to NIL_P macroS-H-GAMELINKS2022-08-191-1/+1
|
View Lorry Controller Status