class Parser::Lexer

line 3 “lib/parser/lexer.rl”

BEFORE YOU START ===

Read the Ruby Hacking Guide chapter 11, available in English at whitequark.org/blog/2013/04/01/ruby-hacking-guide-ch-11-finite-state-lexer/

Remember two things about Ragel scanners:

1) Longest match wins.

2) If two matches have the same length, the first
   in source code wins.

General rules of making Ragel and Bison happy:

* `p` (position) and `@te` contain the index of the character
  they're pointing to ("current"), plus one. `@ts` contains the index
  of the corresponding character. The code for extracting matched token is:

     @source_buffer.slice(@ts...@te)

* If your input is `foooooooobar` and the rule is:

     'f' 'o'+

  the result will be:

     foooooooobar
     ^ ts=0   ^ p=te=9

* A Ragel lexer action should not emit more than one token, unless
  you know what you are doing.

* All Ragel commands (fnext, fgoto, ...) end with a semicolon.

* If an action emits the token and transitions to another state, use
  these Ragel commands:

     emit($whatever)
     fnext $next_state; fbreak;

  If you perform `fgoto` in an action which does not emit a token nor
  rewinds the stream pointer, the parser's side-effectful,
  context-sensitive lookahead actions will break in a hard to detect
  and debug way.

* If an action does not emit a token:

     fgoto $next_state;

* If an action features lookbehind, i.e. matches characters with the
  intent of passing them to another action:

     p = @ts - 1
     fgoto $next_state;

  or, if the lookbehind consists of a single character:

     fhold; fgoto $next_state;

* Ragel merges actions. So, if you have `e_lparen = '(' %act` and
  `c_lparen = '('` and a lexer action `e_lparen | c_lparen`, the result
  _will_ invoke the action `act`.

  e_something stands for "something with **e**mbedded action".

* EOF is explicit and is matched by `c_eof`. If you want to introspect
  the state of the lexer, add this rule to the state:

     c_eof => do_eof;

* If you proceed past EOF, the lexer will complain:

     NoMethodError: undefined method `ord' for nil:NilClass

line 3 “lib/parser/lexer.rl”

BEFORE YOU START ===

Read the Ruby Hacking Guide chapter 11, available in English at whitequark.org/blog/2013/04/01/ruby-hacking-guide-ch-11-finite-state-lexer/

Remember two things about Ragel scanners:

1) Longest match wins.

2) If two matches have the same length, the first
   in source code wins.

General rules of making Ragel and Bison happy:

* `p` (position) and `@te` contain the index of the character
  they're pointing to ("current"), plus one. `@ts` contains the index
  of the corresponding character. The code for extracting matched token is:

     @source_buffer.slice(@ts...@te)

* If your input is `foooooooobar` and the rule is:

     'f' 'o'+

  the result will be:

     foooooooobar
     ^ ts=0   ^ p=te=9

* A Ragel lexer action should not emit more than one token, unless
  you know what you are doing.

* All Ragel commands (fnext, fgoto, ...) end with a semicolon.

* If an action emits the token and transitions to another state, use
  these Ragel commands:

     emit($whatever)
     fnext $next_state; fbreak;

  If you perform `fgoto` in an action which does not emit a token nor
  rewinds the stream pointer, the parser's side-effectful,
  context-sensitive lookahead actions will break in a hard to detect
  and debug way.

* If an action does not emit a token:

     fgoto $next_state;

* If an action features lookbehind, i.e. matches characters with the
  intent of passing them to another action:

     p = @ts - 1
     fgoto $next_state;

  or, if the lookbehind consists of a single character:

     fhold; fgoto $next_state;

* Ragel merges actions. So, if you have `e_lparen = '(' %act` and
  `c_lparen = '('` and a lexer action `e_lparen | c_lparen`, the result
  _will_ invoke the action `act`.

  e_something stands for "something with **e**mbedded action".

* EOF is explicit and is matched by `c_eof`. If you want to introspect
  the state of the lexer, add this rule to the state:

     c_eof => do_eof;

* If you proceed past EOF, the lexer will complain:

     NoMethodError: undefined method `ord' for nil:NilClass

Constants

ESCAPE_WHITESPACE
KEYWORDS
KEYWORDS_BEGIN
LEX_STATES
PUNCTUATION

Mapping of strings to parser tokens.

PUNCTUATION_BEGIN

Attributes

_lex_actions[RW]
_lex_eof_trans[RW]
_lex_from_state_actions[RW]
_lex_index_offsets[RW]
_lex_indicies[RW]
_lex_key_spans[RW]
_lex_to_state_actions[RW]
_lex_trans_actions[RW]
_lex_trans_keys[RW]
_lex_trans_targs[RW]
lex_en_expr_arg[RW]
lex_en_expr_beg[RW]
lex_en_expr_cmdarg[RW]
lex_en_expr_dot[RW]
lex_en_expr_end[RW]
lex_en_expr_endarg[RW]
lex_en_expr_endfn[RW]
lex_en_expr_fname[RW]
lex_en_expr_labelarg[RW]
lex_en_expr_mid[RW]
lex_en_expr_value[RW]
lex_en_expr_variable[RW]
lex_en_inside_string[RW]
lex_en_leading_dot[RW]
lex_en_line_begin[RW]
lex_en_line_comment[RW]
lex_error[RW]
lex_start[RW]
cmdarg[RW]
cmdarg_stack[R]
command_start[RW]
comments[RW]
cond[RW]
cond_stack[R]
context[RW]
diagnostics[RW]
force_utf32[RW]
lambda_stack[R]
paren_nest[R]
source_buffer[R]

line 82 “lib/parser/lexer.rl” %

static_env[RW]
tokens[RW]
version[R]

Public Class Methods

new(version) click to toggle source
# File lib/parser/lexer-F0.rb, line 8382
def initialize(version)
  @version    = version
  @static_env = nil
  @context    = nil

  @tokens     = nil
  @comments   = nil

  @_lex_actions =
    if self.class.respond_to?(:_lex_actions, true)
      self.class.send :_lex_actions
    else
      []
    end

  @emit_integer = lambda { |chars, p| emit(:tINTEGER,   chars); p }
  @emit_rational = lambda { |chars, p| emit(:tRATIONAL,  Rational(chars)); p }
  @emit_imaginary = lambda { |chars, p| emit(:tIMAGINARY, Complex(0, chars)); p }
  @emit_imaginary_rational = lambda { |chars, p| emit(:tIMAGINARY, Complex(0, Rational(chars))); p }
  @emit_integer_re = lambda { |chars, p| emit(:tINTEGER,   chars, @ts, @te - 2); p - 2 }
  @emit_integer_if = lambda { |chars, p| emit(:tINTEGER,   chars, @ts, @te - 2); p - 2 }
  @emit_integer_rescue = lambda { |chars, p| emit(:tINTEGER,   chars, @ts, @te - 6); p - 6 }

  @emit_float = lambda { |chars, p| emit(:tFLOAT,     Float(chars)); p }
  @emit_imaginary_float = lambda { |chars, p| emit(:tIMAGINARY, Complex(0, Float(chars))); p }
  @emit_float_if =     lambda { |chars, p| emit(:tFLOAT,     Float(chars), @ts, @te - 2); p - 2 }
  @emit_float_rescue = lambda { |chars, p| emit(:tFLOAT,     Float(chars), @ts, @te - 6); p - 6 }

  reset
end

Public Instance Methods

advance() click to toggle source

Return next token: [type, value].

# File lib/parser/lexer-F0.rb, line 8542
  def advance
    unless @token_queue.empty?
      return @token_queue.shift
    end

    # Ugly, but dependent on Ragel output. Consider refactoring it somehow.
    klass = self.class
    _lex_trans_keys         = klass.send :_lex_trans_keys
    _lex_key_spans          = klass.send :_lex_key_spans
    _lex_index_offsets      = klass.send :_lex_index_offsets
    _lex_indicies           = klass.send :_lex_indicies
    _lex_trans_targs        = klass.send :_lex_trans_targs
    _lex_trans_actions      = klass.send :_lex_trans_actions
    _lex_to_state_actions   = klass.send :_lex_to_state_actions
    _lex_from_state_actions = klass.send :_lex_from_state_actions
    _lex_eof_trans          = klass.send :_lex_eof_trans
    _lex_actions            = @_lex_actions

    pe = @source_pts.size + 2
    p, eof = @p, pe

    cmd_state = @command_start
    @command_start = false

    
# line 8568 "lib/parser/lexer-F0.rb"
begin # ragel flat
        testEof = false
        _slen, _trans, _keys, _inds, _acts, _nacts = nil
        _goto_level = 0
        _resume = 10
        _eof_trans = 15
        _again = 20
        _test_eof = 30
        _out = 40
        while true
        _trigger_goto = false
        if _goto_level <= 0
        if p == pe
                _goto_level = _test_eof
                next
        end
        if  @cs == 0
                _goto_level = _out
                next
        end
        end
        if _goto_level <= _resume
        _acts = _lex_from_state_actions[ @cs]
        _nacts = _lex_actions[_acts]
        _acts += 1
        while _nacts > 0
                _nacts -= 1
                _acts += 1
                case _lex_actions[_acts - 1]
        when 52 then
# line 1 "NONE"
                begin
 @ts = p
                end
# line 8603 "lib/parser/lexer-F0.rb"
                end # from state action switch
        end
        if _trigger_goto
                next
        end
        _keys =  @cs << 1
        _inds = _lex_index_offsets[ @cs]
        _slen = _lex_key_spans[ @cs]
        _wide = ( (@source_pts[p] || 0))
        _trans = if (   _slen > 0 && 
                        _lex_trans_keys[_keys] <= _wide && 
                        _wide <= _lex_trans_keys[_keys + 1] 
                    ) then
                        _lex_indicies[ _inds + _wide - _lex_trans_keys[_keys] ] 
                 else 
                        _lex_indicies[ _inds + _slen ]
                 end
        end
        if _goto_level <= _eof_trans
         @cs = _lex_trans_targs[_trans]
        if _lex_trans_actions[_trans] != 0
                _acts = _lex_trans_actions[_trans]
                _nacts = _lex_actions[_acts]
                _acts += 1
                while _nacts > 0
                        _nacts -= 1
                        _acts += 1
                        case _lex_actions[_acts - 1]
        when 0 then
# line 539 "lib/parser/lexer.rl"
                begin

    # Record position of a newline for precise location reporting on tNL
    # tokens.
    #
    # This action is embedded directly into c_nl, as it is idempotent and
    # there are no cases when we need to skip it.
    @newline_s = p
                end
        when 1 then
# line 673 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_integer              end
        when 2 then
# line 674 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_rational             end
        when 3 then
# line 675 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_imaginary            end
        when 4 then
# line 676 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_imaginary_rational           end
        when 5 then
# line 677 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_integer_re           end
        when 6 then
# line 678 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_integer_if           end
        when 7 then
# line 679 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_integer_rescue               end
        when 8 then
# line 682 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_float                end
        when 9 then
# line 683 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_imaginary_float              end
        when 10 then
# line 684 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_float_if             end
        when 11 then
# line 688 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_rational             end
        when 12 then
# line 689 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_imaginary_rational           end
        when 13 then
# line 690 "lib/parser/lexer.rl"
                begin
 @num_xfrm = @emit_float_rescue                 end
        when 14 then
# line 696 "lib/parser/lexer.rl"
                begin

    e_lbrace
                end
        when 15 then
# line 700 "lib/parser/lexer.rl"
                begin

    if @strings.close_interp_on_current_literal(p)
      p = p - 1;
       @cs = 128;
        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    end

    @paren_nest -= 1
                end
        when 16 then
# line 723 "lib/parser/lexer.rl"
                begin

    p = on_newline(p)
                end
        when 17 then
# line 733 "lib/parser/lexer.rl"
                begin
 @sharp_s = p - 1               end
        when 18 then
# line 736 "lib/parser/lexer.rl"
                begin
 emit_comment_from_range(p, pe)                 end
        when 19 then
# line 777 "lib/parser/lexer.rl"
                begin
 tm = p                 end
        when 20 then
# line 778 "lib/parser/lexer.rl"
                begin
 tm = p - 2             end
        when 21 then
# line 783 "lib/parser/lexer.rl"
                begin
 tm = p                 end
        when 22 then
# line 784 "lib/parser/lexer.rl"
                begin
 tm = p - 2             end
        when 23 then
# line 785 "lib/parser/lexer.rl"
                begin
 tm = p - 2             end
        when 24 then
# line 786 "lib/parser/lexer.rl"
                begin
 tm = p - 2             end
        when 25 then
# line 787 "lib/parser/lexer.rl"
                begin
 tm = p - 3             end
        when 26 then
# line 792 "lib/parser/lexer.rl"
                begin
 tm = p - 2             end
        when 27 then
# line 797 "lib/parser/lexer.rl"
                begin
 tm = p - 2             end
        when 28 then
# line 803 "lib/parser/lexer.rl"
                begin

    @cond.push(false); @cmdarg.push(false)

    @paren_nest += 1
                end
        when 29 then
# line 809 "lib/parser/lexer.rl"
                begin

    @paren_nest -= 1
                end
        when 30 then
# line 816 "lib/parser/lexer.rl"
                begin

    @cond.push(false); @cmdarg.push(false)

    @paren_nest += 1

    if version?(18)
      @command_start = true
    end
                end
        when 31 then
# line 826 "lib/parser/lexer.rl"
                begin

    @paren_nest -= 1
                end
        when 32 then
# line 1061 "lib/parser/lexer.rl"
                begin
 tm = p                 end
        when 33 then
# line 1074 "lib/parser/lexer.rl"
                begin
 tm = p                 end
        when 34 then
# line 1102 "lib/parser/lexer.rl"
                begin
 tm = p                 end
        when 35 then
# line 1299 "lib/parser/lexer.rl"
                begin
 heredoc_e      = p             end
        when 36 then
# line 1300 "lib/parser/lexer.rl"
                begin
 new_herebody_s = p             end
        when 37 then
# line 1400 "lib/parser/lexer.rl"
                begin
 tm = p - 1; diag_msg = :ivar_name              end
        when 38 then
# line 1401 "lib/parser/lexer.rl"
                begin
 tm = p - 2; diag_msg = :cvar_name              end
        when 39 then
# line 1488 "lib/parser/lexer.rl"
                begin
 tm = p                 end
        when 40 then
# line 1595 "lib/parser/lexer.rl"
                begin
 ident_tok = tok; ident_ts = @ts; ident_te = @te;               end
        when 41 then
# line 1781 "lib/parser/lexer.rl"
                begin
 @num_base = 16; @num_digits_s = p              end
        when 42 then
# line 1782 "lib/parser/lexer.rl"
                begin
 @num_base = 10; @num_digits_s = p              end
        when 43 then
# line 1783 "lib/parser/lexer.rl"
                begin
 @num_base = 8;  @num_digits_s = p              end
        when 44 then
# line 1784 "lib/parser/lexer.rl"
                begin
 @num_base = 2;  @num_digits_s = p              end
        when 45 then
# line 1785 "lib/parser/lexer.rl"
                begin
 @num_base = 10; @num_digits_s = @ts            end
        when 46 then
# line 1786 "lib/parser/lexer.rl"
                begin
 @num_base = 8;  @num_digits_s = @ts            end
        when 47 then
# line 1787 "lib/parser/lexer.rl"
                begin
 @num_suffix_s = p              end
        when 48 then
# line 1830 "lib/parser/lexer.rl"
                begin
 @num_suffix_s = p              end
        when 49 then
# line 1831 "lib/parser/lexer.rl"
                begin
 @num_suffix_s = p              end
        when 50 then
# line 2034 "lib/parser/lexer.rl"
                begin
 tm = p                 end
        when 53 then
# line 1 "NONE"
                begin
 @te = p+1
                end
        when 54 then
# line 848 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit_global_var

         @cs = (stack_pop);     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 55 then
# line 848 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_global_var

         @cs = (stack_pop);     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 56 then
# line 855 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_class_var

         @cs = (stack_pop);     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 57 then
# line 862 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_instance_var

         @cs = (stack_pop);     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 58 then
# line 880 "lib/parser/lexer.rl"
                begin
 @act = 4;              end
        when 59 then
# line 884 "lib/parser/lexer.rl"
                begin
 @act = 5;              end
        when 60 then
# line 888 "lib/parser/lexer.rl"
                begin
 @act = 6;              end
        when 61 then
# line 880 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_table(KEYWORDS_BEGIN);
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 62 then
# line 888 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit(:tIDENTIFIER)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 63 then
# line 892 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = @ts - 1
            @cs = 516;  begin
                 @stack[ @top] =  @cs
                 @top+= 1
                 @cs = 129
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 64 then
# line 901 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_table(PUNCTUATION)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 65 then
# line 905 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1; p = p - 1;   begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 66 then
# line 911 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        if version?(23)
          type, delimiter = tok[0..-2], tok[-1].chr
          @strings.push_literal(type, delimiter, @ts)
                begin
                 @cs = 128
                _trigger_goto = true
                _goto_level = _again
                break
        end

        else
          p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

        end
       end
                end
        when 67 then
# line 925 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 68 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 69 then
# line 880 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(KEYWORDS_BEGIN);
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 70 then
# line 884 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tCONSTANT)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 71 then
# line 888 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tIDENTIFIER)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 72 then
# line 892 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = @ts - 1
            @cs = 516;  begin
                 @stack[ @top] =  @cs
                 @top+= 1
                 @cs = 129
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 73 then
# line 901 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(PUNCTUATION)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 74 then
# line 908 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 75 then
# line 922 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 76 then
# line 925 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 77 then
# line 901 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  emit_table(PUNCTUATION)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 78 then
# line 925 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  p = p - 1;      begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 79 then
# line 1 "NONE"
                begin
        case  @act
        when 4 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS_BEGIN);
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 5 then
        begin begin p = (( @te))-1; end
 emit(:tCONSTANT)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 6 then
        begin begin p = (( @te))-1; end
 emit(:tIDENTIFIER)
            @cs = 247;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
end 
                        end
        when 80 then
# line 937 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit(:tLABEL, tok(@ts, @te - 2), @ts, @te - 1)
           p = p - 1;  @cs = 501;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 81 then
# line 941 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        if @version >= 31 && @context.in_argdef
          emit(:tBDOT3, '...'.freeze)
          # emit(:tNL, "\n".freeze, @te - 1, @te)
           @cs = 516;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
          p -= 3;
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

        end
       end
                end
        when 82 then
# line 955 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 83 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 84 then
# line 952 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 85 then
# line 955 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 86 then
# line 955 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  p = p - 1;      begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 87 then
# line 981 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_table(PUNCTUATION)
            @cs = 276;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 88 then
# line 987 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 89 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 90 then
# line 966 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tCONSTANT)
            @cs = (arg_or_cmdarg(cmd_state));   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 91 then
# line 970 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tIDENTIFIER)
            @cs = (arg_or_cmdarg(cmd_state));   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 92 then
# line 974 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tFID, tok(@ts, tm), @ts, tm)
            @cs = (arg_or_cmdarg(cmd_state)); p = tm - 1;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 93 then
# line 981 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(PUNCTUATION)
            @cs = 276;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 94 then
# line 984 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 95 then
# line 987 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 96 then
# line 1065 "lib/parser/lexer.rl"
                begin
 @act = 33;             end
        when 97 then
# line 1075 "lib/parser/lexer.rl"
                begin
 @act = 34;             end
        when 98 then
# line 1114 "lib/parser/lexer.rl"
                begin
 @act = 39;             end
        when 99 then
# line 1119 "lib/parser/lexer.rl"
                begin
 @act = 40;             end
        when 100 then
# line 1047 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        # Unlike expr_beg as invoked in the next rule, do not warn
        p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 101 then
# line 1065 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        check_ambiguous_slash(tm)

        p = tm - 1
                begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 102 then
# line 1086 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1; p = p - 1;   begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 103 then
# line 1094 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = @ts - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 104 then
# line 1103 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = tm - 1;     begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 105 then
# line 1114 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 106 then
# line 1128 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 107 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 108 then
# line 1003 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if version?(18)
          emit(:tLPAREN2, '('.freeze, @te - 1, @te)
           @cs = 508;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
          emit(:tLPAREN_ARG, '('.freeze, @te - 1, @te)
           @cs = 345;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
       end
                end
        when 109 then
# line 1016 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tLPAREN2, '('.freeze)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 110 then
# line 1022 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tLBRACK, '['.freeze, @te - 1, @te)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 111 then
# line 1028 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if @lambda_stack.last == @paren_nest
          @lambda_stack.pop
          emit(:tLAMBEG, '{'.freeze, @te - 1, @te)
        else
          emit(:tLCURLY, '{'.freeze, @te - 1, @te)
        end
        @command_start = true
        @paren_nest += 1
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 112 then
# line 1056 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 113 then
# line 1075 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        diagnostic :warning, :ambiguous_prefix, { :prefix => tok(tm, @te) },
                   range(tm, @te)

        p = tm - 1
                begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 114 then
# line 1091 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 115 then
# line 1114 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 116 then
# line 1119 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 117 then
# line 1122 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin        begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 118 then
# line 1125 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 119 then
# line 1128 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 120 then
# line 1119 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
                end
        when 121 then
# line 1128 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  p = p - 1;      begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 122 then
# line 1 "NONE"
                begin
        case  @act
        when 33 then
        begin begin p = (( @te))-1; end

        check_ambiguous_slash(tm)

        p = tm - 1
                begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end

      end
        when 34 then
        begin begin p = (( @te))-1; end

        diagnostic :warning, :ambiguous_prefix, { :prefix => tok(tm, @te) },
                   range(tm, @te)

        p = tm - 1
                begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end

      end
        when 39 then
        begin begin p = (( @te))-1; end

        p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

      end
        else
        begin begin p = (( @te))-1; end
end
end 
                        end
        when 123 then
# line 1151 "lib/parser/lexer.rl"
                begin
 @act = 46;             end
        when 124 then
# line 1164 "lib/parser/lexer.rl"
                begin
 @act = 47;             end
        when 125 then
# line 1164 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = @ts - 1
                begin
                 @cs = 276
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 126 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 127 then
# line 1141 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit(:tLPAREN_ARG, '('.freeze, @te - 1, @te)
        if version?(18)
           @cs = 508;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
           @cs = 345;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
       end
                end
        when 128 then
# line 1164 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = @ts - 1
                begin
                 @cs = 276
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 129 then
# line 1164 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  p = @ts - 1
                begin
                 @cs = 276
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 130 then
# line 1 "NONE"
                begin
        case  @act
        when 46 then
        begin begin p = (( @te))-1; end

        if @cond.active?
          emit(:kDO_COND, 'do'.freeze, @te - 2, @te)
        else
          emit(:kDO, 'do'.freeze, @te - 2, @te)
        end
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 47 then
        begin begin p = (( @te))-1; end
 p = @ts - 1
                begin
                 @cs = 276
                _trigger_goto = true
                _goto_level = _again
                break
        end
 end
end 
                        end
        when 131 then
# line 1200 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_do(true)
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 132 then
# line 1206 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 133 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 134 then
# line 1187 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if @lambda_stack.last == @paren_nest
          @lambda_stack.pop
          emit(:tLAMBEG, '{'.freeze)
        else
          emit(:tLBRACE_ARG, '{'.freeze)
        end
        @paren_nest += 1
        @command_start = true
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 135 then
# line 1203 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 136 then
# line 1206 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 137 then
# line 1218 "lib/parser/lexer.rl"
                begin
 @act = 54;             end
        when 138 then
# line 1222 "lib/parser/lexer.rl"
                begin
 @act = 55;             end
        when 139 then
# line 1230 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 140 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 141 then
# line 1222 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = @ts - 1;  begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 142 then
# line 1224 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 143 then
# line 1227 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 144 then
# line 1230 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 145 then
# line 1 "NONE"
                begin
        case  @act
        when 54 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 55 then
        begin begin p = (( @te))-1; end
 p = @ts - 1;   begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
 end
end 
                        end
        when 146 then
# line 1245 "lib/parser/lexer.rl"
                begin
 @act = 60;             end
        when 147 then
# line 1348 "lib/parser/lexer.rl"
                begin
 @act = 67;             end
        when 148 then
# line 1442 "lib/parser/lexer.rl"
                begin
 @act = 76;             end
        when 149 then
# line 1483 "lib/parser/lexer.rl"
                begin
 @act = 80;             end
        when 150 then
# line 1489 "lib/parser/lexer.rl"
                begin
 @act = 81;             end
        when 151 then
# line 1495 "lib/parser/lexer.rl"
                begin
 @act = 82;             end
        when 152 then
# line 1586 "lib/parser/lexer.rl"
                begin
 @act = 86;             end
        when 153 then
# line 831 "lib/parser/lexer.rl"
                begin
 @act = 87;             end
        when 154 then
# line 1632 "lib/parser/lexer.rl"
                begin
 @act = 91;             end
        when 155 then
# line 1245 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit(:tUNARY_NUM, tok(@ts, @ts + 1), @ts, @ts + 1)
        p = p - 1;  @cs = 516;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 156 then
# line 1262 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        type = delimiter = tok[0].chr
        @strings.push_literal(type, delimiter, @ts)

        p = p - 1;
                begin
                 @cs = 128
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 157 then
# line 1272 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        type, delimiter = @source_buffer.slice(@ts, 1).chr, tok[-1].chr
        @strings.push_literal(type, delimiter, @ts)
                begin
                 @cs = 128
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 158 then
# line 1280 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        type, delimiter = tok[0..-2], tok[-1].chr
        @strings.push_literal(type, delimiter, @ts)
                begin
                 @cs = 128
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 159 then
# line 1357 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        p = p - 1; p = p - 1;
        emit(:tSYMBEG, tok(@ts, @ts + 1), @ts, @ts + 1)
                begin
                 @cs = 134
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 160 then
# line 1365 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        type, delimiter = tok, tok[-1].chr
        @strings.push_literal(type, delimiter, @ts);

                begin
                 @cs = 128
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 161 then
# line 1375 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit(:tSYMBOL, tok(@ts + 1, @ts + 2))
         @cs = 516;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 162 then
# line 1389 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        gvar_name = tok(@ts + 1)

        if @version >= 33 && gvar_name.start_with?('$0') && gvar_name.length > 2
          diagnostic :error, :gvar_name, { :name => gvar_name }, range(@ts + 1, @te)
        end

        emit(:tSYMBOL, gvar_name, @ts)
         @cs = 516;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 163 then
# line 1416 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        p, next_state = @strings.read_character_constant(@ts)
        p = p - 1; # Ragel will do `p += 1` to consume input, prevent it

        # If strings lexer founds a character constant (?a) emit it,
        # otherwise read ternary operator
        if @token_queue.empty?
                begin
                 @cs = (next_state)
                _trigger_goto = true
                _goto_level = _again
                break
        end

        else
           @cs = (next_state);
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
       end
                end
        when 164 then
# line 1431 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        diagnostic :fatal, :incomplete_escape, nil, range(@ts, @ts + 1)
       end
                end
        when 165 then
# line 1483 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_table(PUNCTUATION_BEGIN)
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 166 then
# line 1504 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        p = p - 1;

        if version?(18)
          ident = tok(@ts, @te - 2)

          emit((@source_buffer.slice(@ts, 1) =~ /[A-Z]/) ? :tCONSTANT : :tIDENTIFIER,
               ident, @ts, @te - 2)
          p = p - 1; # continue as a symbol

          if !@static_env.nil? && @static_env.declared?(ident)
             @cs = 516;
          else
             @cs = (arg_or_cmdarg(cmd_state));
          end
        else
          emit(:tLABEL, tok(@ts, @te - 2), @ts, @te - 1)
           @cs = 501;
        end

                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 167 then
# line 1542 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        # Here we scan and conditionally emit "\n":
        # + if it's there
        #   + and emitted we do nothing
        #   + and not emitted we return `p` to "\n" to process it on the next scan
        # + if it's not there we do nothing
        followed_by_nl = @te - 1 == @newline_s
        nl_emitted = false
        dots_te = followed_by_nl ? @te - 1 : @te

        if @version >= 30
          if @lambda_stack.any? && @lambda_stack.last + 1 == @paren_nest
            # To reject `->(...)` like `->...`
            emit(:tDOT3, '...'.freeze, @ts, dots_te)
          else
            emit(:tBDOT3, '...'.freeze, @ts, dots_te)

            if @version >= 31 && followed_by_nl && @context.in_argdef
              emit(:tNL, @te - 1, @te)
              nl_emitted = true
            end
          end
        elsif @version >= 27
          emit(:tBDOT3, '...'.freeze, @ts, dots_te)
        else
          emit(:tDOT3, '...'.freeze, @ts, dots_te)
        end

        if followed_by_nl && !nl_emitted
          # return "\n" to process it on the next scan
          p = p - 1;
        end

         @cs = 345;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 168 then
# line 1597 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit(:tIDENTIFIER, ident_tok, ident_ts, ident_te)
        p = ident_te - 1

        if !@static_env.nil? && @static_env.declared?(ident_tok) && @version < 25
           @cs = 247;
        else
           @cs = 307;
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 169 then
# line 1616 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        p = @ts - 1
        @cs_before_block_comment = @cs
                begin
                 @cs = 710
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 170 then
# line 1632 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = @ts - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 171 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 172 then
# line 1245 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit(:tUNARY_NUM, tok(@ts, @ts + 1), @ts, @ts + 1)
        p = p - 1;  @cs = 516;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 173 then
# line 1252 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tSTAR, '*'.freeze)
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 174 then
# line 1287 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        diagnostic :fatal, :string_eof, nil, range(@ts, @ts + 1)
       end
                end
        when 175 then
# line 1301 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        tok(@ts, heredoc_e) =~ /^<<(-?)(~?)(["'`]?)(.*)\3$/m

        indent      = !$1.empty? || !$2.empty?
        dedent_body = !$2.empty?
        type        =  $3.empty? ? '<<"'.freeze : ('<<'.freeze + $3)
        delimiter   =  $4

        if @version >= 27
          if delimiter.count("\n") > 0 || delimiter.count("\r") > 0
            diagnostic :error, :unterminated_heredoc_id, nil, range(@ts, @ts + 1)
          end
        elsif @version >= 24
          if delimiter.count("\n") > 0
            if delimiter.end_with?("\n")
              diagnostic :warning, :heredoc_id_ends_with_nl, nil, range(@ts, @ts + 1)
              delimiter = delimiter.rstrip
            else
              diagnostic :fatal, :heredoc_id_has_newline, nil, range(@ts, @ts + 1)
            end
          end
        end

        if dedent_body && version?(18, 19, 20, 21, 22)
          emit(:tLSHFT, '<<'.freeze, @ts, @ts + 2)
          p = @ts + 1
           @cs = 345;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
          @strings.push_literal(type, delimiter, @ts, heredoc_e, indent, dedent_body);
          @strings.herebody_s ||= new_herebody_s

          p = @strings.herebody_s - 1
           @cs = 128;
        end
       end
                end
        when 176 then
# line 1348 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        diagnostic :error, :unterminated_heredoc_id, nil, range(@ts, @ts + 1)
       end
                end
        when 177 then
# line 1381 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit(:tSYMBOL, tok(@ts + 1, tm), @ts, tm)
        p = tm - 1
         @cs = 516;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 178 then
# line 1389 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        gvar_name = tok(@ts + 1)

        if @version >= 33 && gvar_name.start_with?('$0') && gvar_name.length > 2
          diagnostic :error, :gvar_name, { :name => gvar_name }, range(@ts + 1, @te)
        end

        emit(:tSYMBOL, gvar_name, @ts)
         @cs = 516;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 179 then
# line 1403 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_colon_with_digits(p, tm, diag_msg)

         @cs = 516;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 180 then
# line 1431 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        diagnostic :fatal, :incomplete_escape, nil, range(@ts, @ts + 1)
       end
                end
        when 181 then
# line 1459 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if @lambda_stack.last == @paren_nest
          @lambda_stack.pop
          @command_start = true
          emit(:tLAMBEG, '{'.freeze)
        else
          emit(:tLBRACE, '{'.freeze)
        end
        @paren_nest += 1
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 182 then
# line 1473 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tLBRACK, '['.freeze)
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 183 then
# line 1478 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tLPAREN, '('.freeze)
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 184 then
# line 1483 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(PUNCTUATION_BEGIN)
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 185 then
# line 1489 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:kRESCUE, 'rescue'.freeze, @ts, tm)
           p = tm - 1
            @cs = 321;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 186 then
# line 1531 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if @version >= 27
          emit(:tBDOT2)
        else
          emit(:tDOT2)
        end

         @cs = 345;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 187 then
# line 1542 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        # Here we scan and conditionally emit "\n":
        # + if it's there
        #   + and emitted we do nothing
        #   + and not emitted we return `p` to "\n" to process it on the next scan
        # + if it's not there we do nothing
        followed_by_nl = @te - 1 == @newline_s
        nl_emitted = false
        dots_te = followed_by_nl ? @te - 1 : @te

        if @version >= 30
          if @lambda_stack.any? && @lambda_stack.last + 1 == @paren_nest
            # To reject `->(...)` like `->...`
            emit(:tDOT3, '...'.freeze, @ts, dots_te)
          else
            emit(:tBDOT3, '...'.freeze, @ts, dots_te)

            if @version >= 31 && followed_by_nl && @context.in_argdef
              emit(:tNL, @te - 1, @te)
              nl_emitted = true
            end
          end
        elsif @version >= 27
          emit(:tBDOT3, '...'.freeze, @ts, dots_te)
        else
          emit(:tDOT3, '...'.freeze, @ts, dots_te)
        end

        if followed_by_nl && !nl_emitted
          # return "\n" to process it on the next scan
          p = p - 1;
        end

         @cs = 345;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 188 then
# line 1586 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 189 then
# line 831 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
    emit(:tIDENTIFIER)

    if !@static_env.nil? && @static_env.declared?(tok)
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    elsif @version >= 32 && tok =~ /\A_[1-9]\z/
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    else
       @cs = (arg_or_cmdarg(cmd_state));        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    end
   end
                end
        when 190 then
# line 1613 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 191 then
# line 1616 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        p = @ts - 1
        @cs_before_block_comment = @cs
                begin
                 @cs = 710
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 192 then
# line 1632 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = @ts - 1;  begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 193 then
# line 1287 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
        diagnostic :fatal, :string_eof, nil, range(@ts, @ts + 1)
       end
                end
        when 194 then
# line 1348 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
        diagnostic :error, :unterminated_heredoc_id, nil, range(@ts, @ts + 1)
       end
                end
        when 195 then
# line 831 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
    emit(:tIDENTIFIER)

    if !@static_env.nil? && @static_env.declared?(tok)
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    elsif @version >= 32 && tok =~ /\A_[1-9]\z/
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    else
       @cs = (arg_or_cmdarg(cmd_state));        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    end
   end
                end
        when 196 then
# line 1613 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
                end
        when 197 then
# line 1632 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  p = @ts - 1;    begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 198 then
# line 1 "NONE"
                begin
        case  @act
        when 60 then
        begin begin p = (( @te))-1; end

        emit(:tUNARY_NUM, tok(@ts, @ts + 1), @ts, @ts + 1)
        p = p - 1;  @cs = 516;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 67 then
        begin begin p = (( @te))-1; end

        diagnostic :error, :unterminated_heredoc_id, nil, range(@ts, @ts + 1)
      end
        when 76 then
        begin begin p = (( @te))-1; end

        if @version >= 27
          emit(:tPIPE, tok(@ts, @ts + 1), @ts, @ts + 1)
          p = p - 1;
           @cs = 345;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
          p -= 2
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

        end
      end
        when 80 then
        begin begin p = (( @te))-1; end
 emit_table(PUNCTUATION_BEGIN)
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 81 then
        begin begin p = (( @te))-1; end
 emit(:kRESCUE, 'rescue'.freeze, @ts, tm)
           p = tm - 1
            @cs = 321;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 82 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS_BEGIN)
           @command_start = true
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 86 then
        begin begin p = (( @te))-1; end
 p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
 end
        when 87 then
        begin begin p = (( @te))-1; end

    emit(:tIDENTIFIER)

    if !@static_env.nil? && @static_env.declared?(tok)
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    elsif @version >= 32 && tok =~ /\A_[1-9]\z/
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    else
       @cs = (arg_or_cmdarg(cmd_state));        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    end
  end
        when 91 then
        begin begin p = (( @te))-1; end
 p = @ts - 1;   begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
 end
end 
                        end
        when 199 then
# line 1652 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 200 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 201 then
# line 1640 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 202 then
# line 1643 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
      if @context.in_kwarg
        p = p - 1;      begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end

      else
                begin
                 @cs = 710
                _trigger_goto = true
                _goto_level = _again
                break
        end

      end
     end
                end
        when 203 then
# line 1652 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 204 then
# line 1662 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = @ts - 1
                begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 205 then
# line 1667 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        @strings.push_literal(tok, tok, @ts)
                begin
                 @cs = 128
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 206 then
# line 1678 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = p - 1;      begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 207 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 208 then
# line 1672 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 209 then
# line 1675 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin        begin
                 @cs = 710
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 210 then
# line 1678 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = p - 1;    begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 211 then
# line 1678 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  p = p - 1;      begin
                 @cs = 345
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 212 then
# line 1697 "lib/parser/lexer.rl"
                begin
 @act = 104;            end
        when 213 then
# line 1726 "lib/parser/lexer.rl"
                begin
 @act = 105;            end
        when 214 then
# line 1730 "lib/parser/lexer.rl"
                begin
 @act = 106;            end
        when 215 then
# line 1735 "lib/parser/lexer.rl"
                begin
 @act = 107;            end
        when 216 then
# line 1740 "lib/parser/lexer.rl"
                begin
 @act = 108;            end
        when 217 then
# line 1745 "lib/parser/lexer.rl"
                begin
 @act = 109;            end
        when 218 then
# line 1749 "lib/parser/lexer.rl"
                begin
 @act = 110;            end
        when 219 then
# line 1760 "lib/parser/lexer.rl"
                begin
 @act = 111;            end
        when 220 then
# line 1774 "lib/parser/lexer.rl"
                begin
 @act = 112;            end
        when 221 then
# line 1788 "lib/parser/lexer.rl"
                begin
 @act = 113;            end
        when 222 then
# line 1806 "lib/parser/lexer.rl"
                begin
 @act = 115;            end
        when 223 then
# line 1818 "lib/parser/lexer.rl"
                begin
 @act = 116;            end
        when 224 then
# line 1833 "lib/parser/lexer.rl"
                begin
 @act = 117;            end
        when 225 then
# line 1862 "lib/parser/lexer.rl"
                begin
 @act = 119;            end
        when 226 then
# line 831 "lib/parser/lexer.rl"
                begin
 @act = 123;            end
        when 227 then
# line 1884 "lib/parser/lexer.rl"
                begin
 @act = 124;            end
        when 228 then
# line 1910 "lib/parser/lexer.rl"
                begin
 @act = 126;            end
        when 229 then
# line 1916 "lib/parser/lexer.rl"
                begin
 @act = 127;            end
        when 230 then
# line 1689 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit(:tLAMBDA, '->'.freeze, @ts, @ts + 2)

        @lambda_stack.push @paren_nest
         @cs = 247;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 231 then
# line 1730 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_singleton_class
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 232 then
# line 1851 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        type, delimiter = tok, tok[-1].chr
        @strings.push_literal(type, delimiter, @ts, nil, false, false, true);
                begin
                 @cs = 128
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 233 then
# line 1870 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = @ts - 1;    begin
                 @stack[ @top] =  @cs
                 @top+= 1
                 @cs = 129
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 234 then
# line 1877 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_table(PUNCTUATION)
            @cs = 255;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 235 then
# line 1901 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit_table(PUNCTUATION)
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 236 then
# line 1910 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit_table(PUNCTUATION);
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 237 then
# line 1937 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit(:tOP_ASGN, tok(@ts, @te - 1))
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 238 then
# line 1941 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit(:tEH, '?'.freeze)
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 239 then
# line 1949 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        if @paren_nest == 0
          diagnostic :warning, :triple_dot_at_eol, nil, range(@ts, @te - 1)
        end

        emit(:tDOT3, '...'.freeze, @ts, @te - 1)
        p = p - 1;
         @cs = 345;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 240 then
# line 1960 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit_table(PUNCTUATION)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 241 then
# line 1973 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit(:tSEMI, ';'.freeze)
           @command_start = true
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 242 then
# line 1977 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        diagnostic :error, :bare_backslash, nil, range(@ts, @ts + 1)
        p = p - 1;
       end
                end
        when 243 then
# line 1983 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        diagnostic :fatal, :unexpected, { :character => tok.inspect[1..-2] }
       end
                end
        when 244 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 245 then
# line 1697 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if @lambda_stack.last == @paren_nest
          @lambda_stack.pop

          if tok == '{'.freeze
            emit(:tLAMBEG, '{'.freeze)
          else # 'do'
            emit(:kDO_LAMBDA, 'do'.freeze)
          end
        else
          if tok == '{'.freeze
            emit(:tLCURLY, '{'.freeze)
          else # 'do'
            emit_do
          end
        end
        if tok == '{'.freeze
          @paren_nest += 1
        end
        @command_start = true

         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 246 then
# line 1726 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(KEYWORDS)
            @cs = 134;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 247 then
# line 1730 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_singleton_class
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 248 then
# line 1740 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(KEYWORDS)
           @command_start = true
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 249 then
# line 1788 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        digits = numeric_literal_int

        if version?(18, 19, 20)
          emit(:tINTEGER, digits.to_i(@num_base), @ts, @num_suffix_s)
          p = @num_suffix_s - 1
        else
          p = @num_xfrm.call(digits.to_i(@num_base), p)
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 250 then
# line 1801 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        diagnostic :error, :no_dot_digit_literal
       end
                end
        when 251 then
# line 1833 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        digits = tok(@ts, @num_suffix_s)

        if version?(18, 19, 20)
          emit(:tFLOAT, Float(digits), @ts, @num_suffix_s)
          p = @num_suffix_s - 1
        else
          p = @num_xfrm.call(digits, p)
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 252 then
# line 1862 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tCONSTANT)
            @cs = (arg_or_cmdarg(cmd_state));   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 253 then
# line 1866 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tCONSTANT, tok(@ts, tm), @ts, tm)
           p = tm - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 254 then
# line 1870 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = @ts - 1;  begin
                 @stack[ @top] =  @cs
                 @top+= 1
                 @cs = 129
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 255 then
# line 1877 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(PUNCTUATION)
            @cs = 255;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 256 then
# line 831 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
    emit(:tIDENTIFIER)

    if !@static_env.nil? && @static_env.declared?(tok)
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    elsif @version >= 32 && tok =~ /\A_[1-9]\z/
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    else
       @cs = (arg_or_cmdarg(cmd_state));        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    end
   end
                end
        when 257 then
# line 1884 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if tm == @te
          # Suffix was consumed, e.g. foo!
          emit(:tFID)
        else
          # Suffix was not consumed, e.g. foo!=
          emit(:tIDENTIFIER, tok(@ts, tm), @ts, tm)
          p = tm - 1
        end
         @cs = 276;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 258 then
# line 1901 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_table(PUNCTUATION)
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 259 then
# line 1910 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_table(PUNCTUATION);
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 260 then
# line 1916 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(PUNCTUATION)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 261 then
# line 1920 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_rbrace_rparen_rbrack

        if tok == '}'.freeze || tok == ']'.freeze
          if @version >= 25
             @cs = 516;
          else
             @cs = 313;
          end
        else # )
          # fnext expr_endfn; ?
        end

                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 262 then
# line 1945 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tLBRACK2, '['.freeze)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 263 then
# line 1960 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit_table(PUNCTUATION)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 264 then
# line 1967 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 265 then
# line 1970 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin        begin
                 @cs = 696
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 266 then
# line 1983 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        diagnostic :fatal, :unexpected, { :character => tok.inspect[1..-2] }
       end
                end
        when 267 then
# line 1788 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
        digits = numeric_literal_int

        if version?(18, 19, 20)
          emit(:tINTEGER, digits.to_i(@num_base), @ts, @num_suffix_s)
          p = @num_suffix_s - 1
        else
          p = @num_xfrm.call(digits.to_i(@num_base), p)
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 268 then
# line 1801 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
        diagnostic :error, :no_dot_digit_literal
       end
                end
        when 269 then
# line 1833 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
        digits = tok(@ts, @num_suffix_s)

        if version?(18, 19, 20)
          emit(:tFLOAT, Float(digits), @ts, @num_suffix_s)
          p = @num_suffix_s - 1
        else
          p = @num_xfrm.call(digits, p)
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
        when 270 then
# line 1983 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
        diagnostic :fatal, :unexpected, { :character => tok.inspect[1..-2] }
       end
                end
        when 271 then
# line 1 "NONE"
                begin
        case  @act
        when 104 then
        begin begin p = (( @te))-1; end

        if @lambda_stack.last == @paren_nest
          @lambda_stack.pop

          if tok == '{'.freeze
            emit(:tLAMBEG, '{'.freeze)
          else # 'do'
            emit(:kDO_LAMBDA, 'do'.freeze)
          end
        else
          if tok == '{'.freeze
            emit(:tLCURLY, '{'.freeze)
          else # 'do'
            emit_do
          end
        end
        if tok == '{'.freeze
          @paren_nest += 1
        end
        @command_start = true

         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 105 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS)
            @cs = 134;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 106 then
        begin begin p = (( @te))-1; end
 emit_singleton_class
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 107 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 108 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS)
           @command_start = true
            @cs = 508;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 109 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS)
            @cs = 321;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 110 then
        begin begin p = (( @te))-1; end

        emit_table(KEYWORDS)

        if version?(18) && tok == 'not'.freeze
           @cs = 345;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
           @cs = 276;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
      end
        when 111 then
        begin begin p = (( @te))-1; end

        if version?(18)
          emit(:tIDENTIFIER)

          unless !@static_env.nil? && @static_env.declared?(tok)
             @cs = (arg_or_cmdarg(cmd_state));
          end
        else
          emit(:k__ENCODING__, '__ENCODING__'.freeze)
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 112 then
        begin begin p = (( @te))-1; end
 emit_table(KEYWORDS)
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 113 then
        begin begin p = (( @te))-1; end

        digits = numeric_literal_int

        if version?(18, 19, 20)
          emit(:tINTEGER, digits.to_i(@num_base), @ts, @num_suffix_s)
          p = @num_suffix_s - 1
        else
          p = @num_xfrm.call(digits.to_i(@num_base), p)
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 115 then
        begin begin p = (( @te))-1; end

        if version?(18, 19, 20)
          diagnostic :error,
                     :trailing_in_number, { :character => tok(@te - 1, @te) },
                     range(@te - 1, @te)
        else
          emit(:tINTEGER, tok(@ts, @te - 1).to_i, @ts, @te - 1)
          p = p - 1;    begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
      end
        when 116 then
        begin begin p = (( @te))-1; end

        if version?(18, 19, 20)
          diagnostic :error,
                     :trailing_in_number, { :character => tok(@te - 1, @te) },
                     range(@te - 1, @te)
        else
          emit(:tFLOAT, tok(@ts, @te - 1).to_f, @ts, @te - 1)
          p = p - 1;    begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
      end
        when 117 then
        begin begin p = (( @te))-1; end

        digits = tok(@ts, @num_suffix_s)

        if version?(18, 19, 20)
          emit(:tFLOAT, Float(digits), @ts, @num_suffix_s)
          p = @num_suffix_s - 1
        else
          p = @num_xfrm.call(digits, p)
        end
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 119 then
        begin begin p = (( @te))-1; end
 emit(:tCONSTANT)
            @cs = (arg_or_cmdarg(cmd_state));   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
        when 123 then
        begin begin p = (( @te))-1; end

    emit(:tIDENTIFIER)

    if !@static_env.nil? && @static_env.declared?(tok)
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    elsif @version >= 32 && tok =~ /\A_[1-9]\z/
       @cs = 247;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    else
       @cs = (arg_or_cmdarg(cmd_state));        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

    end
  end
        when 124 then
        begin begin p = (( @te))-1; end

        if tm == @te
          # Suffix was consumed, e.g. foo!
          emit(:tFID)
        else
          # Suffix was not consumed, e.g. foo!=
          emit(:tIDENTIFIER, tok(@ts, tm), @ts, tm)
          p = tm - 1
        end
         @cs = 276;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 126 then
        begin begin p = (( @te))-1; end

        emit_table(PUNCTUATION);
         @cs = 508;     begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

      end
        when 127 then
        begin begin p = (( @te))-1; end
 emit_table(PUNCTUATION)
            @cs = 345;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
end 
                        end
        when 272 then
# line 1999 "lib/parser/lexer.rl"
                begin
 @act = 140;            end
        when 273 then
# line 2038 "lib/parser/lexer.rl"
                begin
 @act = 144;            end
        when 274 then
# line 2024 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit(:tNL, nil, @newline_s, @newline_s + 1)
        if @version < 27
          p = p - 1;  @cs = 710;        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
          emit(:tBDOT3)
           @cs = 345;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
       end
                end
        when 275 then
# line 2035 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = tm - 1;     begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 276 then
# line 2038 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  emit(:tNL, nil, @newline_s, @newline_s + 1)
           p = p - 1;  @cs = 710;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 277 then
# line 1999 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        if @version < 27
          # Ruby before 2.7 doesn't support comments before leading dot.
          # If a line after "a" starts with a comment then "a" is a self-contained statement.
          # So in that case we emit a special tNL token and start reading the
          # next line as a separate statement.
          #
          # Note: block comments before leading dot are not supported on any version of Ruby.
          emit(:tNL, nil, @newline_s, @newline_s + 1)
          p = p - 1;  @cs = 710;        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
       end
                end
        when 278 then
# line 2013 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit(:tNL, nil, @newline_s, @newline_s + 1)
        if @version < 27
          p = p - 1;  @cs = 710;        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        else
          emit(:tBDOT2)
           @cs = 345;   begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
       end
                end
        when 279 then
# line 2035 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  p = tm - 1;   begin
                 @cs = 516
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 280 then
# line 2038 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  emit(:tNL, nil, @newline_s, @newline_s + 1)
           p = p - 1;  @cs = 710;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 281 then
# line 1999 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin 
        if @version < 27
          # Ruby before 2.7 doesn't support comments before leading dot.
          # If a line after "a" starts with a comment then "a" is a self-contained statement.
          # So in that case we emit a special tNL token and start reading the
          # next line as a separate statement.
          #
          # Note: block comments before leading dot are not supported on any version of Ruby.
          emit(:tNL, nil, @newline_s, @newline_s + 1)
          p = p - 1;  @cs = 710;        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
       end
                end
        when 282 then
# line 2038 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  emit(:tNL, nil, @newline_s, @newline_s + 1)
           p = p - 1;  @cs = 710;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
  end
                end
        when 283 then
# line 1 "NONE"
                begin
        case  @act
        when 140 then
        begin begin p = (( @te))-1; end

        if @version < 27
          # Ruby before 2.7 doesn't support comments before leading dot.
          # If a line after "a" starts with a comment then "a" is a self-contained statement.
          # So in that case we emit a special tNL token and start reading the
          # next line as a separate statement.
          #
          # Note: block comments before leading dot are not supported on any version of Ruby.
          emit(:tNL, nil, @newline_s, @newline_s + 1)
          p = p - 1;  @cs = 710;        begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

        end
      end
        when 144 then
        begin begin p = (( @te))-1; end
 emit(:tNL, nil, @newline_s, @newline_s + 1)
           p = p - 1;  @cs = 710;       begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end
 end
end 
                        end
        when 284 then
# line 2048 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        emit_comment(@eq_begin_s, @te)
                begin
                 @cs = (@cs_before_block_comment)
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 285 then
# line 2053 "lib/parser/lexer.rl"
                begin
 @te = p+1
                end
        when 286 then
# line 2048 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        emit_comment(@eq_begin_s, @te)
                begin
                 @cs = (@cs_before_block_comment)
                _trigger_goto = true
                _goto_level = _again
                break
        end

       end
                end
        when 287 then
# line 2056 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin 
        diagnostic :fatal, :embedded_document, nil,
                   range(@eq_begin_s, @eq_begin_s + '=begin'.length)
       end
                end
        when 288 then
# line 2066 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  @eq_begin_s = @ts
                begin
                 @cs = 704
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 289 then
# line 2070 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  p = pe - 3  end
                end
        when 290 then
# line 2073 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin  cmd_state = true; p = p - 1;    begin
                 @cs = 508
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 291 then
# line 566 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
    # Sit at EOF indefinitely. #advance would return $eof each time.
    # This allows to feed the lexer more data if needed; this is only used
    # in tests.
    #
    # Note that this action is not embedded into e_eof like e_nl and e_bs
    # below. This is due to the fact that scanner state at EOF is observed
    # by tests, and encapsulating it in a rule would break the introspection.
    p = p - 1;  begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

   end
                end
        when 292 then
# line 2063 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1;              end
        when 293 then
# line 2066 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  @eq_begin_s = @ts
                begin
                 @cs = 704
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 294 then
# line 2073 "lib/parser/lexer.rl"
                begin
 @te = p
p = p - 1; begin  cmd_state = true; p = p - 1;  begin
                 @cs = 508
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 295 then
# line 2073 "lib/parser/lexer.rl"
                begin
 begin p = (( @te))-1; end
 begin  cmd_state = true; p = p - 1;    begin
                 @cs = 508
                _trigger_goto = true
                _goto_level = _again
                break
        end
  end
                end
        when 296 then
# line 2080 "lib/parser/lexer.rl"
                begin
 @te = p+1
 begin 
        p, next_state = @strings.advance(p)

        p = p - 1; # Ragel will do `p += 1` to consume input, prevent it
         @cs = (next_state);
                begin
                p += 1
                _trigger_goto = true
                _goto_level = _out
                break
        end

       end
                end
# line 12621 "lib/parser/lexer-F0.rb"
                        end # action switch
                end
        end
        if _trigger_goto
                next
        end
        end
        if _goto_level <= _again
        _acts = _lex_to_state_actions[ @cs]
        _nacts = _lex_actions[_acts]
        _acts += 1
        while _nacts > 0
                _nacts -= 1
                _acts += 1
                case _lex_actions[_acts - 1]
        when 51 then
# line 1 "NONE"
                begin
 @ts = nil;             end
# line 12641 "lib/parser/lexer-F0.rb"
                end # to state action switch
        end
        if _trigger_goto
                next
        end
        if  @cs == 0
                _goto_level = _out
                next
        end
        p += 1
        if p != pe
                _goto_level = _resume
                next
        end
        end
        if _goto_level <= _test_eof
        if p == eof
        if _lex_eof_trans[ @cs] > 0
                _trans = _lex_eof_trans[ @cs] - 1;
                _goto_level = _eof_trans
                next;
        end
        end
        end
        if _goto_level <= _out
                break
        end
        end
        end

# line 281 "lib/parser/lexer.rl"
    # %

    # Ragel creates a local variable called `testEof` but it doesn't use
    # it in any assignment. This dead code is here to swallow the warning.
    # It has no runtime cost because Ruby doesn't produce any instructions from it.
    if false
      testEof
    end

    @p = p

    if @token_queue.any?
      @token_queue.shift
    elsif @cs == klass.lex_error
      [ false, [ '$error'.freeze, range(p - 1, p) ] ]
    else
      eof = @source_pts.size
      [ false, [ '$eof'.freeze,   range(eof, eof) ] ]
    end
  end
dedent_level() click to toggle source
# File lib/parser/lexer-F0.rb, line 8537
def dedent_level
  @strings.dedent_level
end
encoding() click to toggle source
# File lib/parser/lexer-F0.rb, line 8490
def encoding
  @source_buffer.source.encoding
end
pop_cmdarg() click to toggle source
# File lib/parser/lexer-F0.rb, line 8524
def pop_cmdarg
  @cmdarg = @cmdarg_stack.pop
end
pop_cond() click to toggle source
# File lib/parser/lexer-F0.rb, line 8533
def pop_cond
  @cond = @cond_stack.pop
end
push_cmdarg() click to toggle source
# File lib/parser/lexer-F0.rb, line 8519
def push_cmdarg
  @cmdarg_stack.push(@cmdarg)
  @cmdarg = StackState.new("cmdarg.#{@cmdarg_stack.count}")
end
push_cond() click to toggle source
# File lib/parser/lexer-F0.rb, line 8528
def push_cond
  @cond_stack.push(@cond)
  @cond = StackState.new("cond.#{@cond_stack.count}")
end
reset(reset_state=true) click to toggle source
# File lib/parser/lexer-F0.rb, line 8413
def reset(reset_state=true)
  # Ragel state:
  if reset_state
    # Unit tests set state prior to resetting lexer.
    @cs     = self.class.lex_en_line_begin

    @cond   = StackState.new('cond')
    @cmdarg = StackState.new('cmdarg')
    @cond_stack   = []
    @cmdarg_stack = []
  end

  @force_utf32   = false # Set to true by some tests

  @source_pts    = nil # @source as a codepoint array

  @p             = 0   # stream position (saved manually in #advance)
  @ts            = nil # token start
  @te            = nil # token end
  @act           = 0   # next action

  @stack         = []  # state stack
  @top           = 0   # state stack top pointer

  # Lexer state:
  @token_queue   = []

  @eq_begin_s    = nil # location of last encountered =begin
  @sharp_s       = nil # location of last encountered #

  @newline_s     = nil # location of last encountered newline

  @num_base      = nil # last numeric base
  @num_digits_s  = nil # starting position of numeric digits
  @num_suffix_s  = nil # starting position of numeric suffix
  @num_xfrm      = nil # numeric suffix-induced transformation

  # Ruby 1.9 ->() lambdas emit a distinct token if do/{ is
  # encountered after a matching closing parenthesis.
  @paren_nest    = 0
  @lambda_stack  = []

  # If the lexer is in `command state' (aka expr_value)
  # at the entry to #advance, it will transition to expr_cmdarg
  # instead of expr_arg at certain points.
  @command_start = true

  # State before =begin / =end block comment
  @cs_before_block_comment = self.class.lex_en_line_begin

  @strings = Parser::LexerStrings.new(self, @version)
end
source_buffer=(source_buffer) click to toggle source
# File lib/parser/lexer-F0.rb, line 8466
def source_buffer=(source_buffer)
  @source_buffer = source_buffer

  if @source_buffer
    source = @source_buffer.source

    if source.encoding == Encoding::UTF_8
      @source_pts = source.unpack('U*')
    else
      @source_pts = source.unpack('C*')
    end

    if @source_pts[0] == 0xfeff
      # Skip byte order mark.
      @p = 1
    end
  else
    @source_pts = nil
  end

  @strings.source_buffer = @source_buffer
  @strings.source_pts = @source_pts
end
state() click to toggle source
# File lib/parser/lexer-F0.rb, line 8511
def state
  LEX_STATES.invert.fetch(@cs, @cs)
end
state=(state) click to toggle source
# File lib/parser/lexer-F0.rb, line 8515
def state=(state)
  @cs = LEX_STATES.fetch(state)
end

Protected Instance Methods

arg_or_cmdarg(cmd_state) click to toggle source
# File lib/parser/lexer-F0.rb, line 12738
def arg_or_cmdarg(cmd_state)
  if cmd_state
    self.class.lex_en_expr_cmdarg
  else
    self.class.lex_en_expr_arg
  end
end
check_ambiguous_slash(tm) click to toggle source
# File lib/parser/lexer-F0.rb, line 12800
def check_ambiguous_slash(tm)
  if tok(tm, tm + 1) == '/'.freeze
    # Ambiguous regexp literal.
    if @version < 30
      diagnostic :warning, :ambiguous_literal, nil, range(tm, tm + 1)
    else
      diagnostic :warning, :ambiguous_regexp, nil, range(tm, tm + 1)
    end
  end
end
diagnostic(type, reason, arguments=nil, location=range, highlights=[]) click to toggle source
# File lib/parser/lexer-F0.rb, line 12762
def diagnostic(type, reason, arguments=nil, location=range, highlights=[])
  @diagnostics.process(
      Parser::Diagnostic.new(type, reason, arguments, location, highlights))
end
e_lbrace() click to toggle source
# File lib/parser/lexer-F0.rb, line 12768
def e_lbrace
  @cond.push(false); @cmdarg.push(false)

  current_literal = @strings.literal
  if current_literal
    current_literal.start_interp_brace
  end
end
emit(type, value = tok, s = @ts, e = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12712
def emit(type, value = tok, s = @ts, e = @te)
  token = [ type, [ value, range(s, e) ] ]

  @token_queue.push(token)

  @tokens.push(token) if @tokens

  token
end
emit_class_var(ts = @ts, te = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12821
def emit_class_var(ts = @ts, te = @te)
  if tok(ts, te) =~ /^@@[0-9]/
    diagnostic :error, :cvar_name, { :name => tok(ts, te) }
  end

  emit(:tCVAR, tok(ts, te), ts, te)
end
emit_colon_with_digits(p, tm, diag_msg) click to toggle source
# File lib/parser/lexer-F0.rb, line 12849
def emit_colon_with_digits(p, tm, diag_msg)
  if @version >= 27
    diagnostic :error, diag_msg, { name: tok(tm, @te) }, range(tm, @te)
  else
    emit(:tCOLON, tok(@ts, @ts + 1), @ts, @ts + 1)
    p = @ts
  end
  p
end
emit_comment(s = @ts, e = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12746
def emit_comment(s = @ts, e = @te)
  if @comments
    @comments.push(Parser::Source::Comment.new(range(s, e)))
  end

  if @tokens
    @tokens.push([ :tCOMMENT, [ tok(s, e), range(s, e) ] ])
  end

  nil
end
emit_comment_from_range(p, pe) click to toggle source
# File lib/parser/lexer-F0.rb, line 12758
def emit_comment_from_range(p, pe)
  emit_comment(@sharp_s, p == pe ? p - 2 : p)
end
emit_do(do_block=false) click to toggle source
# File lib/parser/lexer-F0.rb, line 12728
def emit_do(do_block=false)
  if @cond.active?
    emit(:kDO_COND, 'do'.freeze)
  elsif @cmdarg.active? || do_block
    emit(:kDO_BLOCK, 'do'.freeze)
  else
    emit(:kDO, 'do'.freeze)
  end
end
emit_global_var(ts = @ts, te = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12811
def emit_global_var(ts = @ts, te = @te)
  if tok(ts, te) =~ /^\$([1-9][0-9]*)$/
    emit(:tNTH_REF, tok(ts + 1, te).to_i, ts, te)
  elsif tok =~ /^\$([&`'+])$/
    emit(:tBACK_REF, tok(ts, te), ts, te)
  else
    emit(:tGVAR, tok(ts, te), ts, te)
  end
end
emit_instance_var(ts = @ts, te = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12829
def emit_instance_var(ts = @ts, te = @te)
  if tok(ts, te) =~ /^@[0-9]/
    diagnostic :error, :ivar_name, { :name => tok(ts, te) }
  end

  emit(:tIVAR, tok(ts, te), ts, te)
end
emit_rbrace_rparen_rbrack() click to toggle source
# File lib/parser/lexer-F0.rb, line 12837
def emit_rbrace_rparen_rbrack
  emit_table(PUNCTUATION)

  if @version < 24
    @cond.lexpop
    @cmdarg.lexpop
  else
    @cond.pop
    @cmdarg.pop
  end
end
emit_singleton_class() click to toggle source
# File lib/parser/lexer-F0.rb, line 12859
def emit_singleton_class
  emit(:kCLASS, 'class'.freeze, @ts, @ts + 5)
  emit(:tLSHFT, '<<'.freeze,    @te - 2, @te)
end
emit_table(table, s = @ts, e = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12722
def emit_table(table, s = @ts, e = @te)
  value = tok(s, e)

  emit(table[value], value, s, e)
end
numeric_literal_int() click to toggle source
# File lib/parser/lexer-F0.rb, line 12777
def numeric_literal_int
  digits = tok(@num_digits_s, @num_suffix_s)

  if digits.end_with? '_'.freeze
    diagnostic :error, :trailing_in_number, { :character => '_'.freeze },
               range(@te - 1, @te)
  elsif digits.empty? && @num_base == 8 && version?(18)
    # 1.8 did not raise an error on 0o.
    digits = '0'.freeze
  elsif digits.empty?
    diagnostic :error, :empty_numeric
  elsif @num_base == 8 && (invalid_idx = digits.index(/[89]/))
    invalid_s = @num_digits_s + invalid_idx
    diagnostic :error, :invalid_octal, nil,
               range(invalid_s, invalid_s + 1)
  end
  digits
end
on_newline(p) click to toggle source
# File lib/parser/lexer-F0.rb, line 12796
def on_newline(p)
  @strings.on_newline(p)
end
range(s = @ts, e = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12708
def range(s = @ts, e = @te)
  Parser::Source::Range.new(@source_buffer, s, e)
end
stack_pop() click to toggle source
# File lib/parser/lexer-F0.rb, line 12699
def stack_pop
  @top -= 1
  @stack[@top]
end
tok(s = @ts, e = @te) click to toggle source
# File lib/parser/lexer-F0.rb, line 12704
def tok(s = @ts, e = @te)
  @source_buffer.slice(s, e - s)
end
version?(*versions) click to toggle source
# File lib/parser/lexer-F0.rb, line 12695
def version?(*versions)
  versions.include?(@version)
end