class Antelope::Grammar::Token

Defines a token type for productions/rules.

@abstract This class should be inherited to define a real token.

A base class does not match any token; however, any token can
match the base class.

Attributes

from[R]

The from state that this token is transitioned from. This is the source. This is used in the constructor in order to handle lookahead sets.

@return [Recognizer::State]

id[RW]
name[R]

The name of the token.

@return [Symbol]

to[R]

The to state that this token is transitioned to. This is the destination. This is used in the constructor in order to handle lookahead sets.

@return [Recognizer::State]

type[R]

The type of the token. This is given by a caret argument to the grammar. This is primarily used for generators.

@return [String]

Public Class Methods

new(name, type = nil, id = nil, value = nil) click to toggle source

Initialize.

@param name [Symbol] the name of the token. @param type [String?] the type of the token. For definitions,

this is the given type of the token (for typed language
output).

@param id [String?] the id of the token in the production.

For some languages, this allows references to the token via
the id.

@param value [String?] the value of the token. This is only

used in output representation to the developer.
# File lib/antelope/grammar/token.rb, line 54
def initialize(name, type = nil, id = nil, value = nil)
  @name  = name
  @value = value
  @type  = type
  @id    = id
  @from  = nil
  @to    = nil
end

Public Instance Methods

<=>(other) click to toggle source

Compares this class to any other object. If the other object is a token, it converts both this class and the other object to an array and compares the array. Otherwise, it delegates the comparison.

@param other [Object] the other object to compare. @return [Numeric]

Calls superclass method
# File lib/antelope/grammar/token.rb, line 168
def <=>(other)
  if other.is_a? Token
    to_a <=> other.to_a
  else
    super
  end
end
===(other) click to toggle source

Compares this class and another object, fuzzily. If the other object is a token, it removes the transitions (to and from) on both objects and compares them like that. Otherwise, it delegates the comparison.

@param other [Object] the other object to compare. @return [Boolean] if they are equal.

Calls superclass method
# File lib/antelope/grammar/token.rb, line 185
def ===(other)
  if other.is_a? Token
    without_transitions == other.without_transitions
  else
    super
  end
end
epsilon?() click to toggle source

Whether or not the token is an epsilon token.

@abstract @return [Boolean]

# File lib/antelope/grammar/token.rb, line 85
def epsilon?
  false
end
error?() click to toggle source

Whether or not the token is an error token.

@abstract @return [Boolean]

# File lib/antelope/grammar/token.rb, line 93
def error?
  false
end
from=(state) click to toggle source

Sets the from state of the token and invalidates the cache.

@param state [Recognizer::State] @return [void]

# File lib/antelope/grammar/token.rb, line 101
def from=(state)
  invalidate_cache!
  @from = state
end
hash() click to toggle source

Generates a hash for this class.

@note This is not intended for use. It is only defined to be

compatible with Hashs (and by extension, Sets).

@private @return [Object]

# File lib/antelope/grammar/token.rb, line 214
def hash
  @_hash ||= to_a.hash
end
inspect() click to toggle source

Returns a nice inspect.

@return [String]

# File lib/antelope/grammar/token.rb, line 156
def inspect
  "#<#{self.class} from=#{from.id if from} to=#{to.id if to} " \
    "name=#{name.inspect} value=#{@value.inspect}>"
end
invalidate_cache!() click to toggle source

Invalidates the cache.

@return [void]

# File lib/antelope/grammar/token.rb, line 203
def invalidate_cache!
  @_hash = nil
  @_array = nil
end
nonterminal?() click to toggle source

Whether or not the token is a nonterminal.

@abstract @return [Boolean]

# File lib/antelope/grammar/token.rb, line 77
def nonterminal?
  false
end
terminal?() click to toggle source

Whether or not the token is a terminal.

@abstract @return [Boolean]

# File lib/antelope/grammar/token.rb, line 69
def terminal?
  false
end
to=(state) click to toggle source

Sets the to state of the token and invalidates the cache.

@param state [Recognizer::State] @return [void]

# File lib/antelope/grammar/token.rb, line 110
def to=(state)
  invalidate_cache!
  @to = state
end
to_a() click to toggle source

Creates an array representation of this class.

@note This is not intended for use. It is only defined to

make equality checking easier, and to create a hash.

@private @return [Array<(Recognizer::State, Recognizer::State, Class, Symbol, String?)>]

# File lib/antelope/grammar/token.rb, line 226
def to_a
  @_array ||= [to, from, self.class, name, @value]
end
to_s() click to toggle source

Gives a string representation of the token. The output is formatted like so: ‘<data>[“(” [<from @see to @see name

# File lib/antelope/grammar/token.rb, line 136
def to_s
  buf = if @value
          @value.inspect
        else
          @name.to_s
        end

  if from || to
    buf << '('
    buf << "#{from.id}" if from
    buf << ":#{to.id}"  if to
    buf << ')'
  end

  buf
end
type=(type) click to toggle source

Sets the type of the token and invalidates the cache.

@param type [String] @return [void]

# File lib/antelope/grammar/token.rb, line 119
def type=(type)
  invalidate_cache!
  @type = type
end
without_transitions() click to toggle source

Creates a new token without to or from states.

@return [Token]

# File lib/antelope/grammar/token.rb, line 196
def without_transitions
  self.class.new(name, @type, @id, @value)
end