class Antelope::Grammar::Token
Defines a token type for productions/rules.
@abstract This class should be inherited to define a real token.
A base class does not match any token; however, any token can match the base class.
Attributes
The from state that this token is transitioned from. This is the source. This is used in the constructor in order to handle lookahead sets.
@return [Recognizer::State]
The name of the token.
@return [Symbol]
The to state that this token is transitioned to. This is the destination. This is used in the constructor in order to handle lookahead sets.
@return [Recognizer::State]
The type of the token. This is given by a caret argument to the grammar. This is primarily used for generators.
@return [String]
Public Class Methods
Initialize.
@param name [Symbol] the name of the token. @param type [String?] the type of the token. For definitions,
this is the given type of the token (for typed language output).
@param id [String?] the id of the token in the production.
For some languages, this allows references to the token via the id.
@param value [String?] the value of the token. This is only
used in output representation to the developer.
# File lib/antelope/grammar/token.rb, line 54 def initialize(name, type = nil, id = nil, value = nil) @name = name @value = value @type = type @id = id @from = nil @to = nil end
Public Instance Methods
Compares this class to any other object. If the other object is a token, it converts both this class and the other object to an array and compares the array. Otherwise, it delegates the comparison.
@param other [Object] the other object to compare. @return [Numeric]
# File lib/antelope/grammar/token.rb, line 168 def <=>(other) if other.is_a? Token to_a <=> other.to_a else super end end
Compares this class and another object, fuzzily. If the other object is a token, it removes the transitions (to and from) on both objects and compares them like that. Otherwise, it delegates the comparison.
@param other [Object] the other object to compare. @return [Boolean] if they are equal.
# File lib/antelope/grammar/token.rb, line 185 def ===(other) if other.is_a? Token without_transitions == other.without_transitions else super end end
Whether or not the token is an epsilon token.
@abstract @return [Boolean]
# File lib/antelope/grammar/token.rb, line 85 def epsilon? false end
Whether or not the token is an error token.
@abstract @return [Boolean]
# File lib/antelope/grammar/token.rb, line 93 def error? false end
Sets the from state of the token and invalidates the cache.
@param state [Recognizer::State] @return [void]
# File lib/antelope/grammar/token.rb, line 101 def from=(state) invalidate_cache! @from = state end
Generates a hash for this class.
@note This is not intended for use. It is only defined to be
compatible with Hashs (and by extension, Sets).
@private @return [Object]
# File lib/antelope/grammar/token.rb, line 214 def hash @_hash ||= to_a.hash end
Returns a nice inspect.
@return [String]
# File lib/antelope/grammar/token.rb, line 156 def inspect "#<#{self.class} from=#{from.id if from} to=#{to.id if to} " \ "name=#{name.inspect} value=#{@value.inspect}>" end
Invalidates the cache.
@return [void]
# File lib/antelope/grammar/token.rb, line 203 def invalidate_cache! @_hash = nil @_array = nil end
Whether or not the token is a nonterminal.
@abstract @return [Boolean]
# File lib/antelope/grammar/token.rb, line 77 def nonterminal? false end
Whether or not the token is a terminal.
@abstract @return [Boolean]
# File lib/antelope/grammar/token.rb, line 69 def terminal? false end
Sets the to state of the token and invalidates the cache.
@param state [Recognizer::State] @return [void]
# File lib/antelope/grammar/token.rb, line 110 def to=(state) invalidate_cache! @to = state end
Creates an array representation of this class.
@note This is not intended for use. It is only defined to
make equality checking easier, and to create a hash.
@private @return [Array<(Recognizer::State, Recognizer::State, Class, Symbol, String?)>]
# File lib/antelope/grammar/token.rb, line 226 def to_a @_array ||= [to, from, self.class, name, @value] end
Gives a string representation of the token. The output is formatted like so: ‘<data>[“(” [<from
@see to
@see name
# File lib/antelope/grammar/token.rb, line 136 def to_s buf = if @value @value.inspect else @name.to_s end if from || to buf << '(' buf << "#{from.id}" if from buf << ":#{to.id}" if to buf << ')' end buf end
Sets the type of the token and invalidates the cache.
@param type [String] @return [void]
# File lib/antelope/grammar/token.rb, line 119 def type=(type) invalidate_cache! @type = type end
Creates a new token without to or from states.
@return [Token]
# File lib/antelope/grammar/token.rb, line 196 def without_transitions self.class.new(name, @type, @id, @value) end