class WikiThat::Lexer
Lexers are disposable objects for translate a Mediawiki document to a Token
list. @author Bryan T. Meyers
Attributes
The output of the translation to HTML
Public Class Methods
Create a new WikiThat::Lexer
@param [String] doc the MediaWiki document
@returns [WikiThat::Lexer] a newly configured Lexer
# File lib/wiki-that/lexer/lexer.rb, line 53 def initialize(doc) @doc = doc @index = 0 @result = [] end
Public Instance Methods
Move the lexer tape forward @param [Integer] dist how many steps to move forward, default 1
# File lib/wiki-that/lexer/lexer.rb, line 91 def advance(dist = 1) @index += dist end
Append Tokens to the result @param [Token] toks the Tokens to append
# File lib/wiki-that/lexer/lexer.rb, line 99 def append(toks) @result.push(*toks) end
Get the character at the pointer @return [String] the character being pointed at
# File lib/wiki-that/lexer/lexer.rb, line 107 def current if end? return '' end @doc[@index] end
Check if the end of the tape has been reached @returns [Boolean] True if at or beyond the end
# File lib/wiki-that/lexer/lexer.rb, line 160 def end? @index >= @doc.length end
Translate the MediaWiki document into a Token
list
@returns [Array] the resulting Token
List
# File lib/wiki-that/lexer/lexer.rb, line 64 def lex until end? case current when *BREAK_SPECIAL lex_break when *HEADER_SPECIAL lex_header when *RULE_SPECIAL lex_horizontal_rule when *LIST_SPECIAL lex_list when *TABLE_SPECIAL lex_table when *TOC_SPECIAL lex_toc when *TAG_SPECIAL lex_tag else lex_text end end end
Determine if the current character matches any in a list @param [Array] chars a list of characters to check @return [Boolean] True iff the current character is in the list
# File lib/wiki-that/lexer/lexer.rb, line 119 def match?(chars) if end? return false end chars.each do |char| if current == char return true end end false end
Determine if the current character does not match any in a list @param [Array] chars a list of characters to check @return [Boolean] True iff the current character is not in the list
# File lib/wiki-that/lexer/lexer.rb, line 136 def not_match?(chars) if end? return true end chars.each do |char| if current == char return false end end true end
Move the lexer tape backward @param [Integer] dist how many steps to move backward, default 1
# File lib/wiki-that/lexer/lexer.rb, line 152 def rewind(dist = 1) @index -= dist end