public abstract class Lexer extends BaseRecognizer implements TokenSource
Modifier and Type | Field and Description |
---|---|
protected CharStream |
input
Where is the lexer drawing characters from?
|
DEFAULT_TOKEN_CHANNEL, HIDDEN, INITIAL_FOLLOW_STACK_SIZE, MEMO_RULE_FAILED, MEMO_RULE_UNKNOWN, NEXT_TOKEN_RULE_NAME, state
Constructor and Description |
---|
Lexer() |
Lexer(CharStream input) |
Lexer(CharStream input,
RecognizerSharedState state) |
Modifier and Type | Method and Description |
---|---|
Token |
emit()
The standard method called to automatically emit a token at the
outermost lexical rule.
|
void |
emit(Token token)
Currently does not support multiple emits per nextToken invocation
for efficiency reasons.
|
String |
getCharErrorDisplay(int c) |
int |
getCharIndex()
What is the index of the current character of lookahead?
|
int |
getCharPositionInLine() |
CharStream |
getCharStream() |
String |
getErrorMessage(RecognitionException e,
String[] tokenNames)
What error message should be generated for the various
exception types?
Not very object-oriented code, but I like having all error message
generation within one method rather than spread among all of the
exception classes.
|
int |
getLine() |
String |
getSourceName()
Where are you getting tokens from? normally the implication will simply
ask lexers input stream.
|
String |
getText()
Return the text matched so far for the current token or any
text override.
|
void |
match(int c) |
void |
match(String s) |
void |
matchAny() |
void |
matchRange(int a,
int b) |
abstract void |
mTokens()
This is the lexer entry point that sets instance var 'token'
|
Token |
nextToken()
Return a token from this source; i.e., match a token on the char
stream.
|
void |
recover(RecognitionException re)
Lexers can normally match any char in it's vocabulary after matching
a token, so do the easy thing and just kill a character and hope
it all works out.
|
void |
reportError(RecognitionException e)
Report a recognition problem.
|
void |
reset()
reset the parser's state; subclasses must rewinds the input stream
|
void |
setCharStream(CharStream input)
Set the char stream and reset the lexer
|
void |
setText(String text)
Set the complete text of this token; it wipes any previous
changes to the text.
|
void |
skip()
Instruct the lexer to skip creating a token for current lexer rule
and look for another token.
|
void |
traceIn(String ruleName,
int ruleIndex) |
void |
traceOut(String ruleName,
int ruleIndex) |
alreadyParsedRule, beginResync, combineFollows, computeContextSensitiveRuleFOLLOW, computeErrorRecoverySet, consumeUntil, consumeUntil, displayRecognitionError, emitErrorMessage, endResync, failed, getBacktrackingLevel, getCurrentInputSymbol, getErrorHeader, getGrammarFileName, getMissingSymbol, getNumberOfSyntaxErrors, getRuleInvocationStack, getRuleInvocationStack, getRuleMemoization, getRuleMemoizationCacheSize, getTokenErrorDisplay, getTokenNames, match, matchAny, memoize, mismatchIsMissingToken, mismatchIsUnwantedToken, pushFollow, recover, recoverFromMismatchedSet, recoverFromMismatchedToken, setBacktrackingLevel, toStrings, traceIn, traceOut
protected CharStream input
public Lexer()
public Lexer(CharStream input)
public Lexer(CharStream input, RecognizerSharedState state)
public void reset()
BaseRecognizer
reset
in class BaseRecognizer
public Token nextToken()
nextToken
in interface TokenSource
public void skip()
public abstract void mTokens() throws RecognitionException
RecognitionException
public void setCharStream(CharStream input)
public CharStream getCharStream()
public String getSourceName()
TokenSource
getSourceName
in interface TokenSource
getSourceName
in class BaseRecognizer
public void emit(Token token)
public Token emit()
public void match(String s) throws MismatchedTokenException
MismatchedTokenException
public void matchAny()
public void match(int c) throws MismatchedTokenException
MismatchedTokenException
public void matchRange(int a, int b) throws MismatchedRangeException
MismatchedRangeException
public int getLine()
public int getCharPositionInLine()
public int getCharIndex()
public String getText()
public void setText(String text)
public void reportError(RecognitionException e)
BaseRecognizer
reportError
in class BaseRecognizer
public String getErrorMessage(RecognitionException e, String[] tokenNames)
BaseRecognizer
getErrorMessage
in class BaseRecognizer
public String getCharErrorDisplay(int c)
public void recover(RecognitionException re)
public void traceIn(String ruleName, int ruleIndex)
public void traceOut(String ruleName, int ruleIndex)
Copyright © 2020. All rights reserved.