Orcus
Loading...
Searching...
No Matches
orcus::detail::thread::parser_token_buffer< _TokensT > Class Template Reference

#include <parser_token_buffer.hpp>

Public Member Functions

 parser_token_buffer (size_t min_token_size, size_t max_token_size)
 
void check_and_notify (tokens_type &parser_tokens)
 
void notify_and_finish (tokens_type &parser_tokens)
 
void abort ()
 
bool next_tokens (tokens_type &tokens)
 
size_t token_size_threshold () const
 

Detailed Description

template<typename _TokensT>
class orcus::detail::thread::parser_token_buffer< _TokensT >

Class that manages synchronization of parser tokens used in multi-threaded parsers.

Member Function Documentation

◆ check_and_notify()

template<typename _TokensT>
void orcus::detail::thread::parser_token_buffer< _TokensT >::check_and_notify ( tokens_type & parser_tokens)
inline

Check the size of the parser token buffer, and if it exceeds specified threshold, move it to the client buffer.

Call this from the parser thread.

Parameters
parser_tokensparser token buffer.

◆ next_tokens()

template<typename _TokensT>
bool orcus::detail::thread::parser_token_buffer< _TokensT >::next_tokens ( tokens_type & tokens)
inline

Retrieve the tokens currently in the client token buffer.

Call this from the client (non-parser) thread.

Parameters
tokensplace to move the tokens in the client token buffer to.
Returns
true if the parsing is still in progress, therefore more tokens are expected, false if this is the last set of tokens.

◆ notify_and_finish()

template<typename _TokensT>
void orcus::detail::thread::parser_token_buffer< _TokensT >::notify_and_finish ( tokens_type & parser_tokens)
inline

Move the current parser token buffer to the client buffer, and signal the end of parsing.

Call this from the parser thread.

Parameters
parser_tokensparser token buffer.

◆ token_size_threshold()

template<typename _TokensT>
size_t orcus::detail::thread::parser_token_buffer< _TokensT >::token_size_threshold ( ) const
inline

Return the current token size threshold. Call this only after the parsing has finished.

Returns
current token size threshold.