module Company
BasicTokenizer breaks given strings to a set of tokens. As tokens are regarded the words and the sequences of
the numbers the string contains.
BasicTokenizer breaks given strings to a set of tokens. As tokens are regarded the words and the sequences of
the numbers the string contains.