module Company

BasicTokenizer breaks given strings to a set of tokens. As tokens are regarded the words and the sequences of

the numbers the string contains.