.process_vocab {wordpiece} | R Documentation |
Process a Vocabulary for Tokenization
Description
Process a Vocabulary for Tokenization
Usage
.process_vocab(v)
## Default S3 method:
.process_vocab(v)
## S3 method for class 'wordpiece_vocabulary'
.process_vocab(v)
## S3 method for class 'character'
.process_vocab(v)
Arguments
v |
An object of class |
Value
A character vector of tokens for tokenization.
[Package wordpiece version 2.1.3 Index]