Is this Token (or more generally Observation) a member of a phrase in the lexicon (including single-word phrases)? The query.
Is the input String in the lexicon.
Is this single word in the lexicon? The input String will not be processed by tokenizer, but will be processed by the lemmatizer.
Is the pre-tokenized sequence of words in the lexicon? The input words are expected to already be processed by the lemmatizer.
Is the pre-tokenized sequence of words in the lexicon? Each of the input words will be processed by the lemmatizer.
The string lemmatizer that simplifies lexicon entries and queries before searching for a match.
An identifier for this lexicon, suitable for adding as a category to a FeatureVectorVariable[String].
The string segmenter that breaks a lexicon entries and queries into (potentially) multi-word phrases.