Package org.luwrain.nlp.ru

Class AbstractTokenizer

java.lang.Object

org.luwrain.nlp.ru.AbstractTokenizer

Direct Known Subclasses:

ReaderTokenizer

public abstract class AbstractTokenizer
extends Object

Field Summary

Fields

Modifier and Type	Field	Description
protected final List<Token>	output

Constructor Summary

Constructors

Constructor	Description
AbstractTokenizer()

Method Summary

Modifier and Type	Method	Description
abstract void	backCh(char ch)
Token[]	getOutput()
abstract boolean	hasCh()
void	tokenize()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

output

protected final List<Token> output

Constructor Details

AbstractTokenizer

public AbstractTokenizer()

Method Details

hasCh

public abstract boolean hasCh()

backCh

public abstract void backCh(char ch)

tokenize

public void tokenize()

getOutput

public Token[] getOutput()