com.sun.speech.freetts
Interface Tokenizer

All Known Implementing Classes:

TokenizerImpl

public interface Tokenizer

Chops a string or text file into Token instances.

Method Summary

java.lang.String	getErrorDescription() If hasErrors returns true, returns a description of the error encountered.
Token	getNextToken() Returns the next token.
boolean	hasErrors() Returns true if there were errors while reading tokens.
boolean	hasMoreTokens() Returns true if there are more tokens, false otherwise.
boolean	isBreak() Determines if the current token should start a new sentence.
void	setInputReader(java.io.Reader reader) Sets the input reader.
void	setInputText(java.lang.String textToTokenize) Sets the text to be tokenized by this tokenizer.
void	setPostpunctuationSymbols(java.lang.String symbols) Sets the postpunctuation symbols of this Tokenizer to the given symbols.
void	setPrepunctuationSymbols(java.lang.String symbols) Sets the prepunctuation symbols of this Tokenizer to the given symbols.
void	setSingleCharSymbols(java.lang.String symbols) Sets the single character symbols of this Tokenizer to the given symbols.
void	setWhitespaceSymbols(java.lang.String symbols) Sets the whitespace symbols of this Tokenizer to the given symbols.

Method Detail

setInputText

void setInputText(java.lang.String textToTokenize)

Sets the text to be tokenized by this tokenizer.

Parameters:

textToTokenize - the text to tokenize

setInputReader

void setInputReader(java.io.Reader reader)

Sets the input reader.

Parameters:

reader - the input source

getNextToken

Token getNextToken()

Returns the next token.

Returns:

the next token if it exists; otherwise null

hasMoreTokens

boolean hasMoreTokens()

Returns true if there are more tokens, false otherwise.

Returns:

true if there are more tokens; otherwise false

hasErrors

boolean hasErrors()

Returns true if there were errors while reading tokens.

Returns:

true if there were errors; otherwise false

getErrorDescription

java.lang.String getErrorDescription()

If hasErrors returns true, returns a description of the error encountered. Otherwise returns null.

Returns:

a description of the last error that occurred

setWhitespaceSymbols

void setWhitespaceSymbols(java.lang.String symbols)

Sets the whitespace symbols of this Tokenizer to the given symbols.

Parameters:

symbols - the whitespace symbols

setSingleCharSymbols

void setSingleCharSymbols(java.lang.String symbols)

Sets the single character symbols of this Tokenizer to the given symbols.

Parameters:

symbols - the single character symbols

setPrepunctuationSymbols

void setPrepunctuationSymbols(java.lang.String symbols)

Sets the prepunctuation symbols of this Tokenizer to the given symbols.

Parameters:

symbols - the prepunctuation symbols

setPostpunctuationSymbols

void setPostpunctuationSymbols(java.lang.String symbols)

Sets the postpunctuation symbols of this Tokenizer to the given symbols.

Parameters:

symbols - the postpunctuation symbols

isBreak

boolean isBreak()

Determines if the current token should start a new sentence.

Returns:

true if a new sentence should be started