You are looking at documentation for an older release.
Not what you want? See the
current release documentation.
Thai Tokenizeredit
A tokenizer of type thai
that segments Thai text into words. This tokenizer
uses the built-in Thai segmentation algorithm included with Java to divide
up Thai text. Text in other languages in general will be treated the same
as standard
.