org.apache.lucene.analysis: public class: WhitespaceTokenizer

org.apache.lucene.analysis
public class: WhitespaceTokenizer [javadoc | source]

java.lang.Object
   org.apache.lucene.util.AttributeSource
      org.apache.lucene.analysis.TokenStream
         org.apache.lucene.analysis.Tokenizer
            org.apache.lucene.analysis.CharTokenizer
               org.apache.lucene.analysis.WhitespaceTokenizer

All Implemented Interfaces:
Closeable

A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

Fields inherited from org.apache.lucene.analysis.Tokenizer:
input

Constructor:
public WhitespaceTokenizer(Reader in) { super(in); } Construct a new WhitespaceTokenizer.
public WhitespaceTokenizer(AttributeSource source, Reader in) { super(source, in); } Construct a new WhitespaceTokenizer using a given AttributeSource .
public WhitespaceTokenizer(AttributeFactory factory, Reader in) { super(factory, in); } Construct a new WhitespaceTokenizer using a given org.apache.lucene.util.AttributeSource.AttributeFactory .

Constructor:

 public WhitespaceTokenizer(Reader in) {
    super(in);
}

Construct a new WhitespaceTokenizer.

 public WhitespaceTokenizer(AttributeSource source,
    Reader in) {
    super(source, in);
}

AttributeSource

 public WhitespaceTokenizer(AttributeFactory factory,
    Reader in) {
    super(factory, in);
}

org.apache.lucene.util.AttributeSource.AttributeFactory

Method from org.apache.lucene.analysis.WhitespaceTokenizer Summary:
isTokenChar

Methods from org.apache.lucene.analysis.CharTokenizer:
end, incrementToken, isTokenChar, normalize, reset

Methods from org.apache.lucene.analysis.Tokenizer:
close, correctOffset, reset

Methods from org.apache.lucene.analysis.TokenStream:
close, end, incrementToken, reset

Methods from org.apache.lucene.util.AttributeSource:
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, restoreState, toString

Methods from java.lang.Object:
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method from org.apache.lucene.analysis.WhitespaceTokenizer Detail:
protected boolean isTokenChar(char c) { return !Character.isWhitespace(c); } Collects only characters which do not satisfy Character#isWhitespace(char) .

Method from org.apache.lucene.analysis.WhitespaceTokenizer Detail:

 protected boolean isTokenChar(char c) {
    return !Character.isWhitespace(c);
}

Character#isWhitespace(char)