Package org.apache.lucene.classification
Class SimpleNaiveBayesClassifier
- java.lang.Object
-
- org.apache.lucene.classification.SimpleNaiveBayesClassifier
-
- All Implemented Interfaces:
Classifier<BytesRef>
- Direct Known Subclasses:
CachingNaiveBayesClassifier,SimpleNaiveBayesDocumentClassifier
public class SimpleNaiveBayesClassifier extends Object implements Classifier<BytesRef>
A simplistic Lucene based NaiveBayes classifier, seehttp://en.wikipedia.org/wiki/Naive_Bayes_classifier- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
-
Field Summary
Fields Modifier and Type Field Description protected AnalyzeranalyzerAnalyzerto be used for tokenizing unseen input textprotected StringclassFieldNamename of the field to be used as a class / category outputprotected IndexReaderindexReaderIndexReaderused to access theClassifier's indexprotected IndexSearcherindexSearcherIndexSearcherto run searches on the index for retrieving frequenciesprotected QueryqueryQueryused to eventually filter the document set to be used to classifyprotected String[]textFieldNamesnames of the fields to be used as input text
-
Constructor Summary
Constructors Constructor Description SimpleNaiveBayesClassifier(IndexReader indexReader, Analyzer analyzer, Query query, String classFieldName, String... textFieldNames)Creates a new NaiveBayes classifier.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description ClassificationResult<BytesRef>assignClass(String inputDocument)Assign a class (with score) to the given text Stringprotected List<ClassificationResult<BytesRef>>assignClassNormalizedList(String inputDocument)Calculate probabilities for all classes for a given input textprotected intcountDocsWithClass()count the number of documents in the index having at least a value for the 'class' fieldList<ClassificationResult<BytesRef>>getClasses(String text)Get all the classes (sorted by score, descending) assigned to the given text String.List<ClassificationResult<BytesRef>>getClasses(String text, int max)Get the firstmaxclasses (sorted by score, descending) assigned to the given text String.protected ArrayList<ClassificationResult<BytesRef>>normClassificationResults(List<ClassificationResult<BytesRef>> assignedClasses)Normalize the classification results based on the max score availableprotected String[]tokenize(String text)tokenize aStringon this classifier's text fields and analyzer
-
-
-
Field Detail
-
indexReader
protected final IndexReader indexReader
IndexReaderused to access theClassifier's index
-
textFieldNames
protected final String[] textFieldNames
names of the fields to be used as input text
-
classFieldName
protected final String classFieldName
name of the field to be used as a class / category output
-
indexSearcher
protected final IndexSearcher indexSearcher
IndexSearcherto run searches on the index for retrieving frequencies
-
-
Constructor Detail
-
SimpleNaiveBayesClassifier
public SimpleNaiveBayesClassifier(IndexReader indexReader, Analyzer analyzer, Query query, String classFieldName, String... textFieldNames)
Creates a new NaiveBayes classifier.- Parameters:
indexReader- the reader on the index to be used for classificationanalyzer- anAnalyzerused to analyze unseen textquery- aQueryto eventually filter the docs used for training the classifier, ornullif all the indexed docs should be usedclassFieldName- the name of the field used as the output for the classifier NOTE: must not be havely analyzed as the returned class will be a token indexed for this fieldtextFieldNames- the name of the fields used as the inputs for the classifier, NO boosting supported per field
-
-
Method Detail
-
assignClass
public ClassificationResult<BytesRef> assignClass(String inputDocument) throws IOException
Description copied from interface:ClassifierAssign a class (with score) to the given text String- Specified by:
assignClassin interfaceClassifier<BytesRef>- Parameters:
inputDocument- a String containing text to be classified- Returns:
- a
ClassificationResultholding assigned class of typeTand score - Throws:
IOException- If there is a low-level I/O error.
-
getClasses
public List<ClassificationResult<BytesRef>> getClasses(String text) throws IOException
Description copied from interface:ClassifierGet all the classes (sorted by score, descending) assigned to the given text String.- Specified by:
getClassesin interfaceClassifier<BytesRef>- Parameters:
text- a String containing text to be classified- Returns:
- the whole list of
ClassificationResult, the classes and scores. Returnsnullif the classifier can't make lists. - Throws:
IOException- If there is a low-level I/O error.
-
getClasses
public List<ClassificationResult<BytesRef>> getClasses(String text, int max) throws IOException
Description copied from interface:ClassifierGet the firstmaxclasses (sorted by score, descending) assigned to the given text String.- Specified by:
getClassesin interfaceClassifier<BytesRef>- Parameters:
text- a String containing text to be classifiedmax- the number of return list elements- Returns:
- the whole list of
ClassificationResult, the classes and scores. Cut for "max" number of elements. Returnsnullif the classifier can't make lists. - Throws:
IOException- If there is a low-level I/O error.
-
assignClassNormalizedList
protected List<ClassificationResult<BytesRef>> assignClassNormalizedList(String inputDocument) throws IOException
Calculate probabilities for all classes for a given input text- Parameters:
inputDocument- the input text as aString- Returns:
- a
ListofClassificationResult, one for each existing class - Throws:
IOException- if assigning probabilities fails
-
countDocsWithClass
protected int countDocsWithClass() throws IOExceptioncount the number of documents in the index having at least a value for the 'class' field- Returns:
- the no. of documents having a value for the 'class' field
- Throws:
IOException- if accessing to term vectors or search fails
-
tokenize
protected String[] tokenize(String text) throws IOException
tokenize aStringon this classifier's text fields and analyzer- Parameters:
text- theStringrepresenting an input text (to be classified)- Returns:
- a
Stringarray of the resulting tokens - Throws:
IOException- if tokenization fails
-
normClassificationResults
protected ArrayList<ClassificationResult<BytesRef>> normClassificationResults(List<ClassificationResult<BytesRef>> assignedClasses)
Normalize the classification results based on the max score available- Parameters:
assignedClasses- the list of assigned classes- Returns:
- the normalized results
-
-