Class Lucene99ScalarQuantizedVectorsWriter
java.lang.Object
org.apache.lucene.codecs.KnnVectorsWriter
org.apache.lucene.codecs.hnsw.FlatVectorsWriter
org.apache.lucene.codecs.lucene99.Lucene99ScalarQuantizedVectorsWriter
- All Implemented Interfaces:
Closeable,AutoCloseable,Accountable
Writes quantized vector values and metadata to index segments.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.codecs.KnnVectorsWriter
KnnVectorsWriter.MergedVectorValues -
Field Summary
Fields inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsWriter
vectorsScorerFields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE -
Constructor Summary
ConstructorsConstructorDescriptionLucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, byte bits, boolean compress, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer) Lucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer) -
Method Summary
Modifier and TypeMethodDescriptionAdd a new field for indexingvoidclose()voidfinish()Called once at the end before closevoidflush(int maxDoc, Sorter.DocMap sortMap) Flush all buffered data on disk *static ScalarQuantizermergeAndRecalculateQuantiles(MergeState mergeState, FieldInfo fieldInfo, Float confidenceInterval, byte bits) Merges the quantiles of the segments and recalculates the quantiles if necessary.voidmergeOneField(FieldInfo fieldInfo, MergeState mergeState) Write field for mergingmergeOneFieldToIndex(FieldInfo fieldInfo, MergeState mergeState) Write the field for merging, providing a scorer over the newly merged flat vectors.longReturn the memory usage of this object in bytes.static DocsWithFieldSetwriteQuantizedVectorData(IndexOutput output, QuantizedByteVectorValues quantizedByteVectorValues, byte bits, boolean compress) Writes the vector values to the output and returns a set of documents that contains vectors.Methods inherited from class org.apache.lucene.codecs.hnsw.FlatVectorsWriter
getFlatVectorScorerMethods inherited from class org.apache.lucene.codecs.KnnVectorsWriter
mapOldOrdToNewOrd, mergeMethods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface org.apache.lucene.util.Accountable
getChildResources
-
Constructor Details
-
Lucene99ScalarQuantizedVectorsWriter
public Lucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer) throws IOException - Throws:
IOException
-
Lucene99ScalarQuantizedVectorsWriter
public Lucene99ScalarQuantizedVectorsWriter(SegmentWriteState state, Float confidenceInterval, byte bits, boolean compress, FlatVectorsWriter rawVectorDelegate, FlatVectorsScorer scorer) throws IOException - Throws:
IOException
-
-
Method Details
-
addField
Description copied from class:FlatVectorsWriterAdd a new field for indexing- Specified by:
addFieldin classFlatVectorsWriter- Parameters:
fieldInfo- fieldInfo of the field to add- Returns:
- a writer for the field
- Throws:
IOException- if an I/O error occurs when adding the field
-
mergeOneField
Description copied from class:KnnVectorsWriterWrite field for merging- Overrides:
mergeOneFieldin classKnnVectorsWriter- Throws:
IOException
-
mergeOneFieldToIndex
public CloseableRandomVectorScorerSupplier mergeOneFieldToIndex(FieldInfo fieldInfo, MergeState mergeState) throws IOException Description copied from class:FlatVectorsWriterWrite the field for merging, providing a scorer over the newly merged flat vectors. This way any additional merging logic can be implemented by the user of this class.- Specified by:
mergeOneFieldToIndexin classFlatVectorsWriter- Parameters:
fieldInfo- fieldInfo of the field to mergemergeState- mergeState of the segments to merge- Returns:
- a scorer over the newly merged flat vectors, which should be closed as it holds a temporary file handle to read over the newly merged vectors
- Throws:
IOException- if an I/O error occurs when merging
-
flush
Description copied from class:KnnVectorsWriterFlush all buffered data on disk *- Specified by:
flushin classKnnVectorsWriter- Throws:
IOException
-
finish
Description copied from class:KnnVectorsWriterCalled once at the end before close- Specified by:
finishin classKnnVectorsWriter- Throws:
IOException
-
ramBytesUsed
public long ramBytesUsed()Description copied from interface:AccountableReturn the memory usage of this object in bytes. Negative values are illegal. -
mergeAndRecalculateQuantiles
public static ScalarQuantizer mergeAndRecalculateQuantiles(MergeState mergeState, FieldInfo fieldInfo, Float confidenceInterval, byte bits) throws IOException Merges the quantiles of the segments and recalculates the quantiles if necessary.- Parameters:
mergeState- The merge statefieldInfo- The field infoconfidenceInterval- The confidence intervalbits- The number of bits- Returns:
- The merged quantiles
- Throws:
IOException- If there is a low-level I/O error
-
writeQuantizedVectorData
public static DocsWithFieldSet writeQuantizedVectorData(IndexOutput output, QuantizedByteVectorValues quantizedByteVectorValues, byte bits, boolean compress) throws IOException Writes the vector values to the output and returns a set of documents that contains vectors.- Throws:
IOException
-
close
- Throws:
IOException
-