Class Lucene90HnswVectorsFormat
java.lang.Object
org.apache.lucene.codecs.KnnVectorsFormat
org.apache.lucene.backward_codecs.lucene90.Lucene90HnswVectorsFormat
- All Implemented Interfaces:
NamedSPILoader.NamedSPI
Lucene 9.0 vector format, which encodes numeric vector values and an optional associated graph
connecting the documents having values. The graph is used to power HNSW search. The format
consists of three files:
.vec (vector data) file
This file stores all the floating-point vector data ordered by field, document ordinal, and vector dimension. The floats are stored in little-endian byte order.
.vex (vector index) file
Stores graphs connecting the documents for each field. For each document having a vector for a given field, this is stored as:
- [int32] the number of neighbor nodes
- array[vint] the neighbor ordinals, delta-encoded (initially subtracting -1)
.vem (vector metadata) file
For each field:
- [int32] field number
- [int32] vector similarity function ordinal
- [vlong] offset to this field's vectors in the .vec file
- [vlong] length of this field's vectors, in bytes
- [vlong] offset to this field's index in the .vex file
- [vlong] length of this field's index data, in bytes
- [int] dimension of this field's vectors
- [int] the number of documents having values for this field
- array[vint] the docids of documents having vectors, in order
- array[vlong] for each document having a vector, the offset (delta-encoded relative to the previous document) of its entry in the .vex file
-
Field Summary
FieldsModifier and TypeFieldDescription(package private) final int
The number of candidate neighbors to track while searching the graph for each newly inserted node.static final int
Default number of the size of the queue maintained while searching and the number of random entry points to sample during a graph construction.static final int
Default number of maximum connections per node(package private) final int
Controls how many of the nearest neighbor candidates are connected to the new node.(package private) static final String
(package private) static final String
(package private) static final String
(package private) static final String
(package private) static final String
(package private) static final String
(package private) static final int
(package private) static final int
Fields inherited from class org.apache.lucene.codecs.KnnVectorsFormat
DEFAULT_MAX_DIMENSIONS, EMPTY
-
Constructor Summary
ConstructorsConstructorDescriptionA constructor for vectors format with default parametersLucene90HnswVectorsFormat
(int maxConn, int beamWidth) A constructor for vectors format -
Method Summary
Modifier and TypeMethodDescriptionfieldsReader
(SegmentReadState state) Returns aKnnVectorsReader
to read the vectors from the index.fieldsWriter
(SegmentWriteState state) Returns aKnnVectorsWriter
to write the vectors to the index.toString()
Methods inherited from class org.apache.lucene.codecs.KnnVectorsFormat
availableKnnVectorsFormats, forName, getMaxDimensions, getName, reloadKnnVectorsFormat
-
Field Details
-
META_CODEC_NAME
- See Also:
-
VECTOR_DATA_CODEC_NAME
- See Also:
-
VECTOR_INDEX_CODEC_NAME
- See Also:
-
META_EXTENSION
- See Also:
-
VECTOR_DATA_EXTENSION
- See Also:
-
VECTOR_INDEX_EXTENSION
- See Also:
-
VERSION_START
static final int VERSION_START- See Also:
-
VERSION_CURRENT
static final int VERSION_CURRENT- See Also:
-
DEFAULT_MAX_CONN
public static final int DEFAULT_MAX_CONNDefault number of maximum connections per node- See Also:
-
DEFAULT_BEAM_WIDTH
public static final int DEFAULT_BEAM_WIDTHDefault number of the size of the queue maintained while searching and the number of random entry points to sample during a graph construction.- See Also:
-
maxConn
final int maxConnControls how many of the nearest neighbor candidates are connected to the new node. Defaults toDEFAULT_MAX_CONN
. SeeHnswGraph
for more details. -
beamWidth
final int beamWidthThe number of candidate neighbors to track while searching the graph for each newly inserted node. Defaults toDEFAULT_BEAM_WIDTH
. SeeHnswGraph
for details.
-
-
Constructor Details
-
Lucene90HnswVectorsFormat
public Lucene90HnswVectorsFormat()A constructor for vectors format with default parameters -
Lucene90HnswVectorsFormat
public Lucene90HnswVectorsFormat(int maxConn, int beamWidth) A constructor for vectors format
-
-
Method Details
-
fieldsWriter
Description copied from class:KnnVectorsFormat
Returns aKnnVectorsWriter
to write the vectors to the index.- Specified by:
fieldsWriter
in classKnnVectorsFormat
- Throws:
IOException
-
fieldsReader
Description copied from class:KnnVectorsFormat
Returns aKnnVectorsReader
to read the vectors from the index.- Specified by:
fieldsReader
in classKnnVectorsFormat
- Throws:
IOException
-
toString
-