Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions modules/indexes/pages/indexing-and-query-perf.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -588,7 +588,7 @@ For further details and examples, see {covering-indexes}[].
[#hyperscale-vector-index]
== Hyperscale Vector Index

Hyperscale vector indexes are a type of <<secondary-index>> which contain a single vector field.
Hyperscale Vector indexes are a type of <<secondary-index>> which contain a single vector field.
They excel at indexing huge datasets that can scale into the billions of documents.
They're optimized for pure vector searches, offering the highest performance of any index for your AI applications.

Expand All @@ -597,7 +597,7 @@ For further details and examples, see {hyperscale-vector-index}[].
[#composite-vector-index]
== Composite Vector Index

Composite vector indexes are a type of <<composite-secondary-index>> which contain a single vector field and one or more scalar fields.
Composite Vector indexes are a type of <<composite-secondary-index>> which contain a single vector field and one or more scalar fields.
Your AI applications can use the index's scalar fields to filter the dataset before performing a vector similarity search.

For further details and examples, see {composite-vector-index}[].
12 changes: 6 additions & 6 deletions modules/n1ql/pages/n1ql-language-reference/altervectorindex.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ For more information, see xref:n1ql:n1ql-intro/queriesandresults.adoc#query-cont

.Create and move an index from one node to another
====
Create a hyperscale vector index on node 172.19.0.2.
Create a Hyperscale Vector index on node 172.19.0.2.

[source,sqlpp]
----
Expand All @@ -91,7 +91,7 @@ To check the node where the index is located, see xref:manage:manage-indexes/man

.Create and move an index replica from one node to another
====
Create a hyperscale vector index on node 172.19.0.2 with a replica on node 172.19.0.3, then move its replica from node 172.19.0.*3* to 172.19.0.*4*.
Create a Hyperscale Vector index on node 172.19.0.2 with a replica on node 172.19.0.3, then move its replica from node 172.19.0.*3* to 172.19.0.*4*.

[source,sqlpp]
----
Expand All @@ -109,7 +109,7 @@ WITH {"action": "move", "nodes": ["172.19.0.2:8091", "172.19.0.4:8091"]};

.Increase the number of replicas
====
Create a hyperscale vector index on node 172.19.0.2 with a replica on nodes 172.19.0.*3*, then increase the number of replicas to 2 and specify that new replicas may be placed on any available index nodes in the cluster.
Create a Hyperscale Vector index on node 172.19.0.2 with a replica on nodes 172.19.0.*3*, then increase the number of replicas to 2 and specify that new replicas may be placed on any available index nodes in the cluster.

[source,sqlpp]
----
Expand All @@ -127,7 +127,7 @@ WITH {"action": "replica_count", "num_replica": 2};

.Increase the number of replicas and specify the nodes
====
Create a hyperscale vector index on node 172.19.0.2 with a replica on node 172.19.0.3, then increase the number of replicas to 2, and specify that replicas may be placed on nodes 172.19.0.*3* and 172.19.0.*4*.
Create a Hyperscale Vector index on node 172.19.0.2 with a replica on node 172.19.0.3, then increase the number of replicas to 2, and specify that replicas may be placed on nodes 172.19.0.*3* and 172.19.0.*4*.

[source,sqlpp]
----
Expand All @@ -149,7 +149,7 @@ WITH {"action": "replica_count",

.Decrease the number of replicas
====
Create a hyperscale vector index on node 172.19.0.2 with replicas on nodes 172.19.0.*3* and 172.19.0.*4*, then decrease the number of replicas to 1.
Create a Hyperscale Vector index on node 172.19.0.2 with replicas on nodes 172.19.0.*3* and 172.19.0.*4*, then decrease the number of replicas to 1.

[source,sqlpp]
----
Expand All @@ -167,7 +167,7 @@ WITH {"action": "replica_count", "num_replica": 1};

.Drop a specific replica
====
Create a hyperscale vector index with 2 replicas, and specify that nodes 172.19.0.2, 172.19.0.3, and 172.19.0.4 should be available for index and replica placement.
Create a Hyperscale Vector index with 2 replicas, and specify that nodes 172.19.0.2, 172.19.0.3, and 172.19.0.4 should be available for index and replica placement.
Then delete replica 2.

[source,sqlpp]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ If any of the indexes specified by BUILD INDEX have already been built, BUILD IN

When building an index which has automatic index replicas, all of the replicas are also built as part of the BUILD INDEX statement, without having to manually specify them.

Hyperscale vector indexes and composite vector indexes require a codebook for the vector field.
Hyperscale Vector indexes and Composite Vector indexes require a codebook for the vector field.
The codebook is the result of sampling the dataset and is saved as part of the index metadata.

The codebook is created as part of the BUILD INDEX process, and is not incrementally updated.
Expand Down
60 changes: 38 additions & 22 deletions modules/n1ql/pages/n1ql-language-reference/createindex.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
:page-toclevels: 2
:imagesdir: ../../assets/images
:keywords: secondary, index, placement
:description: The CREATE INDEX statement allows you to create secondary indexes and composite vector indexes.
:description: The CREATE INDEX statement allows you to create secondary indexes and Composite Vector indexes.

:authorization-overview: xref:learn:security/authorization-overview.adoc
:index-replication: xref:indexes:index-replication.adoc#index-replication
Expand Down Expand Up @@ -35,8 +35,8 @@ The `CREATE INDEX` statement allows you to create secondary indexes.
Secondary indexes contain a filtered or a full set of keys in a given keyspace.
Secondary indexes are optional but increase query efficiency on a keyspace.

In Couchbase Server 8.0 and later, the `CREATE INDEX` statement also allows you to create composite vector indexes.
To create hyperscale vector indexes, use the {create-vector-index}[CREATE VECTOR INDEX] statement.
In Couchbase Server 8.0 and later, the `CREATE INDEX` statement also allows you to create Composite Vector indexes.
To create Hyperscale Vector indexes, use the {create-vector-index}[CREATE VECTOR INDEX] statement.

//tag::purpose[]
== Purpose
Expand All @@ -57,7 +57,7 @@ In Couchbase Server Enterprise Edition, the recommended way to do this is using
In Couchbase Server Community Edition, you need to create multiple identical indexes and place them using the `nodes` option.
See <<index-with,WITH Clause>> for more details.

Hyperscale vector indexes and composite vector indexes require a codebook for the vector field.
Hyperscale Vector indexes and Composite Vector indexes require a codebook for the vector field.
The codebook is the result of sampling the dataset and is saved as part of the index metadata.

The codebook is created as part of the {build-index}[BUILD INDEX] process, and is not incrementally updated.
Expand All @@ -73,8 +73,6 @@ If the data set changes dramatically, you must drop and rebuild the index to upd
To execute the {doctitle} statement, you must have the `Query Manage Index` privilege granted on the keyspace.
For more information about user roles, see
{authorization-overview}[Authorization].

include::partial$n1ql-language-reference/vector-index-no-windows.adoc[]
//end::prerequisites[]

== Syntax
Expand Down Expand Up @@ -210,7 +208,7 @@ include::partial$grammar/ddl.ebnf[tag=index-keys-and-attribs]

image::n1ql-language-reference/index-keys-and-attribs.png["Syntax diagram: see source code listing", align=left]

Secondary indexes and composite vector indexes can have many keys.
Secondary indexes and Composite Vector indexes can have many keys.
Each key may have index attributes, which define the behavior of the index key.

[horizontal]
Expand All @@ -235,7 +233,7 @@ image::n1ql-language-reference/index-key.png["Syntax diagram: see source code li

The index key is a {sqlpp} {expression}[expression] referring to a field in the document, or an ARRAY expression on the field.

For a composite vector index, one index key must refer to a vector field in the document.
For a Composite Vector index, one index key must refer to a vector field in the document.
The index key that refers to a vector field may be the only index key.
If there are multiple index keys, the index key referring to the vector field may be any of the index keys, including the leading index key.

Expand Down Expand Up @@ -281,7 +279,7 @@ See <<index-order>>.
include-missing:: (Optional) If the leading index key is a non-vector field, it may also include the `INCLUDE MISSING` clause.
See <<include-missing>>.

include-vector:: (Optional) In a composite vector index, one index key must include the `VECTOR` keyword.
include-vector:: (Optional) In a Composite Vector index, one index key must include the `VECTOR` keyword.
See <<include-vector>>.

[[index-order]]
Expand Down Expand Up @@ -342,7 +340,7 @@ Used to partition the index.
Index partitioning helps increase the query performance by dividing and spreading a large index of documents across multiple nodes, horizontally scaling out an index as needed.
For details, see {index-partitioning}[Index Partitioning].

With hyperscale vector indexes and composite vector indexes, training is done for each index node independently, and the codebook is provided to all partitions on that node.
With Hyperscale Vector indexes and Composite Vector indexes, training is done for each index node independently, and the codebook is provided to all partitions on that node.
If there are multiple partitions for an index on a node, training is only done once for all partitions.
See {index-training}[The Importance of Index Training].
//end::index-partition[]
Expand Down Expand Up @@ -395,7 +393,7 @@ Use the WITH clause to specify additional options.
expr::
An object with the following properties.

[options="header", cols="1a,4a,1a"]
[options="header", cols="3a,8a,2a"]
|===
|Name|Description|Schema

Expand Down Expand Up @@ -459,9 +457,9 @@ If the value of this property is not less than the number of index nodes in the
//end::index-with[]
|===

Composite vector indexes support the following additional options.
Composite Vector indexes support the following additional options.

[options="header", cols="1a,4a,1a"]
[options="header", cols="3a,8a,2a"]
|===
|Name|Description|Schema
//tag::index-with[]
Expand All @@ -473,7 +471,7 @@ The embedded model you use to embed the vectors determines the number of dimensi
|Integer

|**similarity** +
__required__
__optional__
| Sets the distance metric to use when comparing vectors during index creation.

Couchbase {product-name} uses the following strings to represent the distance metrics:
Expand All @@ -487,10 +485,12 @@ L2_SQUARED::
EUCLIDEAN_SQUARED:: xref:vector-index:vectors-and-indexes-overview.adoc#euclidean-squared[Euclidean Squared Distance]

For the greatest accuracy, use the distance metric you plan to use to query the data.

**Default:** `L2_SQUARED`
|String

|**description** +
__required__
__optional__
| The settings for the quantization and index algorithms.
The string is made up of the following settings:

Expand All @@ -501,7 +501,9 @@ PQ:: For product quantization -- the number of subquantizers, and the number of

For more information, see {algo-settings}[Quantization and Centroid Settings].

[%hardbreaks]
**Pattern:** `^IVF[0-9]*,(SQ[468]&vert;PQ[0-9]+x[0-9]+)$`
**Default:** `IVF,SQ8`
|String

|**scan_nprobes** +
Expand All @@ -522,7 +524,7 @@ Otherwise, the default value is 10% of the index count, or 10{nbsp}&times; the n
**Maximum:** `1000000`
|Integer

ifeval::["{docname}" == "createvectorindex"]
ifeval::["{docname}" == "n1ql-language-reference/createvectorindex"]
|**persist_full_vector** +
__optional__
| If `true` (the default), the index stores the full vector value in addition to the quantized value.
Expand All @@ -546,7 +548,7 @@ See {index-partitioning}[].
[IMPORTANT]
.Attention
====
Do not create (or drop) secondary indexes, composite vector indexes, or hyperscale vector indexes when any Index service node is down, as this may result in duplicate index names.
Do not create (or drop) secondary indexes, Composite Vector indexes, or Hyperscale Vector indexes when any Index service node is down, as this may result in duplicate index names.
====
//end::usage[]

Expand Down Expand Up @@ -604,7 +606,7 @@ The system automatically load-balances an index scan across the index and all it
Adding index replicas enables you to scale scan throughput, in addition to providing high availability.
====

With hyperscale vector indexes and composite vector indexes, training is done by each replica index independently, and the codebook is stored as part of index metadata.
With Hyperscale Vector indexes and Composite Vector indexes, training is done by each replica index independently, and the codebook is stored as part of index metadata.
See {index-training}[The Importance of Index Training].
//end::index-replicas[]

Expand Down Expand Up @@ -729,12 +731,12 @@ For more examples of indexes where the leading key may be missing, see xref:n1ql
====

[[ex-create-rgb-idx]]
.Create a composite vector index
.Create a Composite Vector index
====
For this example, you must install the vector sample data as described in {prerequisites}[Prerequisites].
The path to the required keyspace is specified by the query, so you do not need to set the query context.

Create a composite vector index that indexes the vector field named `colorvect_l2`, as well as the scalar `color` and `brightness` fields.
Create a Composite Vector index that indexes the vector field named `colorvect_l2`, as well as the scalar `color` and `brightness` fields.

[source,sqlpp]
----
Expand All @@ -743,19 +745,33 @@ include::vector-index:example$gsi-vector-idx-examples.sqlpp[tag=create-rgb-idx]
====

[[ex-create-vectors-idx]]
.Create a composite vector index using embedded vectors
.Create a Composite Vector index using embedded vectors
====
For this example, you must install the vector sample data as described in {prerequisites}[Prerequisites].
The path to the required keyspace is specified by the query, so you do not need to set the query context.

Create a composite vector index that indexes the vector field named `embedding-vector-dot`, as well as the scalar `color` and `brightness` fields.
Create a Composite Vector index that indexes the vector field named `embedding-vector-dot`, as well as the scalar `color` and `brightness` fields.

[source,sqlpp]
----
include::vector-index:example$gsi-vector-idx-examples.sqlpp[tag=create-vectors-idx]
----
====

[[ex-create-colors-idx]]
.Create a Composite Vector index with a scalar leading key
====
For this example, you must install the vector sample data as described in {prerequisites}[Prerequisites].
The path to the required keyspace is specified by the query, so you do not need to set the query context.

Create a Composite Vector index that indexes the scalar `color` and `brightness` fields, as well as the vector field named `embedding-vector-dot`.

[source,sqlpp]
----
include::vector-index:example$gsi-vector-idx-examples.sqlpp[tag=create-colors-idx]
----
====

== Related Links

* xref:indexes:indexing-overview.adoc[]
Expand Down
20 changes: 10 additions & 10 deletions modules/n1ql/pages/n1ql-language-reference/createvectorindex.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
:page-toclevels: 2
:imagesdir: ../../assets/images
:keywords: secondary, index, placement
:description: The CREATE VECTOR INDEX statement allows you to create hyperscale vector indexes.
:description: The CREATE VECTOR INDEX statement allows you to create Hyperscale Vector indexes.

:authorization-overview: xref:learn:security/authorization-overview.adoc
:index-replication: xref:indexes:index-replication.adoc#index-replication
Expand Down Expand Up @@ -33,9 +33,9 @@
// TEMP
include::partial$n1ql-language-reference/horizontal-style.adoc[]

The `CREATE VECTOR INDEX` statement allows you to create hyperscale vector indexes.
The `CREATE VECTOR INDEX` statement allows you to create Hyperscale Vector indexes.

To create secondary indexes or composite vector indexes, use the {create-index}[CREATE INDEX] statement.
To create secondary indexes or Composite Vector indexes, use the {create-index}[CREATE INDEX] statement.

include::createindex.adoc[tags=purpose]

Expand Down Expand Up @@ -94,7 +94,7 @@ include::partial$grammar/ddl.ebnf[tags=index-key-and-attrib]

image::n1ql-language-reference/index-key-and-attrib.png["Syntax diagram: see source code listing", align=left]

Hyperscale vector indexes only have one key, which must be a vector field.
Hyperscale Vector indexes only have one key, which must be a vector field.
The index key takes one attribute, the VECTOR keyword.

[horizontal]
Expand Down Expand Up @@ -178,11 +178,11 @@ include::createindex.adoc[tags=defer-index-builds-by-default]
To try the examples in this section, you must install the vector sample data as described in {prerequisites}[Prerequisites].

[[ex-create-rgb-idx]]
.Create a hyperscale vector index
.Create a Hyperscale Vector index
====
For this example, the path to the required keyspace is specified by the query, so you do not need to set the query context.

Create a hyperscale vector index for the vector column named `embedding-vector-dot`.
Create a Hyperscale Vector index for the vector column named `embedding-vector-dot`.

[source,sqlpp]
----
Expand All @@ -191,11 +191,11 @@ include::vector-index:example$hyperscale-idx-examples.sqlpp[tag=create-rgb-idx]
====

[[ex-create-idx-brightness]]
.Create a hyperscale vector index with included scalar values
.Create a Hyperscale Vector index with included scalar values
====
For this example, the path to the required keyspace is specified by the query, so you do not need to set the query context.

Create a hyperscale vector index for the vector column named `embedding-vector-dot`, including the scalar `brightness` field.
Create a Hyperscale Vector index for the vector column named `embedding-vector-dot`, including the scalar `brightness` field.

[source,sqlpp]
----
Expand All @@ -204,11 +204,11 @@ include::vector-index:example$hyperscale-idx-examples.sqlpp[tag=create-idx-brigh
====

[[ex-create-rgb-no-persist]]
.Create a hyperscale vector index with no reranking
.Create a Hyperscale Vector index with no reranking
====
For this example, the path to the required keyspace is specified by the query, so you do not need to set the query context.

Create a hyperscale vector index from the example RGB dataset that does not persist the full vector value.
Create a Hyperscale Vector index from the example RGB dataset that does not persist the full vector value.

[source,sqlpp]
----
Expand Down
4 changes: 2 additions & 2 deletions modules/n1ql/pages/n1ql-language-reference/dropindex.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
= DROP INDEX
:description: The DROP INDEX statement allows you to drop a secondary index, a composite vector index, or a hyperscale vector index.
:description: The DROP INDEX statement allows you to drop a secondary index, a Composite Vector index, or a Hyperscale Vector index.
:page-topic-type: reference
:page-partial:
:page-toclevels: 2
Expand Down Expand Up @@ -305,7 +305,7 @@ If you drop an index which is scheduled for background creation, a warning messa
[IMPORTANT]
.Attention
====
Do not drop (or create) secondary indexes, composite vector indexes, or hyperscale vector indexes when any Index service node is down, as this may result in duplicate index names.
Do not drop (or create) secondary indexes, Composite Vector indexes, or Hyperscale Vector indexes when any Index service node is down, as this may result in duplicate index names.
====
//end::usage[]

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
= DROP VECTOR INDEX
:description: The DROP VECTOR INDEX statement allows you to drop a hyperscale vector index, a composite vector index, or a secondary index.
:description: The DROP VECTOR INDEX statement allows you to drop a Hyperscale Vector index, a Composite Vector index, or a secondary index.
:page-topic-type: reference
:page-status: Couchbase Server 8.0
:page-toclevels: 2
Expand Down Expand Up @@ -52,11 +52,11 @@ include::dropindex.adoc[tags=usage]
To try the examples in this section, you must install the vector sample data as described in {prerequisites}[Prerequisites].

[[ex-1]]
.Drop a hyperscale vector index
.Drop a Hyperscale Vector index
====
For this example, the path to the required keyspace is specified by the query, so you do not need to set the query context.

Drop the hyperscale vector index called `color_desc_hyperscale`, if it exists.
Drop the Hyperscale Vector index called `color_desc_hyperscale`, if it exists.

[source,sqlpp]
----
Expand Down
Loading
Loading