Manage data
In addition to inserting and querying data, there are other ways you can interact with vector data in a Pinecone index. This section walks through the various vector operations available.
Connect to an index
If you're using a Pinecone client library to access an index, you'll need to open a session with the index:
# Connect to the index
index = pinecone.Index("pinecone-index")
# Not applicable
Specify an index endpoint
Pinecone indexes each have their own DNS endpoint. For cURL and other direct
API calls to a Pinecone index, you'll need to know the dedicated endpoint for
your index.
Index endpoints take the following form:
https://{index-name}-{project-name}.svc.YOUR_ENVIRONMENT.pinecone.io
{index-name}
is the name you gave your index when you created it.{project-name}
is the Pinecone project name that your API key is associated
with. This can be retrieved using thewhoami
operation below.YOUR_ENVIRONMENT
is the cloud region for your Pinecone project..
Call whoami
to retrieve your project name.
whoami
to retrieve your project name.The following command retrieves your Pinecone project name.
pinecone.whoami()
curl -i https://controller.YOUR_ENVIRONMENT.pinecone.io/actions/whoami -H 'Api-Key: YOUR_API_KEY'
Describe index statistics
Get statistics about an index, such as vector count per namespace:
index.describe_index_stats()
curl -i -X GET https://YOUR_INDEX-PROJECT_NAME.svc.YOUR_ENVIRONMENT.pinecone.io/describe_index_stats \
-H 'Api-Key: YOUR_API_KEY'
Fetching vectors
The Fetch
operation looks up and returns vectors, by id, from an index. The returned vectors include the vector data and/or metadata. Typical fetch latency is under 5ms.
Fetch items by their ids:
index.fetch(["id-1", "id-2"])
# Returns:
# {'namespace': '',
# 'vectors': {'id-1': {'id': 'id-1',
# 'values': [0.568879, 0.632687092, 0.856837332, ...]},
# 'id-2': {'id': 'id-2',
# 'values': [0.00891787093, 0.581895, 0.315718859, ...]}}}
curl -i -X GET "https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/fetch?ids=id-1&ids=id-2" \
-H 'Api-Key: YOUR_API_KEY'
# Output:
# {
# "vectors": {
# "id-1": {
# "id": "id-1",
# "values": [0.568879, 0.632687092, 0.856837332, ...]
# },
# "id-2": {
# "id": "id-2",
# "values": [0.00891787093, 0.581895, 0.315718859, ...]
# }
# },
# "namespace": ""
# }
Updating vectors
There are two methods for updating vectors and metadata, using full or partial updates.
Full update
Full updates modify the entire item, that is vectors and metadata. Updating an item by id is done the same way as inserting items. (Write operations in Pinecone are idempotent.)
The Upsert
operation writes vectors into an index.
Note
If a new value is upserted for an existing vector id, it will overwrite the
previous value.
- Update the value of the item
("id-3", [3.3, 3.3])
:
index.upsert([("id-3", [3.3, 3.3])])
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/upsert \
-H 'Api-Key: YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"vectors": [
{
"id": "id-0",
"values": [3.3, 3.3]
}
]
}'
- Fetch the item again. We should get
("id-3", [3.3, 3.3])
:
index.fetch(["id-3"])
curl -i -X GET https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/fetch?ids=id-3 \
-H 'Api-Key: YOUR_API_KEY'
Partial update
The Update
operation performs partial updates that allow changes to part of an item. Given an id, we can update the vector value with the values
argument or update metadata with the set_metadata
argument.
Warning
The
Update
operation does not validate the existence of ids within an
index. If a non-existent id is given then no changes are made and a200 OK
will be returned.
To update the value of item ("id-3", [3., 3.], {"type": "doc", "genre": "drama"})
:
index.update(id="id-3", values=[4., 2.])
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/update \
-H 'Api-Key: YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"vectors": [
{
"id": "id-3",
"values": [4., 2.]
}
]
}'
The updated item would now be ("id-3", [4., 2.], {"type": "doc", "genre": "drama"})
.
When updating metadata only specified fields will be modified. If a specified field does not exist, it is added.
Note
Metadata updates apply only to fields passed to the
set_metadata
argument. Any other fields will remain unchanged.
To update the metadata of item ("id-3", [4., 2.], {"type": "doc", "genre": "drama"})
:
index.update(id="id-3", set_metadata={"type": "web", "new": "true"})
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/update \
-H 'Api-Key: YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"vectors": [
{
"id": "id-3",
"set_metadata": {"type": "web", "new": "true"}
}
]
}'
The updated item would now be ("id-3", [4., 2.], {"type": "web", "genre": "drama", "new": "true"})
.
Both vector and metadata can be updated at once by including both values
and set_metadata
arguments. To update the "id-3"
item we write:
index.update(id="id-3", values=[1., 2.], set_metadata={"type": "webdoc"})
curl -i -X POST https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/update \
-H 'Api-Key: YOUR_API_KEY' \
-H 'Content-Type: application/json' \
-d '{
"vectors": [
{
"id": "id-3",
"values": [1., 2.],
"set_metadata": {"type": "webdoc"}
}
]
}'
The updated item would now be ("id-3", [1., 2.], {"type": "webdoc", "genre": "drama", "new": "true"})
.
Deleting vectors
The Delete
operation deletes vectors, by ID, from an index.
Alternatively, it can also delete all vectors from an index or namespace.
When deleting large numbers of vectors, limit the scope of delete operations to hundreds of vectors per operation.
Instead of deleting all vectors in an index, delete the index and recreate it.
Delete vectors by ID
To delete vectors by their IDs, specify an ids
parameter to delete
. The ids
parameter is an array of strings containing vector IDs.
Example
index.delete(ids=["id-1", "id-2"], namespace='example-namespace')
curl -i -X DELETE "https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io.pinecone.io/vectors/delete?ids=id-1&ids=id-2&namespace=example-namespace" \
-H 'Api-Key: YOUR_API_KEY'
Delete vectors by namespace
To delete all vectors from a namespace, specify delete_all=True
and provide a
namespace
parameter.
Note
If you delete all vectors from a single namespace, it will also delete the
namespace.
Example:
index.delete(delete_all=True, namespace='example-namespace')
curl -i -X DELETE "https://YOUR_INDEX-YOUR_PROJECT.svc.YOUR_ENVIRONMENT.pinecone.io/vectors/delete?delete_all=true&namespace=example-namespace" \
-H 'Api-Key: YOUR_API_KEY'
Delete vectors by metadata
To delete vectors by metadata, pass a metadata filter expression to the delete operation.
Updated 3 days ago