Use this file to discover all available pages before exploring further.
Pinecone is eventually consistent, so there can be a slight delay before new or changed records are visible to queries. This page describes two ways of checking the data freshness of a Pinecone index:
When you make a write request to a serverless index namespace, Pinecone assigns a monotonically increasing log sequence number (LSN) to the write operation. The LSN reflects upserts as well as updates and deletes to that namespace. Writes to one namespace do not increase the LSN for other namespaces.You can use LSNs to verify that specific write operations are reflected in your query responses. If the LSN contained in the query response header is greater than or equal to the LSN of the relevant write operation, then that operation is reflected in the query response. If the LSN contained in the query response header is greater than the LSN of the relevant write operation, then subsequent operations are also reflected in the query response.Follow the steps below to compare the LSNs for a write and a subsequent query.
Every time you modify records in your namespace, the HTTP response contains the LSN for the upsert. This is contained in a header called x-pinecone-request-lsn.The following example demonstrates how to get the LSN for an upsert request using the curl option -i. This option tells curl to include headers in the displayed response. Use the same method to get the LSN for an update or delete request.
In the preceding example response, the value of x-pinecone-max-indexed-lsn is 4. This means that the index has performed 4 write operations since its creation.
Every time you query your index, the HTTP response contains the LSN for the query. This is contained in a header called x-pinecone-max-indexed-lsn.By checking the LSN in your query results, you can confirm that the LSN is greater than or equal to the LSN of the relevant write operation, indicating that the results of that operation are present in the query results.The following example makes a query request to the index:
If the LSN of a query is greater than or equal to the LSN for a write operation, then the results of the query reflect the results of the write operation.In step 1, the LSN contained in the response headers is 4.In step 2, the LSN contained in the response headers is 5.5 is greater than or equal to 4; therefore, the results of the query reflect the results of the upsert. However, this does not guarantee that the records upserted are still present or unmodified: the write operation with LSN of 5 may have updated or deleted these records, or upserted additional records.
If you insert new records or delete records, the number of records in the index may change. This means that the record count for an index can indicate whether Pinecone has indexed your latest inserts and deletes: if the record count for the index matches the count you expect after inserting or deleting records, the index is probably up-to-date. However, this is not always true. For example, if you delete the same number of records that you insert, the expected record count may remain the same. Also, some write operations, such as updates to an index configuration or vector data values, do not change the number of records in the index.To verify that your index contains the number of records you expect, view index stats:
# pip install "pinecone[grpc]"from pinecone.grpc import PineconeGRPC as Pineconepc = Pinecone(api_key="YOUR_API_KEY")# To get the unique host for an index, # see https://docs.pinecone.io/guides/manage-data/target-an-indexindex = pc.Index(host="INDEX_HOST")index.describe_index_stats()