Monitoring
This document describes how to configure monitoring for your Pinecone index using Prometheus or compatible tools.
Overview
You can ingest performance metrics from Pinecone indexes into your own Prometheus instances, or into Prometheus- and OpenMetrics-compatible monitoring tools. The Prometheus metric endpoint is for users who want to monitor and store system health metrics using their own Prometheus metrics logger.
Warning
This feature is in public preview and is only available to Enterprise or
Enterprise Dedicated users.
Connect
Metrics are available at a URL like the following:
https://metrics.YOUR_ENVIRONMENT.pinecone.io/metrics
Your API key must be passed via the Authorization header as a bearer token like the following:
Authorization: Bearer \<api-key\>
Only the metrics for the project associated with the API key are available at this URL.
For Prometheus, configure prometheus.yml as follows:
scrape_configs:
- job_name: pinecone-job-1
authorization:
credentials: <api-key-here>
scheme: https
static_configs:
- targets: ['metrics.YOUR_ENVIRONMENT.pinecone.io']
See Prometheus docs for more configuration details.
Available Metrics
The metrics available are as follows:
Name | Type | Description | Labels |
---|---|---|---|
pinecone_vector_count | gauge | The number of records per pod in the index. |
- pid: Process identifier - index_name: Name of the index - project_name: Pinecone project name |
pinecone_request_count_total | counter | The number of data plane calls made by clients. |
- pid: Process identifier - index_name: Name of the index - project_name: Pinecone project name - request_type: One of upsert, delete, fetch, query, describe_index_stats |
pinecone_request_error_count_total | counter | The number of data plane calls made by clients that resulted in errors. |
- pid: Process identifier - index_name: Name of the index - project_name: Pinecone project name - request_type: One of upsert, delete, fetch, query, describe_index_stats |
pinecone_request_latency_seconds | histogram | The distribution of server-side processing latency for pinecone data plane calls. |
- pid: Process identifier - index_name: Name of the index - project_name: Pinecone project name - request_type: One of upsert, delete, fetch, query, describe_index_stats |
pinecone_index_fullness | gauge | The fullness of the index on a scale of 0 to 1. |
- pid: Process identifier - index_name: Name of the index - project_name: Pinecone project name |
Example queries
The following Prometheus queries gather information about your Pinecone index.
Average request latency
The following query returns the average latency in seconds for all requests
against the Pinecone index example-index
.
avg by (request_type) (pinecone_request_latency_seconds{index_name="example-index"})
The following query returns the vector count for the Pinecone index example-index
.
sum ((avg by (app) (pinecone_vector_count{index_name="example-index"})))
The following query returns the total number of requests against the Pinecone index example-index
over one minute.
sum by (request_type)(increase(pinecone_request_count_total{index_name="example-index"}[60s]))
The following query returns the total number of upsert requests against the Pinecone index example-index
over one minute.
sum by (request_type)(increase(pinecone_request_count_total{index_name="example-index", request_type="upsert"}[60s]))
The following query returns the total errors returned by the Pinecone index example-index
over one minute.
sum by (request_type) (increase(pinecone_request_error_count{
index_name="example-index"}[60s]))
The following query returns the index fullness metric for the Pinecone index example-index
.
round(max (pinecone_index_fullness{index_name="example-index"} * 100))
Updated 1 day ago