Monitoring

Pinecone generates time-series performance metrics for each Pinecone index. You can monitor these metrics directly in the Pinecone console or with tools like Prometheus or Datadog.

Monitor in the Pinecone Console

To view performance metrics in the Pinecone console:

Open the Pinecone console.
Select the project containing the index you want to monitor.
Go to Database > Indexes.
Select the index.
Go to the Metrics tab.

Monitor with Datadog

To monitor Pinecone with Datadog, use Datadog’s Pinecone integration.

This feature is available on the Standard, Enterprise, and Dedicated plans.

Monitor with Prometheus

This feature is available on the Standard, Enterprise, and Dedicated plans. On the Dedicated plan (i.e., Bring Your Own Cloud), you must configure Prometheus monitoring within your VPC.

Configuration

To monitor all serverless indexes in a project, insert the following snippet into the scrape_configs section of your prometheus.yml file and update it with values for your Prometheus integration:

This method uses HTTP service discovery to automatically discover and target all serverless indexes across all regions in a project.

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'pinecone-serverless-metrics'
    http_sd_configs:
      - url: https://api.pinecone.io/prometheus/projects/PROJECT_ID/metrics/discovery
        refresh_interval: 1m
        authorization:
          type: Bearer
          credentials: API_KEY
    authorization:
      type: Bearer
      credentials: API_KEY

Replace PROJECT_ID with the unique ID of the project you want to monitor. You can find the project ID in the Pinecone console.
Replace both instances of API_KEY with an API key for the project you want to monitor. The first instance is for service discovery, and the second instance is for the discovered targets. If necessary, you can create an new API key in the Pinecone console.

To monitor all serverless indexes in a project, insert the following snippet into the scrape_configs section of your prometheus.yml file and update it with values for your Prometheus integration:

This method uses HTTP service discovery to automatically discover and target all serverless indexes across all regions in a project.

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'pinecone-serverless-metrics'
    http_sd_configs:
      - url: https://api.pinecone.io/prometheus/projects/PROJECT_ID/metrics/discovery
        refresh_interval: 1m
        authorization:
          type: Bearer
          credentials: API_KEY
    authorization:
      type: Bearer
      credentials: API_KEY

Replace PROJECT_ID with the unique ID of the project you want to monitor. You can find the project ID in the Pinecone console.
Replace both instances of API_KEY with an API key for the project you want to monitor. The first instance is for service discovery, and the second instance is for the discovered targets. If necessary, you can create an new API key in the Pinecone console.

To monitor all pod-based indexes in a specific region of a project, insert the following snippet into the scrape_configs section of your prometheus.yml file and update it with values for your Prometheus integration:

scrape_configs:
  - job_name: "pinecone-pod-metrics"
    scheme: https
    metrics_path: '/metrics'
    authorization:
      credentials: API_KEY
    static_configs:
      - targets: ["metrics.ENVIRONMENT.pinecone.io" ]

Replace API_KEY with an API key for the project you want to monitor. If necessary, you can create an new API key in the Pinecone console.
Replace ENVIRONMENT with the environment of the pod-based indexes you want to monitor.

For more configuration details, see the Prometheus docs.

Available metrics

The following metrics are available when you integrate Pinecone with Prometheus:

Name	Type	Description
`pinecone_db_record_total`	gauge	The total number of records in the index.
`pinecone_db_op_upsert_total`	counter	The number of upsert requests made to an index.
`pinecone_db_op_upsert_duration_total`	counter	The total time taken processing upsert requests for an index in milliseconds.
`pinecone_db_op_query_total`	counter	The number of query requests made to an index.
`pinecone_db_op_query_duration_total`	counter	The total time taken processing query requests for an index in milliseconds.
`pinecone_db_op_fetch_total`	counter	The number of fetch requests made to an index.
`pinecone_db_op_fetch_duration_total`	counter	The total time taken processing fetch requests for an index in milliseconds.
`pinecone_db_op_update_total`	counter	The number of update requests made to an index.
`pinecone_db_op_update_duration_total`	counter	The total time taken processing update requests for an index in milliseconds.
`pinecone_db_op_delete_total`	counter	The number of delete requests made to an index.
`pinecone_db_op_delete_duration_total`	counter	The total time taken processing delete requests for an index in milliseconds.
`pinecone_db_write_unit_total`	counter	The total number of write units consumed by an index.
`pinecone_db_read_unit_total`	counter	The total number of read units consumed by an index.
`pinecone_db_storage_size_bytes`	gauge	The total size of the index in bytes.

Name	Type	Description
`pinecone_db_record_total`	gauge	The total number of records in the index.
`pinecone_db_op_upsert_total`	counter	The number of upsert requests made to an index.
`pinecone_db_op_upsert_duration_total`	counter	The total time taken processing upsert requests for an index in milliseconds.
`pinecone_db_op_query_total`	counter	The number of query requests made to an index.
`pinecone_db_op_query_duration_total`	counter	The total time taken processing query requests for an index in milliseconds.
`pinecone_db_op_fetch_total`	counter	The number of fetch requests made to an index.
`pinecone_db_op_fetch_duration_total`	counter	The total time taken processing fetch requests for an index in milliseconds.
`pinecone_db_op_update_total`	counter	The number of update requests made to an index.
`pinecone_db_op_update_duration_total`	counter	The total time taken processing update requests for an index in milliseconds.
`pinecone_db_op_delete_total`	counter	The number of delete requests made to an index.
`pinecone_db_op_delete_duration_total`	counter	The total time taken processing delete requests for an index in milliseconds.
`pinecone_db_write_unit_total`	counter	The total number of write units consumed by an index.
`pinecone_db_read_unit_total`	counter	The total number of read units consumed by an index.
`pinecone_db_storage_size_bytes`	gauge	The total size of the index in bytes.

Name	Type	Description
`pinecone_vector_count`	gauge	The number of records per pod in the index.
`pinecone_request_count_total`	counter	The number of data plane calls made by clients.
`pinecone_request_error_count_total`	counter	The number of data plane calls made by clients that resulted in errors.
`pinecone_request_latency_seconds`	histogram	The distribution of server-side processing latency for pinecone data plane calls.
`pinecone_index_fullness`	gauge	The fullness of the index on a scale of 0 to 1.

Metric labels

Each metric contains the following labels:

Label	Description
`index_name`	Name of the index to which the metric applies.
`cloud`	Cloud where the index is deployed: `aws`, `gcp`, or `azure`.
`region`	Region where the index is deployed.
`capacity_mode`	Type of index: `serverless` or `byoc`.

Label	Description
`index_name`	Name of the index to which the metric applies.
`cloud`	Cloud where the index is deployed: `aws`, `gcp`, or `azure`.
`region`	Region where the index is deployed.
`capacity_mode`	Type of index: `serverless` or `byoc`.

Label	Description
`pid`	Process identifier.
`index_name`	Name of the index to which the metric applies.
`project_name`	Name of the project containing the index.
`request_type`	Type of request: `upsert`, `delete`, `fetch`, `query`, or `describe_index_stats`. This label is included only in `pinecone_request_*` metrics.

Example queries

Return the total number of records per index:

sum by (index_name) (pinecone_db_record_total)

Return the total number of records in Pinecone index example-index:

pinecone_db_record_total{index_name="example-index"}

Return the total number of upsert requests per index:

sum by (index_name) (pinecone_db_op_upsert_total)

Return the average processing time in millisconds for upsert requests per index:

sum by (index_name) (pinecone_db_op_upsert_duration_total/pinecone_db_op_upsert_total) 

Return the total read units consumed per index:

sum by (index_name) (pinecone_db_read_unit_total)

Return the total write units consumed for the Pinecone index example-index:

pinecone_db_write_unit_total{index_name="example-index"}

Return the total number of records per index:

sum by (index_name) (pinecone_db_record_total)

Return the total number of records in Pinecone index example-index:

pinecone_db_record_total{index_name="example-index"}

Return the total number of upsert requests per index:

sum by (index_name) (pinecone_db_op_upsert_total)

Return the average processing time in millisconds for upsert requests per index:

sum by (index_name) (pinecone_db_op_upsert_duration_total/pinecone_db_op_upsert_total) 

Return the total read units consumed per index:

sum by (index_name) (pinecone_db_read_unit_total)

Return the total write units consumed for the Pinecone index example-index:

pinecone_db_write_unit_total{index_name="example-index"}

Return the average latency in seconds for all requests against the Pinecone index example-index:

avg by (request_type) (pinecone_request_latency_seconds{index_name="example-index"})

Return the vector count for the Pinecone index example-index:

sum ((avg by (app) (pinecone_vector_count{index_name="example-index"})))

Return the total number of requests against the Pinecone index example-index over one minute:

sum by (request_type)(increase(pinecone_request_count_total{index_name="example-index"}[60s]))

Return the total number of upsert requests against the Pinecone index example-index over one minute:

sum by (request_type)(increase(pinecone_request_count_total{index_name="example-index", request_type="upsert"}[60s]))

Return the total errors returned by the Pinecone index example-index over one minute:

sum by (request_type) (increase(pinecone_request_error_count{
      index_name="example-index"}[60s]))

Return the index fullness metric for the Pinecone index example-index:

round(max (pinecone_index_fullness{index_name="example-index"} * 100))

Get started

Indexes

Data

Inference

Operations

Admin

Monitor in the Pinecone Console

Monitor with Datadog

Monitor with Prometheus

Configuration

Available metrics

Metric labels

Example queries

Get started

Indexes

Data

Inference

Operations

Admin

​Monitor in the Pinecone Console

​Monitor with Datadog

​Monitor with Prometheus

​Configuration

​Available metrics

​Metric labels

​Example queries

Monitor in the Pinecone Console

Monitor with Datadog

Monitor with Prometheus

Configuration

Available metrics

Metric labels

Example queries