You can evaluate the correctness and completeness of a response from an assistant or RAG system.
Response evaluation is useful when performing tasks like the following:
You can evaluate responses directly or through the Pinecone Python SDK.
The request body requires the following fields:
Field | Description |
---|---|
question | The question asked to the RAG system. |
answer | The answer provided by the assistant being evaluated. |
ground_truth_answer | The expected answer. |
For example:
Calculated scores between 0
to 1
are returned for the following metrics:
Metric | Description |
---|---|
correctness | Correctness of the RAG system’s answer compared to the ground truth answer. |
completeness | Completeness of the RAG system’s answer compared to the ground truth answer. |
alignment | A combined score of the correctness and completeness scores. |
The response includes explanations for the reasoning behind each metric’s score. This includes a list of evaluated facts with their entailment status:
Status | Description |
---|---|
entailed | The fact is supported by the ground truth answer. |
contradicted | The fact contradicts the ground truth answer. |
neutral | The fact is neither supported nor contradicted by the ground truth answer. |
The response includes the number of tokens used to calculate the metrics. This includes the number of tokens used for the prompt and completion.
Cost is calculated by token usage. See Pricing for up-to-date pricing information.
Response evaluation is only available for Standard and Enterprise plans.