Evaluation overview

Use cases

Evaluate responses to:

Measure how well the Pinecone Assistant captures the facts of the ground truth answer.
Compare the Pinecone Assistant’s answers to those of another RAG system.
Compare the answers of your own RAG system to those of the Pinecone Assistant or another RAG system.

The request body requires the following fields:

Field	Description
`question`	The question asked to the RAG system.
`answer`	The answer provided by the assistant being evaluated.
`ground_truth_answer`	The expected answer.

For a runnable example, see Evaluate answers.

The response returns a score from 0 to 1 for each of the following metrics:

Metric	Description
`correctness`	Correctness of the RAG system’s answer compared to the ground truth answer.
`completeness`	Completeness of the RAG system’s answer compared to the ground truth answer.
`alignment`	A combined score of the correctness and completeness scores.

The response explains the reasoning behind each metric’s score. This includes a list of evaluated facts with their entailment status:

Status	Description
`entailed`	The fact is supported by the ground truth answer.
`contradicted`	The fact contradicts the ground truth answer.
`neutral`	The fact is neither supported nor contradicted by the ground truth answer.

The response includes the number of tokens used to calculate the metrics. This includes tokens for the prompt and completion.

Cost is calculated by token usage. See Pricing for up-to-date pricing information. Response evaluation is only available for Standard and Enterprise plans.