This page shows you how to evaluate responses from Pinecone Assistant or other RAG systems using the Evaluation API.

Calculate evaluation metrics

To evaluate a response, use the https://prod-1-data.ke.pinecone.io/assistant/evaluation/metrics/alignment operation. This operation returns a JSON object.

Response
{
  "metrics": {
	  "correctness": 0.5,
	  "completeness": 0.333,
	  "alignment": 0.398,
  },
  "reasoning":{
	  "evaluated_facts": [
	    {
	      "fact": {"content": "Paris is the capital of France"},
	      "entailment": "entailed",
	    },
	    {
	      "fact": {"content": "London is the capital of England"},
	      "entailment": "neutral"
	    },
	    {
	      "fact": {"content": "Madrid is the capital of Spain"},
	      "entailment": "contradicted",
	    }
	  ]
  },
  "usage": {
     "prompt_tokens": 22,
     "completion_tokens": 33,
     "total_tokens": 55
  }
}