This guide shows you how to set up and use Pinecone Database for high-performance similarity search.

To get started in your browser, use the Quickstart colab notebook. To try Pinecone Database locally before creating an account, use Pinecone Local.

​
1. Install an SDK

Pinecone provides SDKs in multiple languages. For this quickstart, install the Python SDK:

pip install "pinecone[grpc]"

​
2. Get an API key

You need an API key to make calls to your Pinecone project.

Create a new API key in the Pinecone console, or use the widget below to generate a key. If you don’t have a Pinecone account, the widget will sign you up for the free Starter plan.

Your generated API key:

"{{YOUR_API_KEY}}"

​
3. Generate vectors

enable similarity-based search in vector databases like Pinecone. To convert data into this format, you use an embedding model.

For this quickstart, use the multilingual-e5-large embedding model hosted by Pinecone to convert four sentences about apples into vectors, three related to health, one related to cultivation.

# Import the Pinecone library
from pinecone.grpc import PineconeGRPC as Pinecone
from pinecone import ServerlessSpec
import time

# Initialize a Pinecone client with your API key
pc = Pinecone(api_key="YOUR_API_KEY")

# Define a sample dataset where each item has a unique ID, text, and category
data = [
    {
        "id": "rec1",
        "text": "Apples are a great source of dietary fiber, which supports digestion and helps maintain a healthy gut.",
        "category": "digestive system" 
    },
    {
        "id": "rec2",
        "text": "Apples originated in Central Asia and have been cultivated for thousands of years, with over 7,500 varieties available today.",
        "category": "cultivation"
    },
    {
        "id": "rec3",
        "text": "Rich in vitamin C and other antioxidants, apples contribute to immune health and may reduce the risk of chronic diseases.",
        "category": "immune system"
    },
    {
        "id": "rec4",
        "text": "The high fiber content in apples can also help regulate blood sugar levels, making them a favorable snack for people with diabetes.",
        "category": "endocrine system"
    }
]

# Convert the text into numerical vectors that Pinecone can index
embeddings = pc.inference.embed(
    model="multilingual-e5-large",
    inputs=[d["text"] for d in data],
    parameters={
        "input_type": "passage", 
        "truncate": "END"
    }
)

print(embeddings)

The returned object looks like this:

EmbeddingsList(
  model='multilingual-e5-large',
  vector_type='dense',
  data=[
    {'vector_type': dense, 'values': [0.042022705078125, -0.00952911376953125, ..., -0.0504150390625, -0.01045989990234375]},
    {'vector_type': dense, 'values': [0.03314208984375, -0.018524169921875, ..., -0.0095672607421875, -0.0240478515625]},
    {'vector_type': dense, 'values': [0.033599853515625, -0.00632476806640625, ..., -0.005527496337890625, -0.025299072265625]},
    {'vector_type': dense, 'values': [0.00994873046875, -0.01165771484375, ..., -0.0249481201171875, -0.006320953369140625]}
  ],
  usage={'total_tokens': 118}
)

​
4. Create an index

In Pinecone, you store data in an index.

Create a serverless index that matches the dimension (1024) and similarity metric (cosine) of the multilingual-e5-large model you used in the previous step, and choose a cloud and region for hosting the index:

# Create a serverless index
index_name = "example-index"

if not pc.has_index(index_name):
    pc.create_index(
        name=index_name,
        dimension=1024,
        metric="cosine",
        spec=ServerlessSpec(
            cloud="aws", 
            region="us-east-1"
        ) 
    ) 

# Wait for the index to be ready
while not pc.describe_index(index_name).status['ready']:
    time.sleep(1)

​
5. Upsert vectors

Target your index and use the upsert operation to load your vectors into a new namespace. Namespaces let you partition records within an index and are essential for implementing multitenancy when you need to isolate the data of each customer/user.

# Target the index
# In production, target an index by its unique DNS host, not by its name
# See https://docs.pinecone.io/guides/data/target-an-index
index = pc.Index(index_name)

# Prepare the records for upsert
# Each contains an 'id', the vector 'values', 
# and the original text and category as 'metadata'
records = []
for d, e in zip(data, embeddings):
    records.append({
        "id": d["id"],
        "values": e["values"],
        "metadata": {
            "source_text": d["text"],
            "category": d["category"]
        }
    })

# Upsert the records into the index
index.upsert(
    vectors=records,
    namespace="example-namespace"
)

To load large amounts of data, import from object storage or upsert in large batches.

Pinecone is eventually consistent, so there can be a delay before your upserted records are available to query. Use the describe_index_stats operation to check if the current vector count matches the number of vectors you upserted (4):

time.sleep(10)  # Wait for the upserted vectors to be indexed

print(index.describe_index_stats())

The response looks like this:

{'dimension': 1024,
 'index_fullness': 0.0,
 'namespaces': {'example-namespace': {'vector_count': 4}},
 'total_vector_count': 4}

​
6. Search the index

Now, let’s say you want to search your index for information related to “health risks”.

Use the multilingual-e5-large embedding model hosted by Pinecone to convert your query into a vector, and then use the query operation to search for the three vectors that are most semantically similar to the query vector:

# Define your query
query = "Health risks"

# Convert the query into a numerical vector that Pinecone can search with
query_embedding = pc.inference.embed(
    model="multilingual-e5-large",
    inputs=[query],
    parameters={
        "input_type": "query"
    }
)

# Search the index for the three most similar vectors
results = index.query(
    namespace="example-namespace",
    vector=query_embedding[0].values,
    top_k=3,
    include_values=False,
    include_metadata=True
)

print(results)

The response includes only records related to health, not the cultivation of apples:

{'matches': [{'id': 'rec3',
              'metadata': {'category': 'immune system',
                           'source_text': 'Rich in vitamin C and other '
                                          'antioxidants, apples contribute to '
                                          'immune health and may reduce the '
                                          'risk of chronic diseases.'},
              'score': 0.8203537,
              'sparse_values': {'indices': [], 'values': []},
              'values': []},
             {'id': 'rec1',
              'metadata': {'category': 'digestive system',
                           'source_text': 'Apples are a great source of '
                                          'dietary fiber, which supports '
                                          'digestion and helps maintain a '
                                          'healthy gut.'},
              'score': 0.7931527,
              'sparse_values': {'indices': [], 'values': []},
              'values': []},
             {'id': 'rec4',
              'metadata': {'category': 'endocrine system',
                           'source_text': 'The high fiber content in apples '
                                          'can also help regulate blood sugar '
                                          'levels, making them a favorable '
                                          'snack for people with diabetes.'},
              'score': 0.7800932,
              'sparse_values': {'indices': [], 'values': []},
              'values': []}],
 'namespace': 'example-namespace',
 'usage': {'read_units': 6}}

​
Add reranking

You can increase the accuracy of your search by reranking results based on their relevance to the query.

Use the rerank operation and the bge-reranker-v2-m3 reranking model hosted by Pinecone to rerank the values of the documents.source_text fields:

# Rerank the search results based on their relevance to the query
ranked_results = pc.inference.rerank(
    model="bge-reranker-v2-m3",
    query="Health risks",
    documents=[
        {"id": "rec3", "source_text": "Rich in vitamin C and other antioxidants, apples contribute to immune health and may reduce the risk of chronic diseases."},
        {"id": "rec1", "source_text": "Apples are a great source of dietary fiber, which supports digestion and helps maintain a healthy gut."},
        {"id": "rec4", "source_text": "The high fiber content in apples can also help regulate blood sugar levels, making them a favorable snack for people with diabetes."}
    ],
    top_n=3,
    rank_fields=["source_text"],
    return_documents=True,
    parameters={
        "truncate": "END"
    }
)

print(ranked_results)

The two records specifically related to “health risks” (chronic disease and diabetes) are now ranked highest:

RerankResult(
  model='bge-reranker-v2-m3',
  data=[{
    index=0,
    score=0.08021325,
    document={
        id='rec3',
        source_text='Rich in vitamin C and other antioxidants, apples contribute to immune health and may reduce the risk of chronic diseases.'
    }
  },{
    index=2,
    score=0.003976228,
    document={
        id='rec4',
        source_text='The high fiber content in apples can also help regulate blood sugar levels, making them a favorable snack for people with diabetes.'
    }
  },{
    index=1,
    score=0.0009888597,
    document={
        id='rec1',
        source_text='Apples are a great source of dietary fiber, which supports digestion and helps maintain a healthy gut.'
    }
  }],
  usage={'rerank_units': 1}
)

​
Add filtering

You can use a metadata filter to limit your search to records matching a filter expression.

Your upserted records contain a category metadata field. Now use that field as a filter to search for records in the “digestive system” category:

# Search the index with a metadata filter
filtered_results = index.query(
    namespace="example-namespace",
    vector=query_embedding.data[0].values,
    filter={
        "category": {"$eq": "digestive system"}
    },
    top_k=3,
    include_values=False,
    include_metadata=True
)

print(filtered_results)

The response includes only the one record in the “digestive system” category:

{'matches': [{'id': 'rec1',
              'metadata': {'category': 'digestive system',
                           'source_text': 'Apples are a great source of '
                                          'dietary fiber, which supports '
                                          'digestion and helps maintain a '
                                          'healthy gut.'},
              'score': 0.7931527,
              'sparse_values': {'indices': [], 'values': []},
              'values': []}],
 'namespace': 'example-namespace',
 'usage': {'read_units': 6}

​
7. Clean up

When you no longer need your example-index, delete it as follows:

pc.delete_index(index_name)

For production indexes, consider enabling deletion protection.

​
Next steps

Was this page helpful?

OverviewAssistant quickstart