Skip to main content
This guide walks you through the essential steps of using Pinecone Context Engine: creating a repository with a schema, adding JSON documents, and performing searches. You’ll use a product catalog as your example, showing how to search across descriptions, specifications, reviews, and metadata.
Context Engine is currently in alpha, under active development. The alpha release does not include the full capabilities of Context Engine. It provides document repository functionality with basic retrieval. Advanced query planning, intelligent query rewriting, and sophisticated context expansion will be available in future releases.

​
1. Install the SDK

Install the specific version of the Pinecone Python SDK that includes Context Engine support: 
pip install pinecone==7.3.1.alpha1

​
2. Create a repository

Create a repository with a schema that defines how to index your product data. The schema specifies which fields are embedded for semantic and lexical search (descriptions, reviews). All fields are filterable by default for precise matching. First, take a look at the structure of the JSON documents you’ll be upserting: 
{
  "_id": "headphones_at_wh1000xm5",
  "name": "AudioTech WH-1000XM5 Wireless Headphones", 
  "details": {
    "description": "Industry-leading noise canceling headphones with exceptional sound quality and all-day comfort. Features advanced processor technology for premium listening experience during travel, work, and daily activities.",
    "technical_specs": {
      "driver": "30mm dynamic drivers",
      "frequency_response": "4Hz-40kHz",
      "battery": {
        "anc_on": "30 hours",
        "anc_off": "40 hours",
        "quick_charge": "3min = 3hrs playback"
      },
      "weight": "250g",
      "connectivity": {
        "bluetooth_version": "5.2",
        "features": ["multipoint connection"]
      },
      "microphones": {
        "total": 8,
        "beamforming": 4,
        "feedforward_feedback": 4
      }
    }
  },
  "customer_feedback": {
    "reviews": [
      {
        "review_text": "Amazing noise cancellation - perfect for flights and commuting. Sound quality is crisp and balanced.",
        "created_on": 1709304600,
        "user_name": "runningenthusiast"
      },
      {
        "review_text": "Battery life is exactly as advertised. Comfortable for long listening sessions without fatigue.",
        "created_on": 1709218200,
        "user_name": "techreview"
      },
      {
        "review_text": "The multipoint connection works flawlessly between my laptop and phone. Great for work calls.",
        "created_on": 1709131800,
        "user_name": "smartgadgets"
      }
    ],
    "rating": 4.6
  },
  "product_info": {
    "category": "Audio",
    "brand": "AudioTech",
    "price": 349.99,
    "in_stock": true,
    "release_year": 2024
  }
}
Next, create a repository with a schema that defines how to search and filter your product data. The schema needs to follow these requirements:
  • All field paths must start with $. (for example, $.description or $.price).
  • At least one field must be marked as embedded ("embedded": True in Python, "embedded": true in API).
  • Field names can only contain letters, numbers, and underscores.
import pinecone

# Initialize the client
pc = pinecone.Pinecone(api_key="YOUR_API_KEY")

# Create a repository
repository = pc.create_repository(
    name="product-catalog",
    spec={
        "serverless": {
            "cloud": "aws",
            "region": "us-east-1"
        }
    },
    schema={
        "fields": {
            # Embedded fields for search
            "$.details.description": {
                "embedded": True,
                "type": "string",
                "description": "Main product description for search"
            },
            "$.customer_feedback.reviews[*].review_text": {
                "embedded": True,
                "type": "string",
                "description": "Customer review text for sentiment and feature analysis"
            },
            # Metadata fields (filterable by default)
            "$.product_info.category": {
                "type": "string",
                "description": "Product category"
            },
            "$.product_info.brand": {
                "type": "string",
                "description": "Product brand name"
            },
            "$.product_info.price": {
                "type": "float",
                "description": "Product price in USD"
            },
            "$.customer_feedback.rating": {
                "type": "float",
                "description": "Average customer rating (1-5 stars)"
            },
            "$.product_info.in_stock": {
                "type": "bool",
                "description": "Product availability status"
            },
            "$.product_info.release_year": {
                "type": "integer",
                "description": "Product release year"
            }
        }
    }
)

​
3. Upsert documents

Add JSON documents to your repository. Create a repository client using the host URL from the previous step, then upsert documents one at a time. 
# Create a repository client
repo_client = pc.Repository(host=repository.host)

# Add comprehensive product documents to the catalog
headphones = {
    "_id": "headphones_at_wh1000xm5",
    "name": "AudioTech WH-1000XM5 Wireless Headphones",
    "details": {
        "description": "Industry-leading noise canceling headphones with exceptional sound quality and all-day comfort. Features advanced processor technology for premium listening experience during travel, work, and daily activities.",
        "technical_specs": {
            "driver": "30mm dynamic drivers",
            "frequency_response": "4Hz-40kHz",
            "battery": {
                "anc_on": "30 hours",
                "anc_off": "40 hours",
                "quick_charge": "3min = 3hrs playback"
            },
            "weight": "250g",
            "connectivity": {
                "bluetooth_version": "5.2",
                "features": ["multipoint connection"]
            },
            "microphones": {
                "total": 8,
                "beamforming": 4,
                "feedforward_feedback": 4
            }
        }
    },
    "customer_feedback": {
        "reviews": [
            {
                "review_text": "Amazing noise cancellation - perfect for flights and commuting. Sound quality is crisp and balanced.",
                "created_on": 1709823000,
                "user_name": "skytraveler42"
            },
            {
                "review_text": "Battery life is exactly as advertised. Comfortable for long listening sessions without fatigue.",
                "created_on": 1709736600,
                "user_name": "audiophile99" 
            },
            {
                "review_text": "The multipoint connection works flawlessly between my laptop and phone. Great for work calls.",
                "created_on": 1709650200,
                "user_name": "techworker2024"
            },
            {
                "review_text": "Touch controls are intuitive once you get used to them. Quick charge feature is a lifesaver.",
                "created_on": 1709563800,
                "user_name": "gadgetguru"
            },
            {
                "review_text": "Build quality feels premium and durable. Worth the investment for daily use.",
                "created_on": 1709477400,
                "user_name": "qualityseeker"
            }
        ],
        "rating": 4.6
    },
    "product_info": {
        "category": "Audio",
        "brand": "AudioTech",
        "price": 349.99,
        "in_stock": True,
        "release_year": 2024
    }
}

smartphone = {
    "_id": "smartphone_techcorp_x12", 
    "name": "TechCorp X12 Pro Smartphone",
    "details": {
        "description": "Flagship smartphone with professional camera system, all-day battery, and premium build quality. Advanced computational photography and 5G connectivity for power users.",
        "technical_specs": {
            "display": "6.7-inch OLED, 120Hz ProMotion",
            "processor": "A17 Pro chip with Neural Engine",
            "camera": "Triple 48MP system with 5x telephoto",
            "storage": {
                "options": ["256GB", "512GB", "1TB"]
            },
            "battery": {
                "life": "All-day battery",
                "charging": "25W fast charging"
            },
            "connectivity": {
                "cellular": "5G",
                "wifi": "Wi-Fi 6E",
                "bluetooth": "5.3"
            }
        }
    },
    "customer_feedback": {
        "reviews": [
            {
                "review_text": "Camera quality is outstanding, especially in low light conditions. Portrait mode is professional-grade.",
                "created_on": 1709650200,
                "user_name": "photopro"
            },
            {
                "review_text": "Performance is buttery smooth. No lag even with demanding apps and games running simultaneously.",
                "created_on": 1709563800,
                "user_name": "poweruser"
            },
            {
                "review_text": "Battery easily lasts a full day with heavy usage. Fast charging is convenient for quick top-ups.",
                "created_on": 1709477400,
                "user_name": "techreview"
            },
            {
                "review_text": "Build quality feels solid and premium. The titanium finish resists fingerprints well.",
                "created_on": 1709391000,
                "user_name": "designlover"
            },
            {
                "review_text": "Face ID works perfectly even with masks. The display is incredibly vibrant and sharp.",
                "created_on": 1709304600,
                "user_name": "gadgetfan"
            }
        ],
        "rating": 4.8
    },
    "product_info": {
        "category": "Mobile",
        "brand": "TechCorp", 
        "price": 999.99,
        "in_stock": True,
        "release_year": 2024
    }
}

smartwatch = {
    "_id": "smartwatch_techcorp_w3", 
    "name": "TechCorp W3 Pro Smartwatch",
    "details": {
        "description": "Advanced fitness and health tracking smartwatch with always-on display and premium design. Features comprehensive health monitoring, GPS tracking, and seamless smartphone integration.",
        "technical_specs": {
            "display": "1.9-inch AMOLED, Always-On",
            "sensors": {
                "health": ["Heart Rate", "ECG", "Blood Oxygen", "Temperature"],
                "fitness": ["GPS", "Accelerometer", "Gyroscope", "Altimeter"]
            },
            "battery": {
                "life": "Up to 18 days",
                "charging": "Magnetic fast charging"
            },
            "water_resistance": "5ATM",
            "connectivity": {
                "bluetooth": "5.2",
                "wifi": "802.11 b/g/n"
            }
        }
    },
    "customer_feedback": {
        "reviews": [
            {
                "review_text": "Battery life is incredible - almost three weeks with normal use. The always-on display is bright and clear.",
                "created_on": 1709477400,
                "user_name": "poweruser"
            },
            {
                "review_text": "Health tracking features are comprehensive and accurate. Sleep tracking gives detailed insights.",
                "created_on": 1709391000,
                "user_name": "fitnessfreak"
            },
            {
                "review_text": "GPS accuracy is spot-on for runs and bike rides. The fitness metrics are perfect for training.",
                "created_on": 1709304600,
                "user_name": "runningenthusiast"
            },
            {
                "review_text": "Build quality is excellent. The titanium case feels premium and durable.",
                "created_on": 1709218200,
                "user_name": "techreview"
            },
            {
                "review_text": "Smart notifications work flawlessly. The UI is intuitive and responsive.",
                "created_on": 1709131800,
                "user_name": "smartgadgets"
            }
        ],
        "rating": 4.7
    },
    "product_info": {
        "category": "Wearables",
        "brand": "TechCorp",
        "price": 299.99,
        "in_stock": True,
        "release_year": 2024
    }
}

# Add products to the catalog
upsert_headphones = repo_client.upsert(
    namespace="products",
    document=headphones
)

upsert_smartphone = repo_client.upsert(
    namespace="products",
    document=smartphone
)

upsert_smartwatch = repo_client.upsert(
    namespace="products",
    document=smartwatch
)

​
4. Query the repository

Now you can search your catalog using natural language queries that customers would actually use on an e-commerce site. Context Engine automatically searches across all embedded fields — descriptions and reviews — to find relevant products. You can also apply filters on specific fields like:
  • Price ranges ($.product_info.price)
  • Product categories ($.product_info.category)
  • Availability ($.product_info.in_stock)
  • Review dates ($.customer_feedback.reviews[*].created_on)
  • Brands ($.product_info.brand)
# Basic search: Customer searching for travel headphones
travel_search = repo_client.search(
    namespace="products",
    query_str="noise cancelling headphones for travel", 
    top_k=2
)

# Search with filters: Reviews from the last year
filtered_search = repo_client.search(
    namespace="products",
    query_str="premium sound quality professional",
    top_k=2,
    filter={
        "path:$.customer_feedback.reviews[*].created_on": { 
            "$gte": 1709218200 # Matches the middle review timestamp, excluding the oldest review
        }
    }
)

print("# Basic search: Customer searching for travel headphones:")
print(travel_search)
print("\n")
print("# Search with filters: Reviews from the last year:")
print(filtered_search)
Context Engine returns matching documents with their complete content, including all fields like descriptions, specifications, and customer reviews. The results are ordered by relevance to the query: 
# Basic search: Customer searching for travel headphones:
{
    'results': [{
        'document': {
            'customer_feedback': {
                'rating': 4.6,
                'reviews': [
                    {
                        'created_on': 1709823000,
                        'review_text': 'Amazing noise cancellation - perfect for flights and commuting. Sound quality is crisp and balanced.',
                        'user_name': 'skytraveler42'
                    },
                    {
                        'created_on': 1709736600, 
                        'review_text': 'Battery life is exactly as advertised. Comfortable for long listening sessions without fatigue.',
                        'user_name': 'audiophile99'
                    },
                    {
                        'created_on': 1709650200,
                        'review_text': 'The multipoint connection works flawlessly between my laptop and phone. Great for work calls.',
                        'user_name': 'techworker2024'
                    },
                    {
                        'created_on': 1709563800,
                        'review_text': 'Touch controls are intuitive once you get used to them. Quick charge feature is a lifesaver.',
                        'user_name': 'gadgetguru'
                    },
                    {
                        'created_on': 1709477400,
                        'review_text': 'Build quality feels premium and durable. Worth the investment for daily use.',
                        'user_name': 'qualityseeker'
                    }
                ]
            },
            'details': {
                'description': 'Industry-leading noise canceling headphones with exceptional sound quality and all-day comfort. Features advanced processor technology for premium listening experience during travel, work, and daily activities.',
                'technical_specs': {
                    'battery': {
                        'anc_off': '40 hours',
                        'anc_on': '30 hours',
                        'quick_charge': '3min = 3hrs playback'
                    },
                    'connectivity': {
                        'bluetooth_version': '5.2',
                        'features': ['multipoint connection']
                    },
                    'driver': '30mm dynamic drivers',
                    'frequency_response': '4Hz-40kHz',
                    'microphones': {
                        'beamforming': 4,
                        'feedforward_feedback': 4,
                        'total': 8
                    },
                    'weight': '250g'
                }
            },
            'name': 'AudioTech WH-1000XM5 Wireless Headphones',
            'product_info': {
                'brand': 'AudioTech',
                'category': 'Audio',
                'in_stock': True,
                'price': 349.99,
                'release_year': 2024
            }
        },
        'id': 'headphones_at_wh1000xm5',
        'matching_chunks': [
            {
                'origin_json_path': '$.details.description',
                'score': 11.903067
            },
            {
                'origin_json_path': '$.customer_feedback.reviews[0].review_text',
                'score': 5.7049017
            }
        ],
        'score': 11.903067
    }],
    'usage': {
        'embed_total_tokens': 8,
        'kb_read_units': 4.3
    }
}

# Search with filters: Reviews from the last year:
{
    'results': [
        {
            'document': {
                'customer_feedback': {
                    'rating': 4.6,
                    'reviews': [
                        {
                            'review_text': 'Amazing noise cancellation - perfect for flights and commuting. Sound quality is crisp and balanced.',
                            'created_on': 1709304600,
                            'user_name': 'runningenthusiast'
                        },
                        {
                            'review_text': 'Battery life is exactly as advertised. Comfortable for long listening sessions without fatigue.',
                            'created_on': 1709218200,
                            'user_name': 'techreview'
                        },
                        {
                            'review_text': 'The multipoint connection works flawlessly between my laptop and phone. Great for work calls.',
                            'created_on': 1709131800,
                            'user_name': 'smartgadgets'
                        }
                    ]
                },
                'details': {
                    'description': 'Industry-leading noise canceling headphones with exceptional sound quality and all-day comfort. Features advanced processor technology for premium listening experience during travel, work, and daily activities.',
                    'technical_specs': {
                        'driver': '30mm dynamic drivers',
                        'frequency_response': '4Hz-40kHz',
                        'battery': {
                            'anc_on': '30 hours',
                            'anc_off': '40 hours',
                            'quick_charge': '3min = 3hrs playback'
                        },
                        'weight': '250g',
                        'connectivity': {
                            'bluetooth_version': '5.2',
                            'features': ['multipoint connection']
                        },
                        'microphones': {
                            'total': 8,
                            'beamforming': 4,
                            'feedforward_feedback': 4
                        }
                    }
                },
                'name': 'AudioTech WH-1000XM5 Wireless Headphones',
                'product_info': {
                    'category': 'Audio',
                    'brand': 'AudioTech',
                    'price': 349.99,
                    'in_stock': true,
                    'release_year': 2024
                }
            },
            'id': 'headphones_at_wh1000xm5',
            'matching_chunks': [
                {
                    'origin_json_path': '$.customer_feedback.reviews[0].review_text',
                    'score': 6.9834785
                }
            ],
            'score': 6.9834785
        },
        {
            'document': {
                'customer_feedback': {
                    'rating': 4.8,
                    'reviews': [
                        {
                            'created_on': 1709650200,
                            'review_text': 'Camera quality is outstanding, especially in low light conditions. Portrait mode is professional-grade.',
                            'user_name': 'photopro'
                        },
                        {
                            'created_on': 1709563800,
                            'review_text': 'Performance is buttery smooth. No lag even with demanding apps and games running simultaneously.',
                            'user_name': 'poweruser'
                        },
                        {
                            'created_on': 1709477400,
                            'review_text': 'Battery easily lasts a full day with heavy usage. Fast charging is convenient for quick top-ups.',
                            'user_name': 'techreview'
                        },
                        {
                            'created_on': 1709391000,
                            'review_text': 'Build quality feels solid and premium. The titanium finish resists fingerprints well.',
                            'user_name': 'designlover'
                        },
                        {
                            'created_on': 1709304600,
                            'review_text': 'Face ID works perfectly even with masks. The display is incredibly vibrant and sharp.',
                            'user_name': 'gadgetfan'
                        }
                    ]
                },
                'details': {
                    'description': 'Flagship smartphone with professional camera system, all-day battery, and premium build quality. Advanced computational photography and 5G connectivity for power users.',
                    'technical_specs': {
                        'battery': {
                            'charging': '25W fast charging',
                            'life': 'All-day battery'
                        },
                        'camera': 'Triple 48MP system with 5x telephoto',
                        'connectivity': {
                            'bluetooth': '5.3',
                            'cellular': '5G',
                            'wifi': 'Wi-Fi 6E'
                        },
                        'display': '6.7-inch OLED, 120Hz ProMotion',
                        'processor': 'A17 Pro chip with Neural Engine',
                        'storage': {
                            'options': ['256GB', '512GB', '1TB']
                        }
                    }
                },
                'name': 'TechCorp X12 Pro Smartphone',
                'product_info': {
                    'brand': 'TechCorp',
                    'category': 'Mobile',
                    'in_stock': True,
                    'price': 999.99,
                    'release_year': 2024
                }
            },
            'id': 'smartphone_techcorp_x12',
            'matching_chunks': [
                {
                    'origin_json_path': '$.customer_feedback.reviews[0].review_text',
                    'score': 6.81182
                }
            ],
            'score': 6.81182
        }
    ],
    'usage': {
        'embed_total_tokens': 7,
        'kb_read_units': 7.6
    }
}

​
5. Manage repositories

Use repository management operations to monitor your product catalog, inspect the schema structure, and manage your product data infrastructure.

​
List repositories

View all repositories in your project: 
# List all repositories in your project
repositories = pc.list_repositories()

print(f"Repositories: {repositories}")

​
Describe a repository

Get detailed information about a repository, including its schema: 
# Get repository details
repo_info = pc.describe_repository("product-catalog")

print(f"Repository: {repo_info}")

​
Delete a repository

Remove a repository when you no longer need it. This operation removes all documents and associated data permanently.
Deleting a repository permanently removes all documents and cannot be undone.
# Delete a repository and all its documents
pc.delete_repository("product-catalog")

​
6. Manage documents

Manage individual products in your catalog for inventory updates, content verification, and catalog maintenance.

​
List documents

Retrieve a paginated list of documents in your repository. This operation returns document metadata but excludes embedded content by default to improve performance: 
# List documents in a repository
documents = repository.list_documents(
    namespace="products",
    limit=50  # Paginated results
)

print(f"Documents: {documents}")    # Note: embedded content is excluded by default to minimize response size

​
Fetch a document

Retrieve a specific document by its ID. This operation returns document metadata but excludes embedded content by default to improve performance: 
# Fetch a document by ID 
document = repository.fetch_document(
    namespace="products",
    document_id="product_123"
)

print(f"Document: {document}")

​
Delete a document

Remove one or more documents from your repository. This operation deletes the document and all its associated chunks and embedded content permanently: 
# Delete a single document
repository.delete_document(
    namespace="products",
    document_id="headphones_at_wh1000xm5"
)

# Delete multiple documents by ID
repository.delete_documents(
    namespace="products",
    document_ids=["headphones_at_wh1000xm5", "smartphone_techcorp_x12"]
)
Deleting a document removes all associated chunks and embedded content. This operation cannot be undone.

​
Limitations

Pinecone Context Engine alpha release has several limitations to be aware of: General limitations
  • Pinecone Context Engine is available only on the unstable version of the API or using an alpha release of the Python SDK.
  • The alpha release does not include the full capabilities of Context Engine. It provides document repository functionality with basic retrieval. Advanced query planning, intelligent query rewriting, and sophisticated context expansion will be available in future releases.
Schema and field limitations
  • Field naming: Field keys must be alphanumeric (A-Z, a-z, 0-9, _) and must not start with a number or underscore. Unicode field names are not supported.
  • Embedding models: Only supports default embedding models (llama-text-embed-v2 for dense, pinecone-sparse-english-v0 for sparse). Custom embedding models are not available.
  • Schema immutability: Schema updates and field modifications are not supported in the alpha release.
  • Required embedded field: Each schema must have at least one field marked as embedded.
  • Array limitations: Arrays of arrays (nested arrays) are not supported.
Document and ingestion limitations
  • Synchronous ingestion: All document operations are synchronous, targeting 1-2 second response times.
  • No partial updates: Document updates require full document replacement; partial field updates are not supported.
  • No automatic schema inference: Schema must be explicitly defined during repository creation. All fields are "filterable": true by default unless explicitly set to "filterable": false. Use "filterable": false for fields containing large content (like binary data or images) to save metadata space and reduce costs.
Search and query limitations
  • Basic chunk expansion: Uses simple chunk count strategy with no configuration options.
  • No query planning: Advanced query planning and rewriting features are not available in the alpha.
⌘I