Tools
Spark-Pinecone connector
Use the spark-pinecone
connector to efficiently create, ingest, and update vector embeddings at scale with Databricks and Pinecone.
Install the Spark-Pinecone connector
- Install the Spark-Pinecone connector as a library.
- Configure the library as follows:
-
Select File path/S3 as the Library Source.
-
Enter the S3 URI for the Pinecone assembly JAR file:
Databricks platform users must use the Pinecone assembly jar listed above to ensure that the proper dependecies are installed.
-
Click Install.
-
Batch upsert
To batch upsert embeddings to Pinecone:
For a guide on how to set up batch upserts, refer to the Databricks integration page.
Stream upsert
To stream upsert embeddings to Pinecone:
Learn more
Was this page helpful?