SAMPLE APP
Multi-tenant RAG
A simple multi-tenant RAG app to chat with documents isolated by namespace$ npx create-pinecone-app@latest --template namespace-notes
Namespace Notes is a simple multi-tenant RAG app. The application allows users to create workspaces, upload documents to Pinecone, and to feed the workspace’s chatbot with custom context.
This concept can be used to store anywhere from just a few documents, to many billions of contextual embeddings.
Built with
- Pinecone Serverless
- Vercel AI SDK + OpenAI
- Next.js + tailwind
- Node version 20 or higher
Run the sample app
The fastest way to get started is to use thecreate-pinecone-app
CLI tool to get up and running:Get your API key
You need an API key to make API calls to your Pinecone project:- Open the Pinecone console.
- Select your project.
- Go to API Keys.
- Copy your API key.
Create a Pinecone serverless index
Create a Pinecone index for this project. The index should have the following properties:- dimension:
1536
You can change this as long as you change the default embedding model. - metric:
cosine
- region:
us-east-1
Start the project
Requires Node version 20+To start the project, clone the sample-apps repo and navigate to thenamespace-notes
directory.
You will need two separate terminal instances, one for running the client and one for the server.Client setup
From the project root directory, run the following command:.env
with relevant keys:Server setup
From the project root directory, run the following command:.env
with relevant keys: You may notice that Digital Ocean Spaces is available as document storage. Using Digital Ocean Spaces is entirely optional. The project has a class defined to store document files locally on the server for quick project spin-up.
Project structure
Simple multi-tenant RAG methodology
This project uses a basic RAG architecture that achieves multitenancy through the use of namespaces. Files are uploaded to the server where they are chunked, embedded and upserted into Pinecone.Tenant isolationWe use namespaces as the mechanism to separate context between workspaces. When we add documents, we check for a namespaceId or generate a new id if the workspace is being created.pdf-parse
to stream and parse pdf content and leverage a best effort paragraph chunking strategy with a defined minChunkSize
and maxChunkSize
to
account for documents with longer or shorter paragraph sizes. This helps us provide sizable content chunks for our Pinecone record metadata which will later be used by the LLM during retreival.text-embedding-3-small
:
’ symbol.PineconeRecord
.
This allows us to provide the reference text and url as metadata for use by our retreival system./chat
endpoint for retrieval.
We then send the top_k
most similar results back from Pinecone via our context route.We populate a CONTEXT BLOCK
that is wrapped with system prompt instructions for our chosen LLM to take advantage of in the response output.It’s important to note that different LLMs will have different context windows, so your choice of LLM will influence the top_k
value you should return from Pinecone and along with the size of your chunks.
If the context block / prompt is longer than the context window of the LLM, it will not be fully included in generation results.documentId:
to identify all the chunks associated with a particular document and then we perform deletions until we have successfully deleted all document chunks.deleteAll()
on the relevant namespace.Further optimizations for the RAG pipeline
This is a relatively simple RAG pipeline - in practice there are improvements that could be made depending on a particular set of requirements.Using rerankersFor example, a reranker could be used in order to provide the most relevant set of retrieved results from Pinecone to the LLM. A reranker could allow us to increase thetop_k
requested from Pinecone significantly and then constrain the output to a highly relevant set of records ordered by relevance all while abiding by the context length restrictions of the LLM.Follow our RAG series for more optimizationsOptimizing chunking strategyThis project uses a paragraph chunker, which can provide good results for some use cases. Often, the quality of a chunk will play a significant role in the quality of the retrieval system as a whole.Learn more about various chunking strategiesEnhancing metadata structureThe metadata in this project consists simply of a reference url to the original content and the particular text snippet. You could extract richer metadata from the PDFs to provide improved context to the LLM.
This, of course, assumes a given PDF upload contains additional metadata and that it would be useful (page count, title, author(s), etc).Read more about vectorizing structured text.