Multi-tenant RAG

Built with

Pinecone Database
Vercel AI SDK and OpenAI
Next.js and Tailwind CSS
Node.js 20 or later

Run the sample app

To create the app from the template, use the create-pinecone-app CLI:

Terminal

npx -y create-pinecone-app@latest --template namespace-notes

Get your Pinecone API key

You need an API key to make calls to your Pinecone project:

Then copy your generated key:

.env

PINECONE_API_KEY="{{YOUR_API_KEY}}"

You can also copy an existing key from the API keys page in the Pinecone console.

Create a Pinecone index

Create a serverless index that stores dense vectors, with the following settings:

Dimension: 1536. You can change this if you also change the default embedding model.
Metric: cosine
Region: us-east-1

You can create the index in the Pinecone console or with the API. For more information, see Create an index.

Clone the repository

The app requires Node.js 20 or later. Clone the sample-apps repository and go to the namespace-notes directory.The client and the server run separately, so open two terminals, one for each.

Set up and start the client

From the project root directory, install the client dependencies:

Terminal

cd client && npm install

In the client .env file, set your OpenAI API key and the server URL:

.env

# You must first activate a Billing Account here: https://platform.openai.com/account/billing/overview
# Then get your OpenAI API Key here: https://platform.openai.com/account/api-keys
OPENAI_API_KEY="YOUR_OPENAI_API_KEY"

# The URL of the server (only used for development)
SERVER_URL="http://localhost:4001"

Start the client:

Terminal

npm run dev

Set up and start the server

From the project root directory, install the server dependencies:

Terminal

cd server && npm install

In the server .env file, set your Pinecone and OpenAI values:

.env

PINECONE_API_KEY="YOUR_PINECONE_API_KEY"
PINECONE_INDEX_NAME="YOUR_INDEX_NAME"
OPENAI_API_KEY="YOUR_OPENAI_API_KEY"

# Digital Ocean Spaces (OPTIONAL - for public file hosting)
DO_SPACES_ACCESS_KEY_ID="YOUR_DO_SPACES_ACCESS_KEY_ID"
DO_SPACES_SECRET_ACCESS_KEY="YOUR_DO_SPACES_SECRET_ACCESS_KEY"
DO_SPACES_ENDPOINT="YOUR_DO_SPACES_ENDPOINT"
DO_SPACES_BUCKET_NAME="YOUR_DO_SPACES_BUCKET_NAME"

Digital Ocean Spaces is optional. By default, the server stores uploaded document files locally, so you can run the project without it.

Start the server:

Terminal

npm run dev

Project structure

Namespace Notes architecture: a Next.js client calls an Express server, which chunks, embeds, and upserts documents into a Pinecone namespace per workspace

The app separates the frontend client from the backend server, so you can swap either one for a stack of your choice.

Frontend client: A Next.js app with Tailwind CSS and components from the Vercel AI SDK that power the chatbot. Its API routes call the server to fetch document references and context for the UI and the LLM. The client stores workspace information in local storage.
Backend server: A Node.js and Express server that handles file uploads, validation, chunking, upserts, and context retrieval.

Multi-tenant RAG methodology

The app uses a basic RAG architecture and achieves multitenancy with namespaces. The client uploads files to the server, which chunks and embeds them and upserts them into Pinecone.

Tenant isolation

Each workspace has its own namespace, which keeps context separate between workspaces. When the server adds documents, it uses the existing namespace ID or generates a new one if the workspace is new:

TypeScript

/**
* Adds a new document.
* @param req - The request object.
* @param res - The response object.
* @returns A promise that resolves to the added document.
*/
async addDocuments(req: Request, res: Response) {
// This is effectively the ID of the workspace / tenant
let namespaceId = req.body.namespaceId;
//...

Chunking

The app uses pdf-parse to stream and parse PDF content, then splits it with a best-effort paragraph chunking strategy. A minChunkSize and maxChunkSize account for documents with longer or shorter paragraphs, so each chunk is large enough to be useful as record metadata that the LLM reads during retrieval:

TypeScript

/**
 * Splits a given text into chunks of 1 to many paragraphs.
*
* @param text - The input text to be chunked.
* @param maxChunkSize - The maximum size (in characters) allowed for each chunk. Default is 1500.
* @param minChunkSize - The minimum size (in characters) required for each chunk. Default is 500.
* @returns An array of chunked text, where each chunk contains 1 or multiple "paragraphs"
*/
function chunkTextByMultiParagraphs(
text: string,
maxChunkSize = 1500,
minChunkSize = 500
): string[] {
const chunks: string[] = [];
let currentChunk = "";

let startIndex = 0;
while (startIndex < text.length) {
    let endIndex = startIndex + maxChunkSize;
    if (endIndex >= text.length) {
    endIndex = text.length;
    } else {
    // Just using this to find the nearest paragraph boundary
    const paragraphBoundary = text.indexOf("\n\n", endIndex);
    if (paragraphBoundary !== -1) {
        endIndex = paragraphBoundary;
    }
    }

    const chunk = text.slice(startIndex, endIndex).trim();
    if (chunk.length >= minChunkSize) {
    chunks.push(chunk);
    currentChunk = "";
    } else {
    currentChunk += chunk + "\n\n";
    }

    startIndex = endIndex + 1;
}

if (currentChunk.length >= minChunkSize) {
    chunks.push(currentChunk.trim());
} else if (chunks.length > 0) {
    chunks[chunks.length - 1] += "\n\n" + currentChunk.trim();
} else {
    chunks.push(currentChunk.trim());
}

return chunks;
}

Embedding

The server embeds the chunks in batches with text-embedding-3-small:

TypeScript

/**
 * Embed a piece of text using an embedding model or service.
*
* @param text The text to embed.
* @returns The embedded representation of the text.
*/
export async function embedChunks(chunks: string[]): Promise<any> {
// You can use any embedding model or service here.
// In this example, we use OpenAI's text-embedding-3-small model.
const openai = new OpenAI({
    apiKey: config.openAiApiKey,
    organization: config.openAiOrganizationId,
});
try {
    const response = await openai.embeddings.create({
    model: "text-embedding-3-small",
    input: chunks,
    encoding_format: "float",
    dimensions: 1536,
    });
    return response.data;
} catch (error) {
    console.error("Error embedding text with OpenAI:", error);
    throw error;
}
}

RAG document management

To store multiple documents in one namespace, the app needs a way to target the chunks that belong to a particular document. It does this with ID prefixes. The server generates a document ID for each uploaded document and, before the upsert, uses it as a prefix for each chunk ID. The following code builds each chunk ID from the document ID and the chunk index, separated by ::

TypeScript

// Combine the chunks and their corresponding embeddings
// Construct the id prefix using the documentId and the chunk index
for (let i = 0; i < chunks.length; i++) {
document.chunks.push({
    id: `${document.documentId}:${i}`,
    values: embeddings[i].embedding,
    text: chunks[i],
});

The prefix makes targeted document updates and deletions possible.

Upsert

The server upserts the embeddings as PineconeRecord objects into the namespace for the tenant. Each record stores the chunk text and the document URL as metadata, which the retrieval system uses later:

TypeScript

/**
 * Upserts a document into the specified Pinecone namespace.
* @param document - The document to upsert.
* @param namespaceId - The ID of the namespace.
*/
async upsertDocument(document: Document, namespaceId: string) {
    // Adjust to use namespaces if you're organizing data that way
    const namespace = index.namespace(namespaceId);

    const vectors: PineconeRecord<RecordMetadata>[] = document.chunks.map(
    (chunk) => ({
        id: chunk.id,
        values: chunk.values,
        metadata: {
        text: chunk.text,
        referenceURL: document.documentUrl,
        },
    })
    );

    // Batch the upsert operation
    const batchSize = 200;
    for (let i = 0; i < vectors.length; i += batchSize) {
    const batch = vectors.slice(i, i + batchSize);
    await namespace.upsert(batch);
    }
}

Context

When a user asks a question in the chat component, the Vercel AI SDK calls the /chat endpoint for retrieval. The context route returns the top_k most similar results from Pinecone.The server wraps those results in a CONTEXT BLOCK with system prompt instructions for the LLM to use in its response.LLMs have different context windows, so your choice of LLM affects the top_k value and the chunk size you should use. If the context block and prompt are longer than the LLM’s context window, the LLM won’t use all of it when it generates a response.The following code builds the prompt with the context block:

TypeScript

import { getContext } from "./context";

export async function createPrompt(messages: any[], namespaceId: string) {
try {
    // Get the last message
    const lastMessage = messages[messages.length - 1]["content"];

    // Get the context from the last message
    const context = await getContext(lastMessage, namespaceId);

    const prompt = [
    {
        role: "system",
        content: `AI assistant is a brand new, powerful, human-like artificial intelligence.
        DO NOT SHARE REFERENCE URLS THAT ARE NOT INCLUDED IN THE CONTEXT BLOCK.
        AI assistant will not apologize for previous responses, but instead will indicate new information was gained.
        If user asks about or refers to the current "workspace" AI will refer to the content after START CONTEXT BLOCK and before END OF CONTEXT BLOCK as the CONTEXT BLOCK. 
        If AI sees a REFERENCE URL in the provided CONTEXT BLOCK, please use reference that URL in your response as a link reference right next to the relevant information in a numbered link format e.g. ([reference number](link))
        If link is a pdf and you are CERTAIN of the page number, please include the page number in the pdf href (e.g. .pdf#page=x ).
        If AI is asked to give quotes, please bias towards providing reference links to the original source of the quote.
        AI assistant will take into account any CONTEXT BLOCK that is provided in a conversation. It will say it does not know if the CONTEXT BLOCK is empty.
        AI assistant will not invent anything that is not drawn directly from the context.
        AI assistant will not answer questions that are not related to the context.
        START CONTEXT BLOCK
        ${context}
        END OF CONTEXT BLOCK
    `,
    },
    ];
    return { prompt };
} catch (e) {
    throw e;
}
}

Document deletion

To delete a document from a workspace, the server uses the documentId: prefix to list all the chunks for that document, then deletes them until none remain:

TypeScript

// We retrieve a paginated list of chunks from the namespace
const listResult = await namespace.listPaginated({
prefix: `${documentId}:`,
limit: limit,
paginationToken: paginationToken,
});

...

// Then we delete each of the chunks based on their ids
async deleteDocumentChunks(chunkIds: string[], namespaceId: string) {
console.log("Deleting Document Chunks")
const namespace = index.namespace(namespaceId);
await namespace.deleteMany(chunkIds);
}

Workspace deletion

To offboard a workspace, call deleteAll() on its namespace:

TypeScript

/**
 * Deletes a Pinecone namespace.
* 
* @param namespaceId - The ID of the namespace to delete all records.
* @returns A Promise that resolves when the namespace is deleted successfully.
*/
async deletePineconeNamespace(namespaceId: string) {
    console.log("Deleting Workspace")
    const namespace = index.namespace(namespaceId);
    await namespace.deleteAll();
    console.log("Workspace deleted from Pinecone successfully")
}

Further optimizations for the RAG pipeline

This is a basic RAG pipeline. Depending on your requirements, you can improve it in the following ways.

Use a reranker

A reranker sends the LLM the most relevant subset of the results from Pinecone. With a reranker, you can request a much higher top_k from Pinecone and then narrow the results to the most relevant records, ordered by relevance, while staying within the LLM’s context length.For more optimizations, see the RAG series.

Optimize the chunking strategy

This app uses a paragraph chunker, which works well for some use cases. Chunk quality often has a large effect on the quality of the whole retrieval system.For more information, see Chunking strategies.

Enrich the metadata

The metadata in this app is a reference URL to the original content and the text of the chunk. You could extract richer metadata from the PDFs, such as page count, title, and authors, to give the LLM better context. This assumes the PDFs contain that metadata and that it’s useful for your use case.For more information, see Vectorizing structured text.

Troubleshooting

If you run into problems, see the Namespace Notes README. If that doesn’t solve it, open an issue in the sample apps repository or ask in the Pinecone Discord community.

​Multi-tenant RAG

​Built with

​Run the sample app

Get your Pinecone API key

Create a Pinecone index

Clone the repository

Set up and start the client

Set up and start the server

​Project structure

​Multi-tenant RAG methodology

​Tenant isolation

​Chunking

​Embedding

​RAG document management

​Upsert

​Context

​Document deletion

​Workspace deletion

​Further optimizations for the RAG pipeline

​Use a reranker

​Optimize the chunking strategy

​Enrich the metadata

​Troubleshooting

Built with

Run the sample app

Project structure

Multi-tenant RAG methodology

Tenant isolation

Chunking

Embedding

RAG document management

Upsert

Context

Document deletion

Workspace deletion

Further optimizations for the RAG pipeline

Use a reranker

Optimize the chunking strategy

Enrich the metadata

Troubleshooting