Chat with an assistant

After uploading files to an assistant, there are two ways to chat with the assistant:

• chat_assistant: This is the recommended way to chat with an assistant, as it offers more functionality and control over the assistant’s responses and references than the chat_completion_assistant operation. For more information, see Chat with an assistant.

• chat_completion_assistant: This operation is based on the OpenAI Chat Completion API, a commonly used and adopted API. It is useful if you need inline citations or OpenAI-compatible responses, but has limited functionality compared to the chat_assistant operation. For more information, see Chat through an OpenAI-compatible interface.

​

Chat with an assistant

To chat with a Pinecone assistant, use the chat_assistant endpoint. It returns either a JSON object or a text stream.

This is the recommended way to chat with an assistant, as it offers more functionality and control over the assistant’s responses and references. However, if you need your assistant to be OpenAI-compatible or need inline citations, use the chat_completion_assistant endpoint.

​

Default response

The following example sends a message and requests a response in the default format (JSON object):

The example above returns a result like the following:

{
    'id': '0000000000000000163008a05b317b7b', 
    'model': 'gpt-4o-2024-05-13', 
    'usage': {
        'prompt_tokens': 9259, 
        'completion_tokens': 30, 
        'total_tokens': 9289
        }, 
        'message': {
            'content': 'The Chief Financial Officer (CFO) of Netflix is Spencer Neumann.', 
            'role': '"assistant"'
            }, 
            'finish_reason': 'stop', 
            'citations': [
                {
                    'position': 63, 
                    'references': [
                        {
                            'pages': [78, 72, 79], 
                            'file': {
                                'name': 'Netflix-10-K-01262024.pdf', 
                                'id': '76a11dd1...', 
                                'metadata': {
                                    'company': 'netflix', 
                                    'document_type': 'form 10k'
                                    }, 
                                    'created_on': '2024-12-06T01:29:07.369208590Z', 
                                    'updated_on': '2024-12-06T01:29:50.923493799Z', 
                                    'status': 'Available', 
                                    'percent_done': 1.0, 
                                    'signed_url': 'https://storage.googleapis.com/...', 
                                    "error_message": null, 
                                    'size': 1073470.0
                                }
                            }
                        ]
                    }
                ]
            }

​

Streaming response

The following example sends a message and requests a streaming response:

The example above returns a result like the following:

data:{"type":"message_start","id":"0000000000000000111b35de85e8a8f9","model":"gpt-4o-2024-05-13","role":"assistant"}

data:{"type":"content_chunk","id":"0000000000000000111b35de85e8a8f9","model":"gpt-4o-2024-05-13","delta":{"content":"The"}}

...

data:{"type":"citation","id":"0000000000000000111b35de85e8a8f9","model":"gpt-4o-2024-05-13","citation":{"position":406,"references":[{"file":{"status":"Available","id":"ae79e447-b89e-4994-994b-3232ca52a654","name":"Pride-and-Prejudice.pdf","size":2973077,"metadata":null,"updated_on":"2024-06-14T15:01:57.385425746Z","created_on":"2024-06-14T15:01:02.910452398Z","percent_done":0.0,"signed_url":"https://storage.googleapis.com/...", "error_message":null},"pages":[1]}]}}

data:{"type":"message_end","id":"0000000000000000111b35de85e8a8f9","model":"gpt-4o-2024-05-13","finish_reason":"stop","usage":{"prompt_tokens":9736,"completion_tokens":102,"total_tokens":9838}}

There are four types of chunks in a streaming chat response:

• Starting chunk: Includes "role":"assistant", which indicates that the assistant is responding to the user’s message.
• Content chunk: Includes a value in the content field (e.g., "content":"The"), which is part of the assistant’s streamed response to the user’s message.
• Citation chunk: Includes a citation to the document that the assistant used to generate the response.
• Ending chunk: Includes "finish_reason":"stop", which indicates that the assistant has finished responding to the user’s message.

​

JSON response

The following example uses the json_response parameter to instruct the assistant to return a JSON response:

​

Chat through an OpenAI-compatible interface

The chat_completion_assistant endpoint is based on the OpenAI Chat Completion API, a commonly used and adopted API. It is useful if you need inline citations or OpenAI-compatible responses, but has limited functionality compared to the chat_assistant endpoint. It returns either a JSON object or a text stream.

​

Default response

The following example sends a message and requests a response in the default format (JSON object):

The example above returns a result like the following:

{"chat_completion":
  {
    "id":"chatcmpl-9OtJCcR0SJQdgbCDc9JfRZy8g7VJR",
    "choices":[
      {
        "finish_reason":"stop",
        "index":0,
        "message":{
          "role":"assistant",
          "content":"The maximum height of a red pine (Pinus resinosa) is up to 25 meters."
        }
      }
    ],
    "model":"my_assistant"
  }
}

​

Streaming response

The following example sends a messages and requests a streaming response:

The example above returns a result like the following:

{
  'id': '000000000000000009de65aa87adbcf0', 
  'choices': [
      {
      'index': 0, 
      'delta': 
        {
        'role': 'assistant', 
        'content': 'The'
        }, 
      'finish_reason': None
      }
    ], 
  'model': 'gpt-4o-2024-05-13'
}

...

{
  'id': '00000000000000007a927260910f5839',
  'choices': [
      {
      'index': 0,
      'delta':
        {
          'role': '', 
          'content': 'The'
        }, 
      'finish_reason': None
      }
    ], 
  'model': 'gpt-4o-2024-05-13'
}

...

{
  'id': '00000000000000007a927260910f5839', 
  'choices': [
    {
      'index': 0, 
      'delta': 
        {
        'role': None, 
        'content': None
        }, 
      'finish_reason': 'stop'
      }
    ], 
  'model': 'gpt-4o-2024-05-13'
}

There are three types of chunks in a chat completion response:

• Starting chunk: Includes "role":"assistant", which indicates that the assistant is responding to the user’s message.
• Content chunk: Includes a value in the content field (e.g., "content":"The"), which is part of the assistant’s streamed response to the user’s message.
• Ending chunk: Includes "finish_reason":"stop", which indicates that the assistant has finished responding to the user’s message.

​

Provide conversation history in a chat request

Models lack memory of previous requests, so any relevant messages from earlier in the conversation must be present in the messages object.

In the following example, the messages object includes prior messages that are necessary for interpreting the newest message.

The above example request returns a response like the following:

{"chat_completion":
  {
    "id":"chatcmpl-9OtJCcR0SJQdgbCDc9JfRZy8g7VJR",
    "choices":[
      {
        "finish_reason":"stop",
        "index":0,
        "message":{
          "role":"assistant",
          "content":"The maximum diameter of a red pine (Pinus resinosa) is 75 centimeters [1, pp. 1]"
        }
      }
    ],
    "model":"my_assistant"
  }
}

​

Filter chat with metadata

You can filter which documents to use for chat completions. The following example filters the responses to use only documents that include the metadata "resource": "encyclopedia".

​

Choose a model for your assistant

Pinecone Assistant uses the gpt-4o model by default. Alternatively, you can use the claude-3-5-sonnet model. Select the LLM to use by setting the model parameter in the request:

​

Extract the response content

Both the chat_assistant and chat_completion_assistant operations return a JSON response object containing the assistant’s chat response along with other information. The message string is contained in the following JSON object:

• choices.[0].message.content for a JSON chat response
• choices[0].delta.content for a streaming chat response

You can extract the message content and print it to the console:

• JSON response
• Streaming response

A red pine, scientifically known as *Pinus resinosa*, is a medium-sized tree that can grow up to 25 meters high and 75 centimeters in diameter. [1, pp. 1]