usage
parameter with prompt_tokens
, completion_tokens
, and total_tokens
generated.
prompt_tokens
are based on the messages sent to the assistant and the context snippets retrieved from the assistant and sent to a model. Messages sent to the assistant can include messages from the chat history in addition to the newest message.
prompt_tokens
appear as Assistants Input Tokens on invoices.
completion_tokens
are based on the answer from the model.
completion_tokens
appear as Assistants Output Tokens on invoices.
total_tokens
is the sum of prompt_tokens
and completion_tokens
.