Column name | Parquet type | Description |
---|---|---|
id | STRING | Required. The unique identifier for each record. |
values | LIST<FLOAT> | Required. A list of floating-point values that make up the dense vector embedding. |
sparse_values | STRUCT<indices: LIST<UINT_32>, values: LIST<FLOAT>> | Optional. A list of floating-point values that make up the sparse vector embedding. To omit from specific rows, use NULL . |
metadata | STRING | Optional. Additional metadata for each record. To omit from specific rows, use NULL . |
Column name | Parquet type | Description |
---|---|---|
id | STRING | Required. The unique identifier for each record. |
sparse_values | LIST<INT> and LIST<FLOAT> | Required. A list of floating-point values (sparse values) and a list of integer values (sparse indices) that make up the sparse vector embedding. |
metadata | STRING | Optional. Additional metadata for each record. To omit from specific rows, use NULL . |
start_import
operation to start an asynchronous import of vectors from object storage into an index.
uri
, specify the URI of the bucket and import directory containing the namespaces and Parquet files you want to import, for example, s3://BUCKET_NAME/IMPORT_DIR
for Amazon S3 or gs://BUCKET_NAME/IMPORT_DIR
for Google Cloud Storage.
integration_id
, set the Integration ID of the Amazon S3 or Google Cloud Storage integration you created. The ID is found on the Storage integrations page of the Pinecone console.
error_mode
, use CONTINUE
or ABORT
.
ABORT
, the operation will stop if any records fail to import.CONTINUE
, the operation will continue on error and complete, but there will not be any notification about which records, if any, failed to import. To see how many records were successfully imported, use the describe_import
operation.operation_id
that you can use to check the status of the import:
InProgress
, but 100.0
percent complete. Once all the imported records are indexed and fully available for querying, the import operation will be set to Completed
.
list_imports
operation to list all of the recent and ongoing imports. By default, the operation returns up to 100 imports per page. If the limit
parameter is passed, the operation returns up to that number of imports per page instead. For example, if limit=3
, up to 3 imports are returned per page. Whenever there are additional imports to return, the response includes a pagination_token
for fetching the next page of imports.
list_import
paginates automatically.describe_import
operation to get details about a specific import.
cancel_import
operation cancels an import if it is not yet finished. It has no effect if the import is already complete.
Metric | Limit |
---|---|
Max size per import request | 2 TB or 200,000,000 records |
Max namespaces per import request | 10,000 |
Max files per import request | 100,000 |
Max size per file | 10 GB |