Asynchronous dataset imports

Import a CSV or JSONL file into a dataset in a few steps: create the import, upload the file, check how its columns map, and confirm. Confirming queues the import and returns at once; the rows are added in the background.

Limits and endpoints

GET  /api/v1/projects/{owner_slug}/{project_slug}/datasets/limits
POST /api/v1/projects/{owner_slug}/{project_slug}/datasets/{dataset_id}/imports
PUT  /api/v1/projects/{owner_slug}/{project_slug}/imports/{import_id}/content
POST /api/v1/projects/{owner_slug}/{project_slug}/imports/{import_id}/profile
POST /api/v1/projects/{owner_slug}/{project_slug}/imports/{import_id}/preview
POST /api/v1/projects/{owner_slug}/{project_slug}/imports/{import_id}/confirm
GET  /api/v1/projects/{owner_slug}/{project_slug}/imports
GET  /api/v1/projects/{owner_slug}/{project_slug}/imports/{import_id}
POST /api/v1/projects/{owner_slug}/{project_slug}/imports/{import_id}/cancel

Read /datasets/limits first. It publishes upload and dataset bounds plus used and remaining capacity. Chat datasets report examples, messages and text bytes. Embedding datasets report examples and text bytes. The exact dataset ID and kind determine authorization; an unknown or other-project ID never falls back to the default dataset.

Workflow

1. Create an import with optional filename and advisory size_bytes.
2. Upload the file as the response's upload object describes (method, url and headers). When upload.kind is backend, that is a PUT of the file bytes to the import's /content route with your API key.
3. Call /profile for headers, samples, exact byte size, SHA-256 and a suggested mapping.
4. Call /preview with a mapping and invalid_row_policy (fail or skip). The preview shows samples, not a complete validation of the file.
5. Call /confirm with the same mapping and policy, expected_sha256, and a stable idempotency_key. The digest binds confirmation to the profiled file. Confirmation reserves the dataset capacity the rows need and queues the import.
6. Poll the import detail until succeeded, failed or cancelled.

The workflow also uses created, uploaded, profiling, ready_for_mapping, importing and purging states. An unfinished chat import remains invisible to listing and training; an embedding import stages rows and publishes them only after promotion succeeds. Cancellation releases its reservation. Failed or cancelled staged rows are purged in bounded batches.

Embedding imports

An embedding dataset accepts UTF-8 CSV or JSONL (.jsonl/.ndjson, one flat object per line). Supply an explicit mapping:

{"anchor":"query","positive":"passage","negative":"hard_negative","metadata":["language"]}

anchor and positive are required. negative and metadata are optional. Mapped JSONL values must be strings. A duplicate JSON key is refused; a nested or non-string mapped value is an invalid row. Each valid row creates a new pair, reuses an existing pair with the same negative, or is reported as a conflict when the negative differs. Reuse keeps the existing example and records another source-row link. The shared 512 MiB text budget counts stored example text and metadata. Confirm reserves the exact mapped valid-row bytes; promotion settles the reservation against rows actually created.

Errors use {"detail":{"code","message"}}. Capacity failures have typed 409 codes; a changed upload returns 409 import_object_changed. A finished import reports its counts and a bounded sample of error reasons.

The imports list returns {data, next_cursor, total_count}, newest first. Use limit (default 25, maximum 50) and pass next_cursor as cursor for the next page. Optional dataset_id filters both the rows and total. Keep the same dataset filter when following a cursor.