Skip to main content

Quickstart for embeddings

Train an embedding model on your own query and passage pairs, host it, and call it through the OpenAI embeddings API. Every request below uses an API key:

export REOPENLY_API_KEY="<your key>"
export API="https://reopenly.com/api/v1"

1. Create an embedding project​

curl "$API/projects" \
  -H "Authorization: Bearer $REOPENLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "Citations", "type": "embedding"}'

The response carries the project's owner_slug and slug; together they are the project's URL. The type is chosen once and never changes.

export P="$API/projects/<owner_slug>/<slug>"

2. Add training pairs​

A new embedding project has a default dataset. Find its id:

curl "$P/datasets" -H "Authorization: Bearer $REOPENLY_API_KEY"

Each example is a query (anchor), a passage that answers it (positive), and optionally a passage that looks related but does not (negative).

Add enough independent examples for both training and validation. This example creates three triplets with distinct texts; one row cannot produce a valid split. Examples sharing text stay in the same split to prevent leakage.

for example in \
  '{"anchor":"How long is the warranty on the X200?","positive":"The X200 is covered by a two-year limited warranty.","negative":"The X100 ships with a one-year warranty."}' \
  '{"anchor":"How long do I have to return an order?","positive":"You can return an order within thirty days of delivery.","negative":"Standard delivery takes five business days."}' \
  '{"anchor":"When is shipping free?","positive":"Orders over fifty dollars qualify for free shipping.","negative":"Express shipping arrives on the next business day."}'
do
  curl "$P/datasets/<dataset_id>/examples" \
    -H "Authorization: Bearer $REOPENLY_API_KEY" \
    -H "Content-Type: application/json" \
    -d "$example"
done

For more than a handful, import a CSV or JSONL file instead: see Dataset imports.

3. Preview and estimate the run​

Pick a base model and read the bounds of every setting:

curl "$P/training-jobs/profiles" -H "Authorization: Bearer $REOPENLY_API_KEY"

Preview the run with the dataset, the base's base_artifact_id and a number of steps. The preview returns the exact split, planned_steps_per_epoch and a selection_sha256; set max_steps to a whole number of epochs (planned_steps_per_epoch × epochs):

curl "$P/training-jobs/preview" \
  -H "Authorization: Bearer $REOPENLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dataset_id": <dataset_id>, "base_artifact_id": <base_artifact_id>, "max_steps": <steps>}'

POST $P/training-jobs/estimate with the same body returns admission_ceiling_credits, the credits that must be available to start.

4. Train​

Start the run with the previewed selection_sha256 and a unique Idempotency-Key:

curl "$P/training-jobs" \
  -H "Authorization: Bearer $REOPENLY_API_KEY" \
  -H "Idempotency-Key: <unique-uuid>" \
  -H "Content-Type: application/json" \
  -d '{
    "dataset_id": <dataset_id>,
    "base_artifact_id": <base_artifact_id>,
    "max_steps": <steps>,
    "selection_sha256": "<selection_sha256>"
  }'

Follow it with GET $P/training-jobs/<job_id> until model_available is true. Only the training loop is billed. The job page shows training validation and the kept checkpoint; see Training jobs.

5. Host the model​

Give the trained model a stable name:

curl "$P/models" \
  -H "Authorization: Bearer $REOPENLY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Citation encoder",
    "slug": "citation-encoder",
    "task": "embedding",
    "source": {"kind": "training_job", "training_job_id": <job_id>}
  }'

6. Embed​

Call it with any OpenAI client at the project's base URL:

from openai import OpenAI

client = OpenAI(
    base_url="https://reopenly.com/api/projects/<owner_slug>/<slug>/openai/v1",
    api_key="<your key>",
)
response = client.embeddings.create(
    model="citation-encoder",
    input=["How long is the warranty on the X200?"],
    extra_body={"input_type": "query"},
)
print(len(response.data[0].embedding))

Send input_type: "query" for queries and "document" for passages. To try the model on a whole dataset first, build a Search index: see Playground and Search. To keep improving it from real searches, see Search feedback.