OpenAI-compatible inference

ReOpenly exposes a project-scoped OpenAI-compatible API. The available model can be a catalog base, a finished training job, or an active hosted alias. Inference is dispatched through the control plane to an isolated worker or a managed serving unit; there is no prototype-only simulated generation path.

Endpoints

GET  /api/projects/{owner_slug}/{project_slug}/openai/v1/models
POST /api/projects/{owner_slug}/{project_slug}/openai/v1/chat/completions
POST /api/projects/{owner_slug}/{project_slug}/openai/v1/completions

The first endpoint returns only models the bearer can invoke:

{
  "object": "list",
  "data": [
    {
      "id": "support-assistant",
      "object": "model",
      "created": 1764547200,
      "owned_by": "reopenly"
    }
  ]
}

Use the returned id for a hosted alias. A catalog model is addressed as base#<catalog-model-id>, for example base#unsloth/Qwen3.5-9B. A finished job can be addressed as training-job#<job-id> when the caller has access to that project.

Chat completions

curl "${REOPENLY_OPENAI_BASE}/chat/completions" \
  -H "Authorization: Bearer ${REOPENLY_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "base#unsloth/Qwen3.5-9B",
    "messages": [
      {"role": "user", "content": "Give me one sentence about datasets."}
    ],
    "max_tokens": 64,
    "temperature": 0.2,
    "stream": false
  }'

The request supports model, messages, max_tokens, temperature, top_p, reasoning_effort, function tools, stream, and stream_options.include_usage. The response is an OpenAI chat completion with choices, a finish reason, and usage fields:

{
  "id": "chatcmpl-…",
  "object": "chat.completion",
  "created": 1764547200,
  "model": "base#unsloth/Qwen3.5-9B",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "…"},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 18,
    "completion_tokens": 12,
    "total_tokens": 30,
    "reasoning_tokens": 0,
    "cached_tokens": 0
  }
}

Token counts come from the serving path when available. A missing upstream count is represented as zero; clients must not treat it as a client-side estimate.

Defaults for omitted fields

Omit an optional field and the server applies:

Field	Applied when omitted
max_tokens	1024
temperature	1.0
top_p	1.0
stream	false
reasoning_effort	nothing — the serving model's own default

GET /api/base-models publishes the same table as an inference block (defaults and capabilities), so a client can read it rather than hardcode it. The first-party playground initializes from that block, which is what makes a prompt typed there and the same prompt sent through this API the same request.

reasoning_effort deserves care: none is an explicit instruction to disable reasoning, while omitting the field asks the model for whatever it does by default. They are different requests. The explicit values are none, low, and high; the playground offers a fourth choice, "model default", which sends no field at all.

Streaming

Set stream: true to receive text/event-stream frames. Successful streams end with a finish chunk followed by:

data: [DONE]

Request the standard final usage chunk when needed:

{
  "model": "support-assistant",
  "messages": [{"role": "user", "content": "Summarize this."}],
  "stream": true,
  "stream_options": {"include_usage": true}
}

The usage frame has an empty choices array and a usage object. Do not assume every text frame carries a role or content: tool-call deltas and the finish frame use different shapes.

If the request fails before the HTTP stream starts, the server returns the normal JSON error response. If the worker fails after 200 OK and SSE headers have been sent, the final frame is a JSON error envelope and the stream ends without [DONE]. See Errors and retries.

Legacy completions

POST /completions accepts a string or list of strings in prompt and returns an OpenAI text_completion response. It supports model, prompt, max_tokens, temperature, top_p, and reasoning_effort. The current contract is non-streaming for this endpoint.

Context limits

There are two checks:

1. The catalog check uses max_context_tokens from GET /api/base-models. Production currently advertises 4096 for Qwen3.5 9B.
2. A vLLM serving unit performs an exact pre-admission check against its own max_model_len.

The catalog check is conservative and happens before dispatch. A request can therefore fail with invalid_request even when a worker appears to have free GPU memory. Fetch the catalog for the deployment and leave room for max_tokens; never hardcode a limit from this page.

Billing and retries

Requests are charged according to the serving path only after dispatch work is known to have run. Every successful request on a positive-rate route has a minimum billable duration of one second at that request's frozen rate (0.0005 credits when the rate is 0.03 credits per GPU minute). Explicitly free routes remain free, and failed requests are not charged. A retry is safe only when the response carries a retryable model_warming error and Retry-After. An ambiguous_transport response may mean the request was served; retrying blindly can duplicate work and billing. The first-party v2 playground preserves the original conversation, performs at most three automatic retries after the original request, and leaves a manual Retry action when that bound is exhausted. It never retries a mid-stream error.

The global playground stores its project scope in the page URL: /app/playground?project=owner/project. It still calls the same project-scoped OpenAI endpoint shown above; there is no unscoped inference route.