Serverless Inference

Reference

One endpoint, POST /v1/chat/completions. The request is the OpenAI request; the response adds three fields of our own.

Drop-in

Point your OpenAI SDK at a new base URL. Same calls, no rewrites.

- base_url = "https://api.openai.com/v1"
+ base_url = "https://api.reka.ai/v1"
- api_key  = os.environ["OPENAI_API_KEY"]
+ api_key  = os.environ["REKA_API_KEY"]
  • Chat Completions. the same /v1/chat/completions request and response shape you already send.
  • Streaming. token-by-token server-sent events, with usage totals on the final chunk.
  • Prompt caching. repeated prefixes are cached automatically and billed at the cache-read rate.

Request

modelrequiredstringAn id from the models table. No namespace prefix.
messagesrequiredMessage[]Roles system, user, assistant, tool.
streambooleanServer-sent events; usage totals arrive on the final chunk.
max_tokensintegerCapped by the model's max output length. If omitted, the gateway applies a per-model default budget.
temperaturenumberTypically 0–2; not enforced. Supported on every model.
top_pnumberNucleus sampling. Supported on every model.
top_kintegerSample from the k most likely tokens. deepseek4-flash, deepseek-v4-pro, glm5.3, glm5.3-flash, qwen3.8-27b, reka-edge-2603 and reka-flash-3.
min_pnumberDrop tokens below this fraction of the top token's probability. qwen3.8-27b.
repetition_penaltynumberAbove 1.0 discourages repeating tokens from the prompt or response. deepseek-v4-pro, glm5.3, glm5.3-flash, qwen3.8-27b.
frequency_penaltynumber-2.0 to 2.0. Penalizes tokens by how often they have appeared in the response. deepseek-v4-pro, glm5.3, glm5.3-flash, qwen3.8-27b, reka-edge-2603 and reka-flash-3.
presence_penaltynumber-2.0 to 2.0. Penalizes tokens that have appeared at all in the response. glm5.3, glm5.3-flash, qwen3.8-27b, reka-edge-2603 and reka-flash-3.
logit_biasRecord<string, number>Token id to bias (-100 to 100). qwen3.8-27b.
reasoning_effortstringnone, minimal, low, medium, high, xhigh or max; also accepted as reasoning.effort. Each reasoning model maps it onto its own thinking tiers, and models without one ignore it. No effect on reka-flash-3 or reka-edge-2603. Any other value is a 400 with code invalid_value.
stopstring | string[]A string or list of strings.
toolsTool[]Native tool calling on every model except reka-flash-3.
response_formatobjectjson_object and json_schema on deepseek4-flash, deepseek-v4-pro, glm5.3, glm5.3-flash, and qwen3.8-27b. json_object is not declared for reka-edge-2603 or reka-flash-3; use json_schema there. json_schema.name is required.
logprobsbooleandeepseek4-flash, deepseek-v4-pro, glm5.3, glm5.3-flash.
top_logprobsinteger0–20. With logprobs: true, returns the N most likely alternatives per token; without it, top_logprobs is empty.
seedintegerDeclared on glm5.3, glm5.3-flash, qwen3.8-27b, reka-edge-2603, reka-flash-3. Forwarded to deepseek4-flash and deepseek-v4-pro but not reproducible there.
type Message = {
  role: "system" | "user" | "assistant" | "tool";
  content: string;
};

type ChatRequest = {
  model: string;
  messages: Message[];
  stream?: boolean;
  max_tokens?: number;
  temperature?: number;
  top_p?: number;
  top_k?: number;
  min_p?: number;
  repetition_penalty?: number;
  frequency_penalty?: number;
  presence_penalty?: number;
  logit_bias?: Record<string, number>;
  stop?: string | string[];
  tools?: Tool[];
  response_format?: { type: "json_object" | "json_schema" };
  logprobs?: boolean;
};

Response

The OpenAI response shape throughout, except for three fields of our own.

message.reasoning_contentstring | nullThinking trace on reasoning models; null elsewhere. Streamed as delta.reasoning_content. Some models also send it as message.reasoning, OpenRouter's name for it.
usage.reasoning_tokensintegerInside completion_tokens, not billed twice. Also at usage.completion_tokens_details.reasoning_tokens, where OpenAI's SDKs read it.
metadata.weight_versionstringWhich checkpoint served the request.
{
  "id": "ee405e3d0af64c718248dc2776c3afc7",
  "object": "chat.completion",
  "created": 1787725814,
  "model": "deepseek4-flash",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Air scatters blue light most.",
        "reasoning_content": null,
        "tool_calls": null
      },
      "logprobs": null,
      "finish_reason": "stop",
      "matched_stop": 1
    }
  ],
  "usage": {
    "prompt_tokens": 13,
    "total_tokens": 55,
    "completion_tokens": 42,
    "prompt_tokens_details": null,
    "completion_tokens_details": { "reasoning_tokens": 0 },
    "reasoning_tokens": 0
  },
  "metadata": {
    "weight_version": "default"
  }
}

Rate limits

Limits are per API key and shared across all models. We count requests, not tokens. Need more? Write to contact@reka.ai.

POST /v1/chat/completions120 a minute, burst 30
GET /v1/models300 a minute, burst 300

/v1/messages and /v1/responses share the chat limit. Burst is how many requests you can send back to back; after that, capacity refills at the per-minute rate. Every response, successful or not, carries x-ratelimit-limit-requests (the per-minute rate), x-ratelimit-remaining-requests (what you can send right now, at most the burst) and x-ratelimit-reset-requests.

A 429 carries Retry-After; wait that long before retrying. The error code says which limit you hit:

rate_limit_exceededYour key hit its own limit. Slow down to stay within it.
model_at_capacityThe model is full right now, even if your headers show requests left. Retry after Retry-After.

Errors

Errors come back in the OpenAI error envelope, so existing handling works unchanged.

400Invalid request: a malformed body, a value out of range, or a parameter the model does not support. param names the field when we can tell.
401Missing or revoked key. Check the Authorization header.
404Unknown model id. The ids are unprefixed, so reka/glm5.3 is a 404.
429Your key's limit or a full model; the code says which. Retry after Retry-After. See Rate limits above.
5xxOurs. Retry idempotently and send us the response id.

Next steps