Reka Gateway

Serverless Inference

How serverless inference works on Reka: billing, what to read on each response, prompt caching, model lifecycle, and when to choose serverless over an on-demand deployment.

What is serverless inference

Serverless inference is multi-tenant inference for Reka’s own models and popular open models, running on Reka-managed infrastructure. You point an OpenAI-compatible client at https://api.reka.ai/v1, send tokens, and pay only for what you use. There are no GPUs to size, no autoscaler to tune, and no cold starts to wait through.

Every model on the Models page is available serverlessly, and GET /v1/models returns the list that’s live right now. To make your first call, see the Quickstart.

Serverless at a glance

There is one way to call a serverless model, and it’s the same for every model in the catalog:

One endpoint. POST /v1/chat/completions, with the OpenAI request and response shape. Existing OpenAI SDK code works after changing the base URL and key. See Drop-in.
Plain model ids. Models are addressed by their unprefixed id, for example deepseek4-flash. There are no service tiers or speed variants to choose between; each request is routed the same way and billed at the model’s listed rate.
Streaming, tools, structured outputs, reasoning. Supported through the standard OpenAI parameters. The Parameters page lists which models declare which features.
Also on OpenRouter. The same models are listed under the Reka provider on OpenRouter if you’d rather route through it.

Billing

Serverless is priced per token. Three dimensions are billed:

Input tokens: what you send to the model, at the model’s input rate.
Cached input tokens: input tokens served from the prompt cache, at the model’s discounted cached input rate. The discount is per model rather than a fixed percentage.
Output tokens: what the model produces, at the model’s output rate. On reasoning models, the thinking trace is part of the output and is counted inside completion_tokens, never billed twice.

For example, deepseek4-flash bills $0.11 per million input tokens, $0.007 per million cached input tokens, and $0.66 per million output tokens. Current rates for every model are on the Pricing page.

Other things to know:

The usage object on each response is the source of truth for what was billed. See Request and response.
Requests spend your organisation’s prepaid credit balance, shared across all of its API keys. Top up, break spend down by model and find invoices on Usage & billing.
There is no platform fee or markup on top of the per-token rates, and no charge for writing to the prompt cache.

Request and response

What a serverless caller sets and reads.

Request headers

Header	Notes
`Authorization: Bearer $REKA_API_KEY`	Required for all requests. Create keys on the API keys page.
`Content-Type: application/json`	Required. The OpenAI SDKs set this for you.

There are no routing or affinity headers to set. Prompt caching works without one; see Prompt caching.

Response usage

Per-request metrics are returned in the response body rather than in headers, so they’re identical for streaming and non-streaming calls. On a stream, the final chunk carries them.

JSON

"usage": {
  "prompt_tokens": 3417,
  "completion_tokens": 212,
  "total_tokens": 3629,
  "reasoning_tokens": 148,
  "prompt_tokens_details": { "cached_tokens": 3328 },
  "completion_tokens_details": { "reasoning_tokens": 148 }
}

Field	What it tells you
`prompt_tokens`	Input tokens for the request, cached and uncached together.
`prompt_tokens_details.cached_tokens`	The cached portion of the input, billed at the cached input rate.
`completion_tokens`	Output tokens, including any reasoning trace.
`reasoning_tokens`	How much of the output was thinking. Also at `completion_tokens_details.reasoning_tokens`, where OpenAI’s SDKs read it.
`id`	The response id, on the top-level object. Include it when you report a problem.

Rate and capacity limits come back as 429 in the OpenAI error envelope, with a body that says which was hit. Retry with backoff. The full list is under Errors.

Prompt caching

Prompt caching is on by default for every serverless model that lists a cached input rate. When a request starts with the same tokens as an earlier request, the shared prefix is read from the cache and billed at the discounted rate, and only the tokens after it are processed from scratch. There’s no parameter, header, or account setting to turn it on.

To maximize hits, put stable content such as instructions, tool definitions, and reference material first, and per-request content last. Append to conversations rather than rewriting earlier turns. The cached count comes back in usage.prompt_tokens_details.cached_tokens.

For the full guide, including how to structure prompts for cache hits and the per-model cached rates, see Prompt caching.

Serverless model lifecycle

Serverless models are managed by the Reka team and are added, updated, and retired as new models are released. Additions and changes are posted on the Changelog, and GET /v1/models always reflects what is live.

Reference models by their exact id in your configuration rather than hard-coding them throughout your code, so an upgrade is a one-line change. For production workloads that need a fixed model version for the long term, an on-demand deployment gives you full control over which model runs and when it changes.

Serverless vs on-demand

When serverless fits	When on-demand fits
Pay per token, only for what you use.	Pay for dedicated GPU time, regardless of traffic.
You’re using models Reka already hosts.	You’re running a custom or fine-tuned model that isn’t in the catalog.
You don’t want to manage scaling, replicas, or hardware sizing.	You have specific latency or throughput requirements and want control over hardware and replicas.
You’re fine with the model catalog changing over time.	You need a pinned model version for the long term.

For dedicated infrastructure, see On-demand deployments.