Chat completions

Log probabilities

Return per-token log probabilities to measure model confidence and route low-confidence outputs to a stronger model.

Overview

Log probabilities (logprobs) are the per-token probabilities the model assigns when generating a response. Use them to measure how confident the model is for each token, gate low-confidence outputs, or compare them with alternatives the model considered. Common applications include classification, autocomplete ranking, retrieval evaluation, and content moderation.

Supported models

logprobs is declared on deepseek4-flash, deepseek-v4-pro, glm5.3 and glm5.3-flash, on both streaming and non-streaming requests.

The gateway forwards the parameter unchecked, so other models may return logprobs too, but it isn’t part of their supported contract. Note that reka-flash-3 always reasons before answering and reasoning_effort can’t turn that off, so its first generated tokens are thinking rather than the answer and it isn’t suited to the first-token routing described below.

The feed at GET /v1/models is authoritative. See the model catalog for the current list.

Enable logprobs

Pass logprobs: true on a chat completion request. On reasoning models, set reasoning_effort to none so the first tokens are the answer rather than the chain of thought:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.reka.ai/v1",
    api_key=os.environ["REKA_API_KEY"],
)

completion = client.chat.completions.create(
    model="deepseek4-flash",
    reasoning_effort="none",
    messages=[
        {"role": "user", "content": "What are the top 3 things to do in New York?"}
    ],
    max_tokens=10,
    logprobs=True,
)

print(completion.choices[0].logprobs)

On glm5.3 thinking cannot be fully disabled: none only drops it to the lowest reasoning tier, and the reasoning tokens still appear in logprobs.content ahead of the answer. Skip entries up to and including </think> before reading confidence, or use deepseek4-flash or glm5.3-flash for first-token routing.

The response includes a logprobs object on the choice. Its content field has one entry per generated token, each with the chosen token, its raw bytes, and a logprob:

JSON

{
  "content": [
    {
      "token": "While",
      "bytes": [87, 104, 105, 108, 101],
      "logprob": -0.0104,
      "top_logprobs": []
    },
    // ... one entry per generated token, including the end-of-turn token
  ]
}

content covers every token the model generated, not just the visible text: it includes reasoning tokens when thinking is on and the end-of-turn token (<｜end▁of▁sentence｜>, <|user|> or <|im_end|> depending on the model), so the list is longer than the content you see.

Pass top_logprobs: N (0–20) alongside logprobs: true to receive the N most likely alternatives for each position in each entry’s top_logprobs; without it, top_logprobs is empty.

Logprobs are negative numbers because they’re natural logs of probabilities (which are between 0 and 1). A value closer to 0 means higher confidence; a more negative value means lower confidence.

Convert logprobs to probabilities

To get a probability between 0 and 1, take the exponential of the logprob:

import math


def probability(logprob: float) -> float:
    return math.exp(logprob)


probability(-0.0104)
# 0.9897 → the model was 99% confident in "While" as the first token

For the example above, the first token ("While") has a logprob of -0.0104, which converts to roughly 0.99: the model was about 99% confident in how it opened the answer.

Read the per-token logprob from completion.choices[0].logprobs.content[i].logprob.

Route by confidence

A common pattern is to run a fast, cheap model first, then escalate to a larger model only when the cheap one isn’t confident. Logprobs let you measure that confidence per response.

The example below classifies an email into one of four categories with DeepSeek V4 Flash. If its confidence in the first token falls below a threshold, the request is re-run on DeepSeek V4 Pro:

import math
import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.reka.ai/v1",
    api_key=os.environ["REKA_API_KEY"],
)

EMAIL = (
    "I am writing to request a meeting next week to discuss the "
    "progress of Project X. We have reached several key "
    "milestones, and I believe it would be beneficial to review "
    "our current status and plan next steps together. Could we "
    "schedule a time that works best for you?"
)

messages = [
    {
        "role": "system",
        "content": (
            "You are an email categorizer. Classify the email as one of: "
            "'work', 'personal', 'spam', or 'other'. "
            "Respond with the category name only."
        ),
    },
    {"role": "user", "content": EMAIL},
]

completion = client.chat.completions.create(
    model="deepseek4-flash",
    reasoning_effort="none",
    messages=messages,
    max_tokens=5,
    logprobs=True,
)

label = completion.choices[0].message.content.strip()
first_token = completion.choices[0].logprobs.content[0]
confidence = math.exp(first_token.logprob)

print(f"Label: {label}, confidence: {confidence:.3f}")

if confidence < 0.85:
    # Confidence is low. Re-run on a larger model.
    completion = client.chat.completions.create(
        model="deepseek-v4-pro",
        reasoning_effort="none",
        messages=messages,
        max_tokens=5,
    )
    label = completion.choices[0].message.content.strip()

For this email the model picks work with a first-token logprob of exactly 0.0, which converts to a confidence of 1.0: it was fully certain, well above the threshold.

Output

Label: work, confidence: 1.000

For ambiguous emails the same model often returns confidence in the 0.5 to 0.7 range, which is the signal to escalate.

Keep the label to a single token where you can (short lowercase category names, max_tokens of a few tokens) so the first logprob reflects the whole decision. Reading content[0] only works with thinking off; with reasoning on, the first entries are reasoning tokens. Combine with structured outputs if you need a fixed set of labels, but with a schema the first generated tokens are {"label": ", so read the logprob of the token following the field name rather than content[0].

When not to use logprobs