Token Counting

Estimate request size before sending production traffic.

Overview

Token counting helps you predict whether a request fits a model context window, estimate cost before a call, and route large inputs to the right model. OpenAI's current documentation recommends counting the same payload shape you will send to the Responses API, because images, files, tools, schemas, roles, and request formatting can add tokens that local text tokenizers do not see.

Warning

Feature Not Implemented!

This functionality is currently under development and not yet available in AvalAI. We’ll announce its release through our official channels. Stay tuned for updates!

On AvalAI, treat POST /v1/responses/input_tokens as an OpenAI-compatible pattern when it is enabled for your route, model, and account. If the route is not enabled, fall back to local estimation, enforce conservative request-size limits, and reconcile after the request with the response usage object plus AvalAI's User API.

Recommended Workflow

1. Check the model context window with the Models API.
2. Count input tokens before expensive calls when the token-counting route is available.
3. Reject, summarize, chunk, or route oversized inputs before calling the model.
4. Leave headroom for output and reasoning tokens with max_output_tokens or max_completion_tokens.
5. Log actual usage, cached_tokens, model, endpoint, service tier, and x-request-id.

What Local Tokenizers Miss

Local text tokenizers are useful for quick estimates, but they do not fully represent production request shape. Prefer the API count route for:

• multimodal inputs, including input_image, file_id, file_url, and Base64 file_data;
• tool schemas, structured-output schemas, MCP tools, and long system instructions;
• hidden formatting tokens for roles, message boundaries, tool calls, and response channels;
• model-specific behavior such as reasoning, prompt caching, truncation, and conversation state.

Responses API Example

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

payload = {
    "model": "gpt-5.5",
    "instructions": "You are a concise support assistant.",
    "input": [
        {
            "role": "user",
            "content": "Summarize the refund policy in three bullets.",
        }
    ],
}

count = client.responses.input_tokens.count(**payload)
print(f"estimated input tokens: {count.input_tokens}")

if count.input_tokens > 120_000:
    raise ValueError("Input is too large; summarize or chunk it first.")

response = client.responses.create(
    **payload,
    max_output_tokens=500,
)
print(response.output_text)

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const payload = {
  model: "gpt-5.5",
  instructions: "You are a concise support assistant.",
  input: [
    {
      role: "user",
      content: "Summarize the refund policy in three bullets.",
    },
  ],
};

const count = await client.responses.input_tokens.count(payload);
console.log(`estimated input tokens: ${count.input_tokens}`);

if (count.input_tokens > 120000) {
  throw new Error("Input is too large; summarize or chunk it first.");
}

const response = await client.responses.create({
  ...payload,
  max_output_tokens: 500,
});

console.log(response.output_text);

cURL Compatibility Check

Use this quick check before depending on the route in production. If AvalAI returns an unsupported-route error for your account or model, keep the fallback path described above.

curl https://api.avalai.ir/v1/responses/input_tokens \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-5.5",
    "input": "Tell me a joke."
  }'

CLI Preflight for Scripts

OpenAI's generated CLI can count input tokens with the same Responses payload shape. If your installed CLI supports a custom base URL, point it at AvalAI and use --transform input_tokens so shell scripts receive only the integer count:

OPENAI_API_KEY="$AVALAI_API_KEY" \
  OPENAI_BASE_URL="https://api.avalai.ir/v1" \
  openai responses:input-tokens count \
  --raw-output \
  --transform input_tokens <<'YAML'
model: gpt-5.5
instructions: You are a concise support assistant.
input:
  - role: user
    content: Summarize the refund policy in three bullets.
YAML

Use this for CI gates, batch import scripts, and preflight checks before uploading large files or launching expensive background jobs. If your CLI version does not honor OPENAI_BASE_URL, use the cURL compatibility check instead so the AvalAI endpoint is explicit.

What to Count

• Messages and instructions: roles, boundaries, and formatting can add tokens beyond visible text.
• Images and files: avoid characters / 4 estimates for multimodal inputs; use the route when available.
• Tools and schemas: function definitions and structured-output schemas can become a large fixed prefix.
• Conversation state: include the same history, previous_response_id strategy, or reconstructed messages you will send.

Chat Completions Preflight

OpenAI's token-counting endpoint accepts the Responses request shape. For existing /v1/chat/completions applications, build a preflight payload that mirrors the same content before you call Chat Completions:

Chat Completions field	Token-counting preflight shape
messages	input array with the same role and content items
first system/developer message	instructions, or keep as an item in input when order matters
tools / function schemas	same tools array when the selected Responses-compatible route supports it
max_completion_tokens	reserve output headroom with max_output_tokens in the final Responses equivalent, but keep max_completion_tokens on the actual Chat request

Use the count as a conservative planning signal, not as the final bill. After the Chat call, reconcile with usage.prompt_tokens, usage.completion_tokens, usage.prompt_tokens_details.cached_tokens, and AvalAI's User API transaction record. Keep direct audio-in/audio-out Chat examples on /v1/chat/completions; count the closest text/tool payload you can and validate the real route in staging.

Count Multimodal and Tool Payloads

Use the same request body you plan to send to responses.create. This example counts image input plus a function tool schema, which are both easy to undercount locally.

Python

count = client.responses.input_tokens.count(
    model="gpt-5.5",
    tools=[
        {
            "type": "function",
            "name": "lookup_order",
            "description": "Fetch a customer order by ID.",
            "parameters": {
                "type": "object",
                "properties": {"order_id": {"type": "string"}},
                "required": ["order_id"],
                "additionalProperties": False,
            },
        }
    ],
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_image", "image_url": "https://example.com/receipt.png"},
                {
                    "type": "input_text",
                    "text": "Extract the order ID and summarize the receipt.",
                },
            ],
        }
    ],
)

print(count.input_tokens)

JavaScript

const count = await client.responses.input_tokens.count({
  model: "gpt-5.5",
  tools: [
    {
      type: "function",
      name: "lookup_order",
      description: "Fetch a customer order by ID.",
      parameters: {
        type: "object",
        properties: { order_id: { type: "string" } },
        required: ["order_id"],
        additionalProperties: false,
      },
    },
  ],
  input: [
    {
      role: "user",
      content: [
        { type: "input_image", image_url: "https://example.com/receipt.png" },
        { type: "input_text", text: "Extract the order ID and summarize the receipt." },
      ],
    },
  ],
});

console.log(count.input_tokens);

For private files, upload with the Files API or use Base64 input according to the target route; do not expose private documents through public URLs just to count tokens.

For file inputs, count the exact representation you will send to the model:

File input shape	When to count it
file_id	Reused private files uploaded through /v1/files with purpose="user_data".
file_url	Public or temporary HTTPS files passed directly to Responses.
file_data	Local files encoded as Base64 data URLs.

PDFs can include extracted text plus page images in the model context when the selected route and model support vision-capable PDF parsing. Non-PDF documents are usually text-extracted, and spreadsheet-like files may be summarized or augmented instead of counted as raw cells. Count the same input_file payload you plan to send, then test the final route because provider behavior and account limits can differ on AvalAI.

Output Token Headroom

Reported output usage can include visible text, reasoning tokens, tool-call formatting, and other non-visible tokens. Across providers and models, hidden reasoning tokens are billed at the selected model's output-token rate. When usage.output_tokens already includes them, usage.output_tokens_details.reasoning_tokens is a breakdown, not an additional amount to add. If a route reports visible output and reasoning separately, apply the same output-token rate to both counts.

Do not set max_output_tokens or max_completion_tokens equal to the exact number of words you expect. These parameters provide a shared generation budget, not a reserved visible-answer budget. A reasoning model can consume the limit internally and return no text; for Responses, look for status: "incomplete" with incomplete_details.reason: "max_output_tokens", and for Chat Completions check finish_reason: "length". Leave headroom, then monitor usage.output_tokens, usage.output_tokens_details.reasoning_tokens, and final answer length in production. If exhaustion occurs, increase the limit, lower reasoning effort when supported, or simplify the task. See Reasoning token budgets.

Fallback Estimation

When /v1/responses/input_tokens is unavailable on a route:

• estimate plain text with a local tokenizer only as a lower bound;
• reserve extra budget for roles, tools, schemas, images, files, and reasoning;
• enforce conservative request-size limits before the API call;
• reconcile after the call with usage.input_tokens, usage.output_tokens, cached_tokens, and AvalAI billing records.

Related Resources

• Production Best Practices
• Latency Optimization
• Prompt Caching
• Cost Optimization
• Responses vs Chat Completions
• Models API
• Adapted for AvalAI from OpenAI's Counting tokens guide.