Responses WebSocket Mode

Use Responses WebSocket mode for long-running, tool-heavy workflows that benefit from a persistent connection and incremental inputs.

Warning

Feature Not Implemented!

This functionality is currently under development and not yet available in AvalAI. We’ll announce its release through our official channels. Stay tuned for updates!

Adapted from OpenAI's official WebSocket Mode, conversation state, and compaction documentation, with AvalAI endpoint, API key, route-availability, and fallback notes.

Availability in AvalAI

OpenAI documents WebSocket mode as a persistent transport for /v1/responses. In AvalAI, treat it as route-, model-, and account-dependent. Confirm support in staging before relying on wss://api.avalai.ir/v1/responses; if your route uses a different WebSocket URL, set it as AVALAI_RESPONSES_WS_URL.

If WebSocket mode is unavailable, keep the same Responses request shape over HTTP:

• use POST /v1/responses with previous_response_id when hosted state is supported and retention is acceptable;
• use manual item replay when you need stateless behavior or store: false;
• use stream: true over SSE for token-by-token UI updates.

When to Use It

Use WebSocket mode when a workflow has many model/tool round trips:

• agentic coding loops with repeated tool outputs;
• orchestration workers that keep one task active for many turns;
• low-latency tool chains where reconnecting for every turn adds overhead;
• stateful workflows that already use previous_response_id.

Avoid it for simple one-shot prompts, browser clients that cannot safely hold API keys, or workloads that need many parallel responses on one socket. One WebSocket connection should own one in-flight response at a time.

Transport Model

Topic	Behavior
Connection	Open a persistent WebSocket to the Responses route.
Create turn	Send a response.create JSON event. The payload mirrors POST /v1/responses; transport-only fields such as stream and background are not used.
Continue turn	Send another response.create with previous_response_id and only the new input items.
Cache	The active connection can keep the most recent previous response in memory for fast continuation.
Concurrency	Responses run sequentially; use separate sockets for parallel workflows.
Lifetime	Plan to reconnect before or when the connection reaches the documented 60-minute limit.

Connect and Create a Response

Install a WebSocket client first:

pip install websocket-client
npm install ws

Python

import json
import os
from websocket import create_connection

ws = create_connection(
    os.getenv("AVALAI_RESPONSES_WS_URL", "wss://api.avalai.ir/v1/responses"),
    header=[f"Authorization: Bearer {os.environ['AVALAI_API_KEY']}"],
)

ws.send(
    json.dumps(
        {
            "type": "response.create",
            "model": "gpt-5.5",
            "store": False,
            "input": [
                {
                    "type": "message",
                    "role": "user",
                    "content": [
                        {
                            "type": "input_text",
                            "text": "Find the bottleneck in this worker.",
                        }
                    ],
                }
            ],
            "tools": [],
        }
    )
)

while True:
    event = json.loads(ws.recv())
    if event["type"] == "response.output_text.delta":
        print(event["delta"], end="", flush=True)
    elif event["type"] == "response.completed":
        response_id = event["response"]["id"]
        print(f"\ncompleted: {response_id}")
        break
    elif event["type"] in {"response.failed", "error"}:
        raise RuntimeError(event)

JavaScript

import WebSocket from "ws";

const ws = new WebSocket(
  process.env.AVALAI_RESPONSES_WS_URL ?? "wss://api.avalai.ir/v1/responses",
  {
    headers: {
      Authorization: `Bearer ${process.env.AVALAI_API_KEY}`,
    },
  },
);

ws.on("open", () => {
  ws.send(
    JSON.stringify({
      type: "response.create",
      model: "gpt-5.5",
      store: false,
      input: [
        {
          type: "message",
          role: "user",
          content: [
            { type: "input_text", text: "Find the bottleneck in this worker." },
          ],
        },
      ],
      tools: [],
    }),
  );
});

ws.on("message", (data) => {
  const event = JSON.parse(data.toString());
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  } else if (event.type === "response.completed") {
    console.log(`\ncompleted: ${event.response.id}`);
    ws.close();
  } else if (event.type === "response.failed" || event.type === "error") {
    throw new Error(JSON.stringify(event));
  }
});

Continue with Incremental Inputs

After the first response completes, keep the socket open and send only new input items plus the latest previous_response_id.

Python

ws.send(
    json.dumps(
        {
            "type": "response.create",
            "model": "gpt-5.5",
            "store": False,
            "previous_response_id": response_id,
            "input": [
                {
                    "type": "function_call_output",
                    "call_id": "call_123",
                    "output": "The worker spends 70% of time waiting on Redis.",
                },
                {
                    "type": "message",
                    "role": "user",
                    "content": [
                        {
                            "type": "input_text",
                            "text": "Suggest the safest optimization.",
                        }
                    ],
                },
            ],
            "tools": [],
        }
    )
)

JavaScript

ws.send(
  JSON.stringify({
    type: "response.create",
    model: "gpt-5.5",
    store: false,
    previous_response_id: responseId,
    input: [
      {
        type: "function_call_output",
        call_id: "call_123",
        output: "The worker spends 70% of time waiting on Redis.",
      },
      {
        type: "message",
        role: "user",
        content: [
          { type: "input_text", text: "Suggest the safest optimization." },
        ],
      },
    ],
    tools: [],
  }),
);

Resend important instructions on every turn. previous_response_id carries response context where supported, but it does not make top-level instructions automatically permanent.

State, Retention, and Recovery

Design WebSocket loops with an explicit fallback:

Situation	Recommended action
store: true and the previous response is persisted	Reconnect and continue with previous_response_id plus new input items.
store: false, ZDR-style flow, or uncached ID	Start a new chain with previous_response_id: null and send full context or a compacted window.
previous_response_not_found	Retry as a fresh response with the full input context; do not assume the server can hydrate the chain.
websocket_connection_limit_reached	Open a new WebSocket and continue from the latest durable state.
Failed continuation (4xx or 5xx)	Rebuild state from your application log before retrying.

Even when previous_response_id reduces payload plumbing, budget chained requests as if the relevant prior context can still count as input tokens.

Compaction Patterns

For long-running agents, combine WebSocket continuation with Context Compaction:

• Server-side compaction: if the route supports context_management with compact_threshold, continue normally on the socket with the latest previous_response_id and only new input items.
• Standalone /v1/responses/compact: call the compact endpoint over HTTP when available, then start a new WebSocket response using the returned compacted window as input; omit previous_response_id or set it to null.
• Portable fallback: summarize state in your app with /v1/responses, store that summary under your retention policy, and send it with the next turn.

Do not edit encrypted or opaque compaction items. Treat returned compacted windows as machine state for the next request.

Migration Path from HTTP Responses

1. Build the workflow with normal POST /v1/responses.
2. Add previous_response_id or manual item replay until state handling is correct.
3. Add stream: true if the UI needs incremental text over HTTP.
4. Move only eligible long-running, tool-heavy workers to WebSocket mode after staging confirms support.
5. Keep the HTTP path as the recovery path for reconnects, unsupported routes, and compliance-sensitive workflows.

Production Checklist

• Confirm the exact WebSocket URL, model support, and account entitlement in staging.
• Keep API keys on trusted servers; never expose server API keys in browsers.
• Store your own task ID, response ID, request ID, user/tenant context, and usage.
• Enforce one in-flight response per socket; use a connection pool for parallel work.
• Reconnect before 60 minutes and recover from full context, compacted context, or a stored response ID.
• Handle previous_response_not_found, connection close, timeout, 429, and provider-specific 4xx/5xx errors.

Related Guides

• Responses API Reference
• Conversation State
• Streaming Responses
• Context Compaction
• Error Handling