Background Processing

Long-running reasoning, deep research, code analysis, and report generation can exceed browser, proxy, or mobile client timeouts. OpenAI's background mode pattern starts a Responses task asynchronously, lets clients poll the Response object, and can optionally stream from a durable background run. In AvalAI, treat hosted background Responses as route-, model-, and account-dependent; use the app-managed fallback below when background: true or response cancellation is not available.

Warning

Feature Not Implemented!

This functionality is currently under development and not yet available in AvalAI. We’ll announce its release through our official channels. Stay tuned for updates!

When to Use It

Use background processing for:

• responses that may take minutes, especially high-effort reasoning or deep research;
• user workflows where closing the tab should not lose the job;
• queueable tasks that need progress UI, retries, or cancellation;
• operations where the client can poll by ID instead of holding one HTTP connection open.

Keep normal synchronous or streaming Responses for short chat turns where immediate latency matters.

Hosted Background Responses

When your selected AvalAI route supports OpenAI-style background mode, create the Response with background: true and keep store: true. Background mode depends on stored response state so clients can retrieve it later.

Before using hosted mode in production, run a small compatibility probe for the exact model and route. OpenAI's reference behavior accepts background: true only for stored Responses, lets clients retrieve or cancel by Response ID, and treats queued / in_progress as non-terminal states. AvalAI exposes this pattern where the upstream route supports it; otherwise use the app-managed fallback.

Python

import os
import time
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    input="Produce a detailed migration plan for this monorepo.",
    background=True,
    store=True,
)

while response.status in {"queued", "in_progress"}:
    print(f"Current status: {response.status}")
    time.sleep(2)
    response = client.responses.retrieve(response.id)

print(response.status)
print(response.output_text)

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

let response = await client.responses.create({
  model: "gpt-5.5",
  input: "Produce a detailed migration plan for this monorepo.",
  background: true,
  store: true,
});

while (response.status === "queued" || response.status === "in_progress") {
  console.log(`Current status: ${response.status}`);
  await new Promise((resolve) => setTimeout(resolve, 2000));
  response = await client.responses.retrieve(response.id);
}

console.log(response.status);
console.log(response.output_text);

Bash

curl https://api.avalai.ir/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
    "model": "gpt-5.5",
    "input": "Produce a detailed migration plan for this monorepo.",
    "background": true,
    "store": true
  }'

If the route rejects background, remove that parameter and use your own worker queue. Do not silently retry a multi-minute user request synchronously unless the user experience can tolerate the timeout risk.

Compatibility Probe Checklist

Before promising hosted background behavior to users, verify the exact AvalAI route, model, SDK version, and account tier:

Probe	Expected result
Create with background: true and store: true	Returns a Response ID and a status such as queued or in_progress.
Retrieve by Response ID	Returns the same Response until it reaches a terminal state.
Poll through completion	Leaves queued / in_progress and ends as completed, failed, cancelled, incomplete, or expired.
Cancel while in progress	Returns a terminal cancelled state or the already-final Response; repeated cancel calls are safe.
Stream with background: true and stream: true	Emits typed Responses events with sequence_number values you can persist.
Resume stream with starting_after	Continues from the last cursor when the route supports stream resume; otherwise fall back to your local job stream.

Record the probe results in your deployment notes. If one probe fails, document the app-managed fallback for that capability instead of implying hosted support.

Polling Safely

When polling a hosted background Response, keep the poller boring and bounded:

• poll by Response ID, not by repeating the original prompt;
• use exponential backoff with jitter instead of a fixed tight loop for large fleets;
• stop polling on every terminal state, not only completed;
• store the final Response ID, status, x-request-id, model, and usage in your own job record;
• copy final output before the hosted retention window expires;
• surface a timeout state in your UI if polling exceeds your product SLA.

If your AvalAI route supports webhooks for this workflow, prefer a webhook to mark the job terminal and keep polling as a user-facing fallback.

Terminal Status Handling

Only parse model output after the Response leaves queued and in_progress. Treat every terminal state explicitly so your UI, retry logic, and billing records stay consistent:

Status	What to do
completed	Read output_text, usage, citations, and any tool outputs; then mark your local job complete.
failed	Store a safe error message, request ID, and retry eligibility; avoid retrying non-idempotent work automatically.
cancelled	Show that the user or system stopped the job; ignore late worker output from an in-flight fallback call.
incomplete or expired	Treat as not final output; retry with a smaller task, more explicit instructions, or your app-managed queue.

For hosted background mode, keep store: true; upstream OpenAI rejects stateless background sampling. If AvalAI or the selected provider does not expose stored background state for that route, fall back to your own job table and call Responses with store: false.

App-Managed Fallback

The portable AvalAI pattern is to store a local job record, run the model call in a worker, and expose a polling endpoint from your app.

Python

import os
import uuid
from dataclasses import dataclass
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)


@dataclass
class Job:
    id: str
    status: str = "queued"
    output_text: str | None = None
    error: str | None = None
    request_id: str | None = None


jobs: dict[str, Job] = {}


def create_job(prompt: str) -> Job:
    job = Job(id=f"job_{uuid.uuid4().hex}")
    jobs[job.id] = job

    try:
        job.status = "in_progress"
        response = client.responses.create(
            model="gpt-5.5",
            input=prompt,
            store=False,
        )
        job.request_id = response.id
        job.output_text = response.output_text
        job.status = "completed"
    except Exception as exc:
        job.error = str(exc)
        job.status = "failed"

    return job


job = create_job("Draft a security review checklist for this API.")
print(job.id, job.status)

JavaScript

import OpenAI from "openai";
import crypto from "node:crypto";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const jobs = new Map();

async function createJob(prompt) {
  const job = {
    id: `job_${crypto.randomUUID().replaceAll("-", "")}`,
    status: "queued",
    outputText: null,
    error: null,
    requestId: null,
  };
  jobs.set(job.id, job);

  try {
    job.status = "in_progress";
    const response = await client.responses.create({
      model: "gpt-5.5",
      input: prompt,
      store: false,
    });
    job.requestId = response.id;
    job.outputText = response.output_text;
    job.status = "completed";
  } catch (error) {
    job.error = String(error);
    job.status = "failed";
  }

  return job;
}

const job = await createJob("Draft a security review checklist for this API.");
console.log(job.id, job.status);

In production, move create_job into a real queue worker such as Celery, BullMQ, Sidekiq, SQS, or your internal job system. The API request should only create the job and return its ID.

Streaming and Reconnects

If hosted background streaming is available, create the response with both background: true and stream: true, then persist each event sequence_number so the client can reconnect from the last cursor. If this is not available, stream progress from your own job table instead:

• queued: the request is accepted and waiting for a worker;
• in_progress: the model call has started;
• completed: final answer stored and ready to fetch;
• failed: store the error class and safe message;
• cancelled: user cancelled before completion.

Avoid inventing partial model output if the upstream provider did not return it. Progress messages should describe job state, not fabricated answer content.

Hosted background streaming uses typed Responses events. Store the latest sequence_number from each event; if the connection drops and the route supports resume, reconnect with starting_after so the UI can continue from the last seen event.

Python

stream = client.responses.create(
    model="gpt-5.5",
    input="Write a long market research report with citations.",
    background=True,
    stream=True,
    store=True,
)

last_sequence_number = None
for event in stream:
    print(event.type)
    last_sequence_number = getattr(event, "sequence_number", last_sequence_number)

# If the stream drops, persist last_sequence_number with your local job.
# Reconnect support is route/SDK dependent; use your app-managed job stream if unavailable.

JavaScript

const stream = await client.responses.create({
  model: "gpt-5.5",
  input: "Write a long market research report with citations.",
  background: true,
  stream: true,
  store: true,
});

let lastSequenceNumber = null;
for await (const event of stream) {
  console.log(event.type);
  lastSequenceNumber = event.sequence_number ?? lastSequenceNumber;
}

// If the stream drops, persist lastSequenceNumber with your local job.
// Reconnect support is route/SDK dependent; use your app-managed job stream if unavailable.

Bash

# Create a background stream.
curl https://api.avalai.ir/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
    "model": "gpt-5.5",
    "input": "Write a long market research report with citations.",
    "background": true,
    "stream": true,
    "store": true
  }'

# If the route supports stream resume, reconnect from the last event cursor.
curl "https://api.avalai.ir/v1/responses/resp_123?stream=true&starting_after=42" \
  -H "Authorization: Bearer $AVALAI_API_KEY"

Cancellation and Idempotency

Hosted cancellation, when supported, cancels an in-flight background Response by ID. For the app-managed fallback, cancellation should set your local job state to cancelled, stop queued work before it starts, and ignore late worker output if the model call was already in flight.

Python

cancelled = client.responses.cancel("resp_123")
print(cancelled.status)

JavaScript

const cancelled = await client.responses.cancel("resp_123");
console.log(cancelled.status);

Bash

curl -X POST https://api.avalai.ir/v1/responses/resp_123/cancel \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY"

Cancellation should be idempotent: if the hosted route already reached a terminal state, a repeated cancel request should return the final Response object or an equivalent terminal record. To cancel a synchronous non-background call, close the client connection and design the action so a later retry is safe.

Use an idempotency key or deterministic job key for user actions that may be retried. Store model, prompt version, user ID, and created time so duplicate clicks do not launch duplicate expensive jobs.

Operational Limits

The OpenAI reference guide calls out a few limits that are useful when adapting the pattern to AvalAI:

• Background sampling requires stored response state; stateless background calls are rejected upstream.
• A background stream can only be resumed if the Response was originally created with stream: true.
• Time to first token can be higher for background streams than for synchronous streams.
• Polling data is retained briefly, so copy final results into your own job record before the hosted retention window expires.

Data Retention Notes

OpenAI's reference background mode stores response data for roughly 10 minutes to enable polling, so it is not compatible with strict zero-data-retention expectations. OpenAI notes that background=true may still be accepted for legacy ZDR projects, but using it breaks ZDR guarantees; Modified Abuse Monitoring (MAM) projects can rely on background mode when their account and route support it. Through AvalAI, treat this as provider-, account-, and route-dependent and verify the selected path before documenting a compliance posture for customers.

For AvalAI projects with retention constraints:

• prefer the app-managed fallback with store: false;
• store only the job metadata and final answer your policy allows;
• delete failed or expired jobs on a schedule;
• do not put secrets into prompts just because work is asynchronous.

Related Resources

• Responses API Reference
• Streaming Responses
• Webhooks
• Conversation State
• Data Controls
• Context Compaction
• Batch Processing