Building Agents with AvalAI

Agents are applications that plan, call tools, keep state, and complete multi-step work. OpenAI's current agent guidance separates two layers: use the Responses API when one model loop plus tools is enough, and use an agent framework when your application owns orchestration, approvals, state, and observability. In AvalAI, build that loop with /v1/responses, /v1/chat/completions, function calling, retrieval, and your own product logic.

AvalAI Agent Architecture

Layer	What it does	AvalAI path
Model	Reasons, plans, and decides whether tools are needed.	Start with gpt-5.5, gpt-5.4-pro, claude-opus-4-8, gemini-3.5-flash, or another supported model from Model Details.
Instructions	Defines role, boundaries, output format, and tool-use policy.	Use instructions in /v1/responses or developer/system messages in /v1/chat/completions.
Tools	Lets the agent read or change external systems.	Use Function Calling, Web Search, and application-owned tools.
State	Carries conversation, reasoning, and tool results across turns.	Prefer previous_response_id where supported; otherwise replay prior messages or typed output items.
Knowledge	Grounds answers in private data.	Use Retrieval, Embeddings, and Manual RAG.
Guardrails	Blocks unsafe, unauthorized, or expensive actions.	Validate tool arguments, require approval for risky actions, and run Moderation where appropriate.
Observability	Shows why a run behaved the way it did.	Log prompts, selected model, tool calls, tool outputs, citations, latency, errors, and user feedback.

Choose An Agent Pattern

• Single-call assistant: One /v1/responses request with strong instructions, optional structured output, and no tools.
• Tool loop agent: The model returns function_call items, your server executes approved functions, and you send function_call_output items back until the model returns a final message.
• RAG agent: Your app retrieves chunks first, then asks the model to answer only from cited sources.
• Multi-agent workflow: Split work into specialist steps in your own code: planner, retriever, executor, reviewer, and final responder.
• Voice or multimodal agent: Combine image, audio, or speech APIs with the same state and tool loop.

Use OpenAI's Agents SDK documentation as an architecture reference for loops, handoffs, approvals, and tracing. Unless AvalAI announces a hosted agent runtime for that surface, keep orchestration in your application and call AvalAI endpoints directly.

Hosted Builders, ChatKit, and AvalAI

OpenAI's Agent Builder and ChatKit docs describe hosted OpenAI product surfaces. Agent Builder is scheduled to shut down on November 30, 2026, while ChatKit remains a separate UI/product surface. In AvalAI docs, use those pages for design patterns only:

• Map nodes to application code, strict function tools, retrieval calls, and explicit state objects.
• Map human-approval nodes to your own policy engine or trusted user confirmation UI.
• Map trace graders to Agent Workflow Evals and stored application traces.
• Map ChatKit widgets or themes to your own frontend; do not imply AvalAI hosts the ChatKit runtime.
• Keep OpenAI-hosted availability, workspace permissions, and deprecation timelines separate from AvalAI route support.

Agent Safety Controls

OpenAI's agent-safety guidance highlights two recurring failure modes: prompt injection from untrusted content and accidental private-data leakage through tools or connectors. In AvalAI, reduce those risks by making every boundary explicit before the model can call a tool:

Risk	AvalAI control
Untrusted text overrides policy	Keep retrieved pages, emails, tickets, and user uploads in input, not developer/system instructions. Extract only validated fields before routing them into privileged steps.
Tool over-sharing	Return compact tool outputs and redact secrets before sending data back to the model. Do not forward OAuth tokens, raw customer records, or internal logs as tool output.
Unsafe writes	Require approval before payments, account changes, emails, deletes, external posts, or data exports. Keep write tools separate from read-only tools.
Freeform handoff drift	Use Structured Outputs for planner decisions, handoff targets, risk labels, and final answer schemas.
Hidden regressions	Run Agent Workflow Evals against traces, tool calls, handoffs, and refusal behavior before widening traffic.

Treat model choice as one layer of defense, not the whole safety plan. Stronger instruction-following models can reduce risk, but approvals, schema checks, tenant authorization, and audit logs must live in your application.

Minimal Responses Tool Loop

Python

import json
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)


def get_order_status(order_id: str) -> str:
    return f"Order {order_id} is packed and waiting for pickup."


tools = [
    {
        "type": "function",
        "name": "get_order_status",
        "description": "Look up the current shipping status for an order.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
            "additionalProperties": False,
        },
        "strict": True,
    }
]

response = client.responses.create(
    model="gpt-5.5",
    instructions="You are a support agent. Call tools only when needed.",
    input="Where is order A123?",
    tools=tools,
)

while True:
    tool_outputs = []

    for item in response.output:
        if item.type != "function_call":
            continue

        args = json.loads(item.arguments)
        if item.name == "get_order_status":
            result = get_order_status(args["order_id"])
        else:
            result = "Unsupported tool."

        tool_outputs.append(
            {
                "type": "function_call_output",
                "call_id": item.call_id,
                "output": result,
            }
        )

    if not tool_outputs:
        print(response.output_text)
        break

    response = client.responses.create(
        model="gpt-5.5",
        previous_response_id=response.id,
        input=tool_outputs,
    )

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

function getOrderStatus(orderId) {
  return `Order ${orderId} is packed and waiting for pickup.`;
}

const tools = [
  {
    type: "function",
    name: "get_order_status",
    description: "Look up the current shipping status for an order.",
    parameters: {
      type: "object",
      properties: { order_id: { type: "string" } },
      required: ["order_id"],
      additionalProperties: false,
    },
    strict: true,
  },
];

let response = await client.responses.create({
  model: "gpt-5.5",
  instructions: "You are a support agent. Call tools only when needed.",
  input: "Where is order A123?",
  tools,
});

while (true) {
  const toolOutputs = [];

  for (const item of response.output) {
    if (item.type !== "function_call") continue;

    const args = JSON.parse(item.arguments);
    const output =
      item.name === "get_order_status"
        ? getOrderStatus(args.order_id)
        : "Unsupported tool.";

    toolOutputs.push({
      type: "function_call_output",
      call_id: item.call_id,
      output,
    });
  }

  if (toolOutputs.length === 0) {
    console.log(response.output_text);
    break;
  }

  response = await client.responses.create({
    model: "gpt-5.5",
    previous_response_id: response.id,
    input: toolOutputs,
  });
}

For models or integrations that still use /v1/chat/completions, keep the same architecture but send tool results as role: "tool" messages with the matching tool_call_id.

Handoffs And Specialist Design

A handoff is just a controlled transfer of responsibility. In app-owned AvalAI agents, implement it explicitly:

1. Define each specialist's input contract, allowed tools, and final output schema.
2. Let a planner choose the next specialist only from an allowlist.
3. Pass a compact state object: user goal, constraints, completed steps, source IDs, and pending approvals.
4. Record who owns the final user-visible answer.
5. Use evals to catch loops, premature handoffs, and unsafe tool choices.

Do not let one agent invent arbitrary tools or delegate to an unregistered worker. Keep permissions attached to the tool executor, not to the model prompt.

Trace And Evaluation Plan

Before production, create a small trace dataset that covers happy paths, tool errors, prompt-injection attempts, permission failures, and handoff edge cases. Score each run on:

• tool choice: whether the agent selected the right function and arguments;
• grounding: whether retrieved source IDs support the answer;
• safety: whether the agent refused or escalated risky requests;
• handoff quality: whether the planner chose the right specialist and stopped loops;
• cost and latency: whether retries, tool fan-out, and reasoning effort stayed within budget.

If you do not have hosted trace tooling for your AvalAI route, store an application trace with response.id, model, prompt version, tool calls, tool outputs, approval decisions, final status, latency, token usage, and user feedback. That trace is enough to run offline graders with /v1/responses or to replay failures in a staging environment.

Production Checklist

• Instructions: include goal, forbidden behavior, citation rules, and escalation rules.
• Tools: make schemas strict, validate arguments server-side, and return compact structured results.
• Approvals: require human or policy approval before payments, account changes, emails, deletes, or external writes.
• State: choose previous_response_id for managed state or replay typed items/messages when you need full control.
• Retrieval: enforce tenant and document permissions before similarity search, not after the model answers.
• Streaming: stream progress to users, but execute tools only after complete arguments are available.
• Evaluation: test task success, tool precision, source grounding, latency, cost, refusal behavior, and recovery from tool errors.

Related Resources

• Responses API Reference
• Function Calling
• Tools
• Conversation State
• Retrieval
• Agentic Guardrails Example
• Reasoning Function Calls Example