AvalAI API Best Practices

Use this page as a quick production checklist for building on AvalAI's OpenAI-compatible API. It adapts OpenAI's official production, deployment, prompting, safety, and accuracy guidance to https://api.avalai.ir/v1.

For deeper implementation details, start with the dedicated guides linked below instead of treating this page as the only source.

Recommended Reading Map

Goal	Start here	Why
Ship a new AI feature	API Deployment Checklist	Responses-first setup, reasoning effort, verbosity, caching, background jobs, and long-running workflows.
Prepare for production	Production Best Practices	Scaling, observability, rate limits, cost controls, security, and release discipline.
Improve prompt quality	Prompt Engineering	Instructions, examples, output formats, evaluation loops, and migration from prompt-only fixes.
Reduce cost or latency	Cost Optimization, Latency Optimization	Model routing, token budgets, streaming, caching, batching, and service tiers.
Improve factual accuracy	Accuracy Optimization, Evals	Use evals before changing prompts, retrieval, fine-tuning, or model choice.
Manage safety risk	Safety Best Practices, Safety Checks, Red Teaming	Add user identifiers, moderation, tool approvals, and release-time risk tests.

Core API Usage

• Use Responses first for new work: start with /v1/responses for state, tools, reasoning, structured output, and future model behavior. Keep /v1/chat/completions for stable existing integrations and models that only expose chat compatibility.
• Keep secrets server-side: read AVALAI_API_KEY from environment variables or a secrets manager. Never expose long-lived keys in browsers, mobile apps, public repos, logs, or screenshots.
• Plan rate limits early: monitor response headers, implement exponential backoff with jitter, and use batching or background processing only when it improves throughput without hiding failures.
• Log enough to debug: capture request ID, model, provider, endpoint, latency, retry count, status, usage, and tenant/user correlation IDs. Do not log raw secrets or unnecessary personal data.
• Validate inputs and outputs: enforce schemas, size limits, file-type checks, moderation, and tool allowlists before expensive or risky operations.

Model And API Selection

Use case	Good starting point	Notes
General assistants	gpt-5.5, gpt-5.4-mini, gpt-5.4-nano	Route by quality, latency, and cost. Use text.verbosity to control answer length on Responses-compatible models.
Complex reasoning	gpt-5.5, gpt-5.4-pro, reasoning-capable alternatives	Tune reasoning.effort per task; do not use maximum effort until evals prove it helps.
Code generation	gpt-5.3-codex, gpt-5.5, claude-opus-4-8, kimi-k2.7-code	Prefer Responses for API-based coding workflows; use Codex setup guides for repo-editing agents.
Fast support or routing	smaller GPT, Claude Haiku, Gemini Flash, or Qwen Flash models	Keep prompts short, cap output length, and stream when user experience benefits.
Retrieval and search	/v1/embeddings plus /v1/responses	Build app-side retrieval today; use file-search docs as a planning reference unless hosted support is enabled.
Image, audio, video	dedicated API guides	Start from Images, Audio, and Videos.

Prompting Defaults

• Put durable behavior in instructions for Responses or a system/developer message for Chat Completions.
• State the task, audience, constraints, and output format explicitly.
• Provide only relevant context; use retrieval or file inputs instead of pasting entire knowledge bases.
• Prefer structured outputs or function tools when downstream code depends on exact fields.
• Evaluate prompts with representative examples before changing models, reasoning effort, or fine-tuning.

OpenAI-Aligned Production Invariants

• Parse Responses defensively: use response.output_text for simple text, but inspect response.output by item type when the request can return tool calls, refusals, annotations, files, images, or reasoning metadata.
• Version prompts in code: keep prompt builders, schemas, examples, and eval fixtures in version control. Avoid hosted prompt objects for AvalAI deployments unless the route is explicitly supported and tested.
• Separate final JSON from tool arguments: use Structured Outputs for typed user-facing answers and function calling for app actions. In both cases, validate the result again in your server code.
• Keep tool contracts strict: set strict: true, add additionalProperties: false, list required fields, and use nullable unions for optional values. For writes, payments, or approvals, set parallel_tool_calls: false.
• Preserve state intentionally: with Responses, use previous_response_id when retention is acceptable; otherwise replay only the required output items, including matching call_id values for tool results.
• Show progress for long work: when a task uses retrieval, tools, or background processing, stream a short preamble or progress event before the final answer so users know the request is moving.

Specific Use Cases

Chat Completions

• Maintain conversation history: Include relevant conversation history for context.
• Limit conversation length: Very long conversations consume more tokens and can lead to context loss.
• Use function calling for actions: Use function calling when the model should call your code. Use Structured Outputs when the final answer itself must be JSON.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given location.",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    }
                },
                "required": ["location"],
                "additionalProperties": False,
            },
            "strict": True,
        },
    }
]

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "What's the weather like in Boston?"}],
    tools=tools,
    parallel_tool_calls=False,
)

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

tools = [
    {
        "type": "function",
        "name": "get_current_weather",
        "description": "Get the current weather in a given location.",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"],
            "additionalProperties": False,
        },
        "strict": True,
    }
]

response = client.responses.create(
    model="gpt-5.5",
    input="What's the weather like in Boston?",
    tools=tools,
    parallel_tool_calls=False,
)

for item in response.output:
    if item.type == "function_call":
        print(item.name, item.arguments)
print(response.output_text)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

Embeddings

• Normalize vectors: For similarity comparisons, normalize embedding vectors.
• Use dimensionality reduction: For visualization, use techniques like t-SNE or UMAP.
• Consider chunking: For long documents, consider chunking text into smaller segments.

import numpy as np


# Normalize vectors
def normalize(v):
    norm = np.linalg.norm(v)
    if norm == 0:
        return v
    return v / norm


# Compute cosine similarity
def cosine_similarity(a, b):
    return np.dot(a, b)

Image Generation

• Be detailed and specific: Provide detailed prompts for better image generation results.
• Specify style and medium: Include information about desired artistic style or medium.
• Iterate on prompts: Refine prompts based on generated results.

Good prompt:

A detailed digital painting of a futuristic city at sunset, with flying cars, tall glass skyscrapers with gardens, and holographic advertisements, in the style of cyberpunk art

Less effective prompt:

A futuristic city

Cost Optimization

Token Usage

• Monitor token usage: Keep track of your token usage to avoid unexpected costs.
• Optimize prompt length: Keep prompts concise while providing necessary context.
• Use smaller models when possible: For simpler tasks, smaller models can be more cost-effective.
• Batch requests: When processing multiple inputs, batch them in a single request.

Caching

• Cache responses: For identical or similar requests, implement caching to avoid redundant API calls.
• Implement TTL: Set appropriate time-to-live for cached responses based on your use case.

import hashlib
import json
from functools import lru_cache


@lru_cache(maxsize=100)
def get_embedding_cached(text, model="text-embedding-3-small"):
    # Create a hash of the text and model to use as a cache key
    cache_key = hashlib.md5((text + model).encode()).hexdigest()

    # Check if we have a cached result
    # (In a real implementation, you'd check a database or cache service)

    # If not in cache, call the API
    response = client.embeddings.create(model=model, input=text)

    embedding = response.data[0].embedding

    # Store in cache
    # (In a real implementation, you'd store in a database or cache service)

    return embedding

Security Considerations

Content Filtering

• Implement content filtering: Use moderation endpoints to filter inappropriate content.
• Set appropriate usage policies: Define clear usage policies for your application.

User Data Privacy

• Minimize data sharing: Only share necessary user data with the API.
• Inform users: Be transparent about how user data is used with AI models.
• Implement data retention policies: Define clear policies for how long user data is stored.

Testing and Evaluation

Evaluating Model Outputs

• Define evaluation metrics: Establish clear metrics for evaluating model performance.
• Conduct human evaluation: For subjective tasks, include human evaluation.
• Use automated testing: Implement automated tests for consistent evaluation.

A/B Testing

• Compare model versions: Test different models or prompts with real users.
• Measure key metrics: Track metrics like user satisfaction, task completion rate, etc.

Application Architecture

Asynchronous Processing

For long-running tasks, implement asynchronous processing:

import asyncio
import os
from openai import AsyncOpenAI

client = AsyncOpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)


async def generate_response(prompt):
    response = await client.chat.completions.create(
        model="gpt-5.5", messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content


async def process_batch(prompts):
    tasks = [generate_response(prompt) for prompt in prompts]
    return await asyncio.gather(*tasks)


# Usage
results = asyncio.run(process_batch(["Hello", "How are you?", "What's the weather?"]))

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    instructions="You are a helpful assistant.",
    input="What's the weather like in Boston?",
)

print(response.output_text)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

Streaming Responses

For better user experience, use streaming responses:

from openai import OpenAI
import os
import sys

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Write a story about a space explorer"}],
    stream=True,
)

for chunk in response:
    if chunk.choices[0].delta.content:
        sys.stdout.write(chunk.choices[0].delta.content)
        sys.stdout.flush()

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
import sys
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

stream = client.responses.create(
    model="gpt-5.5",
    instructions="You are a helpful assistant.",
    input="Write a story about a space explorer",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        sys.stdout.write(event.delta)
        sys.stdout.flush()
    elif event.type == "response.completed":
        break
    elif event.type in {"response.failed", "error"}:
        raise RuntimeError(event)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

Getting Help with Documentation

💡 Pro Tip: You can copy any documentation page URL from docs.avalai.ir and paste it directly into your prompt at chat.avalai.ir (AvalAI Chat Platform). When you include a docs URL in your message, the AI models can access that page's content, allowing you to:

• Ask any model to explain specific documentation sections
• Get help debugging issues using the relevant docs
• Request implementation examples based on the documentation
• Clarify complex concepts with interactive Q&A

Simply paste the documentation URL into your chat message along with your question, and the model will fetch and use that documentation to assist you. This enables faster debugging and implementation by combining our comprehensive documentation with AI-powered assistance.

Conclusion

Following these best practices will help you build more effective, efficient, and secure applications with the AvalAI API. As you gain experience with the platform, you'll develop additional practices tailored to your specific use cases.

Remember that the field of AI is rapidly evolving, so staying updated with the latest models, techniques, and best practices is essential for optimal results.