Advanced Tool Usage Guide

This section covers advanced techniques for specific API features and use cases.

Chat Completions

• Maintain conversation history: Include relevant prior messages for context, but be mindful of token limits.
• Limit conversation length: Very long histories consume more tokens and can sometimes lead to the model losing track of earlier context. Consider summarizing or truncating history if needed.
• Use function calling / tools: For structured data extraction or interacting with external systems, leverage the tools parameter to get reliable JSON outputs or trigger actions.

# Example using tools (formerly function calling)
functions = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather in a given location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city and state, e.g. San Francisco, CA",
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "The temperature unit",
                    },
                },
                "required": ["location"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "What's the weather like in Boston?"}],
    tools=functions,
    tool_choice="auto",  # Or specify a function name
)

# Process the response, which might include a tool call
message = response.choices[0].message
if message.tool_calls:
    # Handle tool call...
    pass

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

tools = [
    {
        "type": "function",
        "name": "get_current_weather",
        "description": "Get the current weather in a given location.",
        "parameters": {
            "type": "object",
            "properties": {"location": {"type": "string"}},
            "required": ["location"],
            "additionalProperties": False,
        },
    }
]

response = client.responses.create(
    model="gpt-5.5",
    input="What",
    tools=tools,
)

for item in response.output:
    if item.type == "function_call":
        print(item.name, item.arguments)
print(response.output_text)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

Embeddings

• Normalize vectors: When using embeddings for similarity search (e.g., with cosine similarity), normalize the vectors to unit length for accurate comparisons.
• Use dimensionality reduction: For visualizing high-dimensional embeddings, techniques like t-SNE or UMAP can project them into 2D or 3D space.
• Consider chunking: For embedding long documents, break the text into smaller, meaningful chunks (e.g., paragraphs or sections) before generating embeddings.

import numpy as np

# Normalize vectors
def normalize(v):
    norm = np.linalg.norm(v)
    if norm == 0:
    return v
    return v / norm

# Compute cosine similarity between normalized vectors
def cosine_similarity(a_norm, b_norm):
    return np.dot(a_norm, b_norm)

# Example usage (assuming 'emb_a' and 'emb_b' are embedding vectors)
# norm_a = normalize(emb_a)
# norm_b = normalize(emb_b)
# similarity = cosine_similarity(norm_a, norm_b)

Image Generation

• Be detailed and specific: The more detail you provide in the prompt, the closer the generated image will likely be to your vision. Include object descriptions, setting, style, mood, composition, etc.
• Specify style and medium: Explicitly mention the desired artistic style (e.g., "photorealistic", "impressionist painting", "cyberpunk art", "watercolor sketch") or medium (e.g., "digital painting", "photograph", "claymation").
• Iterate on prompts: Generating the perfect image often requires refinement. Analyze the results and adjust your prompt accordingly. Add negative prompts if needed to exclude elements.

Good prompt:

A detailed digital painting of a bioluminescent forest at night, featuring giant glowing mushrooms, sparkling streams, and mystical creatures hiding in the shadows, in the style of fantasy concept art.

Less effective prompt:

A magic forest.

Reproducible Outputs

Chat Completions are non-deterministic by default (meaning model outputs may differ from request to request). However, you can achieve more consistent outputs using the seed parameter and monitoring the system_fingerprint response field.

To receive mostly deterministic outputs across API calls:

• Set the seed parameter to any integer of your choice and use the same value across requests
• Ensure all other parameters (like prompt or temperature) are exactly the same across requests

Note that determinism may still be affected by necessary changes to model configurations. The system_fingerprint field helps track these changes - if this value differs between responses, you might see different outputs due to system-level changes.

# Example of using seed parameter for reproducible outputs
response1 = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Write a short poem about mountains"}],
    seed=123456,  # Set a specific seed value
)

# The system_fingerprint can be checked
fingerprint = response1.system_fingerprint
print(f"System fingerprint: {fingerprint}")

# Using the same seed and parameters should produce similar results
response2 = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Write a short poem about mountains"}],
    seed=123456,  # Same seed as before
)

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    instructions="You are a helpful assistant.",
    input="Write a short poem about mountains",
)

print(response.output_text)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

Managing Tokens

Language models read and write text in chunks called tokens. In English, a token can be as short as one character or as long as one word (e.g., a or apple), and in some languages tokens can be even shorter than one character or longer than one word.

As a rough rule of thumb, 1 token is approximately 4 characters or 0.75 words for English text.

The total number of tokens in an API call affects:

• How much your API call costs, as you pay per token
• How long your API call takes, as writing more tokens takes more time
• Whether your API call works at all, as total tokens must be below the model's maximum context window

Both input and output tokens count toward these quantities. For example, if your API call used 10 tokens in the message input and received 20 tokens in the message output, you would be billed for 30 tokens.

To see how many tokens are used by an API call, check the usage field in the API response:

response = client.chat.completions.create(
    model="gpt-5.5", messages=[{"role": "user", "content": "Hello, how are you?"}]
)

# Check token usage
print(f"Prompt tokens: {response.usage.prompt_tokens}")
print(f"Completion tokens: {response.usage.completion_tokens}")
print(f"Total tokens: {response.usage.total_tokens}")

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    instructions="You are a helpful assistant.",
    input="Hello, how are you?",
)

print(response.output_text)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

For long conversations, be aware that if they approach the model's token limit, replies may be cut off. For example, a conversation that is close to the model's maximum token limit will have very limited space for the response.

Parameter Details

Frequency and Presence Penalties

The frequency and presence penalties can be used to reduce the likelihood of sampling repetitive sequences of tokens:

• Frequency penalty: Reduces repetition of specific tokens based on their existing frequency in the generated text
• Presence penalty: Reduces repetition of any token that has appeared in the generated text

Reasonable values for penalty coefficients are around 0.1 to 1 if you want to slightly reduce repetitive samples. For stronger suppression of repetition, you can increase coefficients up to 2, but this may noticeably degrade sample quality. Negative values can increase the likelihood of repetition.

# Example using frequency and presence penalties
response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[{"role": "user", "content": "Write a paragraph with varied vocabulary"}],
    frequency_penalty=0.7,  # Reduce repetition of specific tokens
    presence_penalty=0.5,  # Reduce repetition of any token that has appeared
)

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    instructions="You are a helpful assistant.",
    input="Write a paragraph with varied vocabulary",
)

print(response.output_text)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

Token Log Probabilities

For an output token at position t, its log probability is log p(token_t | context, previous tokens). Exponentiating it gives the model's conditional likelihood for that token in that position. It is not the probability that the answer or classification is correct: a highly likely token can still represent a wrong result.

For classification, use short labels that are each expected to be one token, validate the response shape, and send uncertain or malformed results to review:

import math
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

LABELS = {
    "A": "account_access",
    "B": "billing",
    "C": "technical_issue",
    "D": "other",
}
EXAMPLE_THRESHOLD = 0.85

ticket = "I was charged twice for the same invoice."
response = client.chat.completions.create(
    model="gpt-5.5",
    messages=[
        {
            "role": "system",
            "content": (
                "Classify the support request. Return exactly one label with no extra text: "
                "A=account access, B=billing, C=technical issue, D=other."
            ),
        },
        {"role": "user", "content": ticket},
    ],
    temperature=0,
    logprobs=True,
    top_logprobs=4,
)

choice = response.choices[0]
token_logprobs = (
    choice.logprobs.content if choice.logprobs and choice.logprobs.content else []
)
raw_label = choice.message.content or ""

if len(token_logprobs) != 1:
    result = {
        "decision": "human_review",
        "reason": "missing_or_multitoken_logprobs",
    }
else:
    selected_token = token_logprobs[0]
    label = raw_label.strip()
    if label not in LABELS or selected_token.token.strip() != label:
        result = {
            "decision": "human_review",
            "reason": "unexpected_label",
        }
    else:
        token_likelihood = math.exp(selected_token.logprob)
        result = {
            "decision": (
                "accept" if token_likelihood >= EXAMPLE_THRESHOLD else "human_review"
            ),
            "label": LABELS[label],
            "token_likelihood": token_likelihood,
        }

print(result)

EXAMPLE_THRESHOLD is only an example, not a production recommendation. Sweep candidate thresholds on a labeled holdout set, then select one using task-specific precision and recall targets, the cost of false accepts, and the resulting human-review volume. Logprob calibration can change with the model, prompt, label set, language, and data distribution, so rerun the evaluation after any of them changes.

For a sequence, sum its token logprobs and then exponentiate the sum to obtain the joint sequence probability. This score naturally favors shorter outputs. Average per-token logprob, or perplexity exp(-average_logprob), can reduce that length effect, but these metrics are useful only for relative comparisons in a controlled setup with the same model and prompt; neither measures correctness.

Adapted for AvalAI from the official Using logprobs example in the OpenAI Cookbook and the openai/openai-cookbook GitHub repository.

Related Resources

• Rate Limits
• Model Selection
• Streaming Responses