Agent Workflow Evaluations

Agent evaluations test the whole workflow, not just the final answer. Use them when a model can call tools, preserve state across turns, apply guardrails, retrieve context, or hand work to another specialist. This guide adapts OpenAI's agent-evaluation guidance to AvalAI's OpenAI-compatible APIs.

Warning

AvalAI does not currently expose hosted trace grading or /v1/evals endpoints. Treat OpenAI traces, datasets, graders, and eval runs as design references. For production today, log typed /v1/responses output items and run local or CI evals with normal AvalAI API calls.

What to Capture

For every agent run, store enough evidence to replay and grade the trajectory:

• Input context: user request, conversation state strategy, retrieved snippets, and model ID.
• Plan: the goal, constraints, success criteria, and stop condition the agent inferred.
• Tool calls: tool name, JSON arguments, side effects, errors, retries, and tool output.
• Guardrails and handoffs: moderation decisions, schema validation failures, blocked actions, specialist handoff source/target, and the reason for each handoff.
• State: previous_response_id, conversation IDs, selected files, citations, handoff targets, and any returned assistant-item fields such as phase when your client manually replays Responses output items.
• Final answer: user-visible output, cited evidence, refusal text, and request ID.

Do not grade only response.output_text. Most agent failures happen earlier: wrong tool selection, unsafe arguments, missing retrieval filters, looping, or losing state between turns.

Evaluation Surfaces

Surface	Use it for	AvalAI path today
Trace review	Debug a single failure or surprising behavior	Log Responses output items, tool inputs/outputs, request IDs, and timing.
Trace grading	Score whole trajectories at scale	Recreate traces as JSONL and grade them with Promptfoo, pytest, or a custom runner.
Dataset evals	Compare prompts, tools, models, or routing changes	Keep version-controlled eval cases and run them in CI.
Production monitoring	Catch drift after deployment	Sample real runs, redact sensitive data, and add failures back to the dataset.

OpenAI recommends starting with traces while behavior is still changing, then moving to datasets and repeatable eval runs once "good" behavior is clear. In AvalAI, apply the same progression with app-side logs and local runners.

Trace Triage Questions

When reviewing a trace, answer these before changing the prompt or model:

• Did the agent choose the correct tool, avoid forbidden tools, and stop after enough evidence?
• Did a handoff happen when it should, and did the source agent pass the right context?
• Did the workflow violate a developer instruction, safety policy, schema rule, or user constraint?
• Did retrieved context, file IDs, or tool outputs actually support the final answer?
• Did retries, empty results, or timeouts trigger a safe recovery path instead of a loop?
• Did the final answer expose internal tool details, omit required citations, or overstate confidence?

Turn every repeated "yes, this failed" answer into a deterministic grader first. Use LLM judging only when the failure depends on nuanced tone, reasoning quality, or partial credit.

Local Trace-Grading Workflow

Use this loop when you are still debugging agent behavior and do not yet have a stable dataset:

1. Pick representative traces: include one successful run, one failure, and one edge case for each tool or handoff path.
2. Write a grader contract: list required tools, forbidden tools, safe argument rules, required citations, and stop conditions.
3. Run deterministic checks first: parse the trace JSON and fail fast on missing tool calls, invalid arguments, unsafe side effects, or uncited final answers.
4. Add LLM grading only for judgment: use it for nuanced recovery quality, handoff rationale, tone, or partial-credit reasoning after deterministic checks pass.
5. Promote to dataset evals: once the rubric is stable, convert the trace cases into version-controlled JSONL and run them in CI for every prompt, tool, model, or routing change.

Agent Trajectory Rubric

Use explicit pass/fail checks before adding LLM-as-judge scoring:

1. Goal recognition: Did the agent identify the correct task and constraints?
2. Tool choice: Did it select the necessary tool and avoid unnecessary tools?
3. Argument safety: Were tool arguments complete, validated, and within allowed side effects?
4. State handling: Did it preserve previous response state, user preferences, and retrieved context?
5. Recovery: Did it handle empty results, tool errors, refusals, and timeouts without looping?
6. Grounding: Did the final answer cite retrieved evidence or say what was missing?
7. Stop condition: Did it finish at the right time instead of over-calling tools?

For high-impact workflows, require human review in addition to automated graders.

Handoff and Complexity Decisions

OpenAI’s evaluation guidance recommends using evals to decide when an agent workflow needs more complexity. Do not split a workflow into multiple specialists just because the architecture looks cleaner. First prove with traces or datasets that a single prompt/tool loop is failing at a specific boundary.

Add a new agent, handoff, retrieval step, or guardrail only when the eval says which boundary needs it:

• Tool overload: the agent repeatedly chooses the wrong tool because too many tools are active.
• Policy conflict: one task needs stricter safety or compliance instructions than the rest of the workflow.
• Context loss: a specialist needs focused context that should not be mixed into every turn.
• Recovery failure: the current loop cannot recover safely from empty results, tool errors, or user topic changes.

After adding a handoff, add explicit eval cases for “handoff should happen,” “handoff should not happen,” and “handoff should return control.” This catches circular routing and specialist agents that answer outside their lane.

Minimal JSONL Trace Case

Keep trace cases portable so you can run them locally today and migrate later if AvalAI adds hosted evals.

{
  "id": "refund-policy-tool-route",
  "input": "Can I refund unused credits?",
  "expected": {
    "must_call_tool": "search_policy",
    "must_not_call_tools": [
      "issue_refund"
    ],
    "final_answer_must_cite": [
      "refund-policy.md#credits"
    ]
  },
  "trace": [
    {
      "type": "function_call",
      "name": "search_policy",
      "arguments": {
        "query": "unused credits refund policy"
      }
    },
    {
      "type": "function_call_output",
      "name": "search_policy",
      "output": {
        "source_id": "refund-policy.md#credits",
        "text": "Unused credits are refundable within 14 days."
      }
    },
    {
      "type": "message",
      "output_text": "Unused credits are refundable within 14 days [refund-policy.md#credits]."
    }
  ]
}

Grade this with deterministic checks first: tool name, forbidden side effects, required citation, and no unsupported claims.

Responses API Logging Pattern

When building agents with /v1/responses, log the typed output stream or final response.output array.

import json
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    instructions="Use tools only when needed. Cite source IDs in final answers.",
    input="Can I refund unused credits?",
    tools=[
        {
            "type": "function",
            "name": "search_policy",
            "description": "Search internal policy snippets.",
            "parameters": {
                "type": "object",
                "properties": {"query": {"type": "string"}},
                "required": ["query"],
                "additionalProperties": False,
            },
        }
    ],
)

trace_items = [item.model_dump() for item in response.output]
print(json.dumps({"response_id": response.id, "trace": trace_items}, indent=2))

If the model returns a function call, execute it in your application, append a function_call_output, and continue the Responses loop. Keep every loop step in the trace so evals can grade the route, not only the answer. If you are not using previous_response_id and instead replay returned output items yourself, preserve returned assistant-item fields such as phase unchanged when they are present; dropping them can make tool preambles or intermediate updates look like final answers in later turns.

CI Checklist

• Add one eval case for every production incident before changing the prompt.
• Run smoke trajectory evals on pull requests that touch prompts, tools, retrieval, or routing.
• Compare candidate model IDs with the same dataset and the same tool schema.
• Track latency, total tool calls, retry count, and token usage alongside pass/fail.
• Keep secrets out of trace fixtures; store redacted examples in the repo.

Related Guides

• Evaluations
• Agents
• Function Calling
• Conversation State
• Agentic Guardrails for Schema Workflows
• OpenAI Agent Workflow Evaluations