Ask AI

Moderation API Reference

The Moderation API helps you identify potentially harmful content in user or model output. Use it to classify standalone text and, with omni-moderation-latest, image inputs. Treat the result as a policy signal for filtering, review queues, account intervention, or human escalation—not as a complete safety system by itself.

Endpoint

POST https://api.avalai.ir/v1/moderations

Choose a Workflow

Workflow	Use when	Endpoint or parameter
Classify standalone inputs	You need a policy signal for text or image content without generating a response	`POST /v1/moderations`
Moderate generated content	You need moderation scores for both the request and the generated answer	`moderation: {"model": "omni-moderation-latest"}` on supported `/v1/responses` or `/v1/chat/completions` requests
Review and escalation	You need a durable audit trail or a human review queue	Store `flagged`, `categories`, `category_scores`, `category_applied_input_types`, `request_id`, and your hashed `safety_identifier`

Request Body

Parameter	Type	Required	Description
`input`	string, array, or content item array	Yes	Text to classify. With `omni-moderation-latest`, inputs can include text and image URL items.
`model`	string	No	Moderation model to use. Current AvalAI moderation models include `omni-moderation-latest`, `omni-moderation-2024-09-26`, `text-moderation-latest`, `text-moderation-stable`, and `cf.llama-guard-3-8b`.

Tip

omni-moderation-latest supports text and images, but not audio. OpenAI's reference limit for moderation images is 20 MB; confirm AvalAI route and provider limits before relying on that maximum in production.

Examples

Basic Moderation Request

bash

curl https://api.avalai.ir/v1/moderations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
    "model": "omni-moderation-latest",
    "input": "I want to kill them."
}'

Python Example

python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.moderations.create(
    model="omni-moderation-latest",
    input="I want to kill them.",
)

# Check if the text is flagged
if response.results[0].flagged:
    print("This content was flagged!")

# Check specific categories
categories = response.results[0].categories
for category, flagged in categories.items():
    if flagged:
        print(f"Content flagged for {category}")

JavaScript Example

javascript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const response = await client.moderations.create({
  model: "omni-moderation-latest",
  input: "I want to kill them.",
});

// Check if the text is flagged
if (response.results[0].flagged) {
  console.log("This content was flagged!");
}

// Check specific categories
const categories = response.results[0].categories;
for (const [category, flagged] of Object.entries(categories)) {
  if (flagged) {
    console.log(`Content flagged for ${category}`);
  }
}

Image and Text Moderation

Use omni-moderation-latest when you need one request to classify text plus an image URL:

bash

curl https://api.avalai.ir/v1/moderations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
    "model": "omni-moderation-latest",
    "input": [
      { "type": "text", "text": "Describe whether this content is safe." },
      {
        "type": "image_url",
        "image_url": {
          "url": "https://example.com/image.png"
        }
      }
    ]
  }'

Generated Output Moderation

OpenAI-compatible generation endpoints may support a top-level moderation object that returns moderation scores alongside the model input and generated output. When AvalAI support is enabled for your selected route, pass moderation: {"model": "omni-moderation-latest"} to /v1/responses or /v1/chat/completions; otherwise, call /v1/moderations before and/or after generation.

PythonJavaScriptBash

python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    input="A user asks for harmful instructions. Refuse briefly and redirect safely.",
    moderation={"model": "omni-moderation-latest"},
)

input_moderation = response.moderation.input
output_moderation = response.moderation.output

print(input_moderation.flagged)
print(output_moderation.flagged)

javascript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const response = await client.responses.create({
  model: "gpt-5.5",
  input:
    "A user asks for harmful instructions. Refuse briefly and redirect safely.",
  moderation: { model: "omni-moderation-latest" },
});

console.log(response.moderation.input.flagged);
console.log(response.moderation.output.flagged);

bash

curl https://api.avalai.ir/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
    "model": "gpt-5.5",
    "input": "A user asks for harmful instructions. Refuse briefly and redirect safely.",
    "moderation": { "model": "omni-moderation-latest" }
  }'

Tip

Inline moderation results should be reviewed before showing the generated text to a user. For streamed responses, moderation scores arrive after the full output is available, not with partial output deltas.

Multiple Inputs in One Request

You can moderate multiple standalone texts in one /v1/moderations request. This is different from the asynchronous /v1/batch API.

python

response = client.moderations.create(
    model="omni-moderation-latest",
    input=[
        "I want to kill them.",
        "The weather is nice today.",
        "I'm going to harm myself.",
    ],
)

# Process each result
for i, result in enumerate(response.results):
    print(f"Text {i+1}: {'Flagged' if result.flagged else 'Not flagged'}")

Response Format

json

{
  "id": "modr-5MWoLO",
  "model": "omni-moderation-latest",
  "results": [
    {
      "flagged": true,
      "categories": {
        "sexual": false,
        "hate": false,
        "harassment": false,
        "self-harm": false,
        "illicit": false,
        "illicit/violent": false,
        "sexual/minors": false,
        "hate/threatening": false,
        "violence/graphic": false,
        "self-harm/intent": false,
        "self-harm/instructions": false,
        "harassment/threatening": false,
        "violence": true
      },
      "category_scores": {
        "sexual": 3.6988e-06,
        "hate": 0.0034766977,
        "harassment": 0.0124246953,
        "self-harm": 2.0235e-06,
        "illicit": 0.0005227032,
        "illicit/violent": 3.682979e-07,
        "sexual/minors": 3.01e-08,
        "hate/threatening": 0.0017639078,
        "violence/graphic": 2.49108e-05,
        "self-harm/intent": 5.447e-07,
        "self-harm/instructions": 8.4e-09,
        "harassment/threatening": 0.0058918546,
        "violence": 0.9223177433
      },
      "category_applied_input_types": {
        "sexual": [
          "text"
        ],
        "hate": [
          "text"
        ],
        "harassment": [
          "text"
        ],
        "self-harm": [
          "text"
        ],
        "illicit": [
          "text"
        ],
        "illicit/violent": [
          "text"
        ],
        "sexual/minors": [
          "text"
        ],
        "hate/threatening": [
          "text"
        ],
        "violence/graphic": [
          "text"
        ],
        "self-harm/intent": [
          "text"
        ],
        "self-harm/instructions": [
          "text"
        ],
        "harassment/threatening": [
          "text"
        ],
        "violence": [
          "text"
        ]
      }
    }
  ]
}

Response Parameters

Parameter	Type	Description
`id`	string	The unique identifier for the moderation request.
`model`	string	The model used for content moderation.
`results`	array	An array of moderation results, one for each input.

Moderation Result Object

Parameter	Type	Description
`flagged`	boolean	Whether the model classifies the content as potentially harmful. Use this as a first-pass signal.
`categories`	object	Per-category boolean flags. Use these for routing, logging, escalation, and review decisions.
`category_scores`	object	Per-category confidence scores from 0 to 1. Custom thresholds may need recalibration as models improve.
`category_applied_input_types`	object	Input types that each category score applies to, such as `text` or `image`.

Available Models

Model	Description	Pricing
`omni-moderation-latest`	Latest OpenAI moderation model for text and images	Free
`omni-moderation-2024-09-26`	Pinned OpenAI omni moderation snapshot	Free
`text-moderation-latest`	Latest text-only OpenAI moderation alias	Free
`text-moderation-stable`	Stable text-only OpenAI moderation alias	Free
`cf.llama-guard-3-8b`	Cloudflare-hosted Llama Guard moderation model	See pricing

Implementation Best Practices

Content Filtering

Implement a tiered approach to content filtering:

Pre-submission filtering: Apply client-side filters before sending content to the API
API moderation: Use the Moderation API for more accurate detection
Human review: For edge cases, implement human review processes

Category Thresholds

Customize category score thresholds based on your application's requirements:

python

def is_content_allowed(result, custom_thresholds=None):
    thresholds = {
        "sexual": 0.5,
        "hate": 0.5,
        "harassment": 0.5,
        "self-harm": 0.5,
        "illicit": 0.5,
        "illicit/violent": 0.5,
        "sexual/minors": 0.1,  # Stricter threshold
        "hate/threatening": 0.5,
        "violence/graphic": 0.5,
        "self-harm/intent": 0.5,
        "self-harm/instructions": 0.5,
        "harassment/threatening": 0.5,
        "violence": 0.5,
    }

    if custom_thresholds:
        thresholds.update(custom_thresholds)

    scores = result.category_scores
    if hasattr(scores, "model_dump"):
        scores = scores.model_dump()

    for category, threshold in thresholds.items():
        if scores.get(category, 0) >= threshold:
            return False

    return True

Moderation Boundaries

For tool-calling requests, moderation can cover tool-call arguments and tool outputs when they appear in conversation content.
Moderation does not inspect tool names, tool descriptions, tool schemas, or response-format schemas.
For streamed generation, treat partial deltas as unmoderated until the final moderation result arrives.
Inline moderation can return an error object for an input or output moderation step; check the result type before reading category scores in production.

Handling False Positives

To reduce false positives, consider:

Using category scores rather than just the flagged field
Implementing a secondary review for borderline cases
Maintaining an allowlist for known safe content that might be flagged

Multiple-Input Processing

For efficiency, batch multiple content items in a single request:

python

def moderate_batch(texts, batch_size=25):
    results = []

    # Process in batches to avoid hitting request limits
    for i in range(0, len(texts), batch_size):
        batch = texts[i : i + batch_size]
        response = client.moderations.create(
            model="omni-moderation-latest",
            input=batch,
        )
        results.extend(response.results)

    return results

For large offline queues, use the asynchronous Batch API only when /v1/moderations is enabled for your account and workload.

Error Handling

The API may return various error codes:

Status Code	Description
400	Bad Request - Your request is invalid.
401	Unauthorized - Your API key is wrong.
403	Forbidden - You don't have permission to access this resource.
404	Not Found - The specified resource could not be found.
429	Too Many Requests - You have exceeded your rate limit.
500	Internal Server Error - We had a problem with our server.

For more information on handling errors, see the Error Handling guide.

Content Policy Considerations

When implementing content moderation, consider:

Transparency: Inform users about your content moderation policies
Appeals process: Provide a way for users to appeal moderation decisions
Cultural context: Be aware that content moderation may vary across cultures and regions
Regular updates: Keep your moderation systems updated as language evolves

Models - Learn about available moderation models
Authentication - Learn about authentication methods
Rate Limits - Learn about API rate limits

Category	Description	Inputs
`sexual`	Content meant to arouse sexual excitement or promote sexual services, excluding wellness or education contexts	Text and images
`sexual/minors`	Sexual content involving a person under 18	Text only
`hate`	Content that expresses, incites, or promotes hate based on protected identity	Text only
`hate/threatening`	Hateful content that also includes violence or serious harm toward the targeted group	Text only
`harassment`	Content that expresses or promotes harassing language toward a target	Text only
`harassment/threatening`	Harassment that includes violence or serious harm	Text only
`illicit`	Instructions, advice, or facilitation for committing illicit acts	Text only
`illicit/violent`	Illicit content that also involves violence or weapons	Text only
`self-harm`	Content that promotes, encourages, or depicts self-harm	Text and images
`self-harm/intent`	Content where the speaker expresses intent to self-harm	Text and images
`self-harm/instructions`	Instructions or encouragement for self-harm	Text and images
`violence`	Content depicting death, violence, or physical injury	Text and images
`violence/graphic`	Graphic depiction of death, violence, or physical injury	Text and images

Moderation API Reference ​

Endpoint ​

Choose a Workflow ​

Request Body ​

Examples ​

Basic Moderation Request ​

Python Example ​

JavaScript Example ​

Image and Text Moderation ​

Generated Output Moderation ​

Multiple Inputs in One Request ​

Response Format ​

Response Parameters ​

Moderation Result Object ​

Categories ​

Available Models ​

Implementation Best Practices ​

Content Filtering ​

Category Thresholds ​

Moderation Boundaries ​

Handling False Positives ​

Multiple-Input Processing ​

Error Handling ​

Content Policy Considerations ​

Related Resources ​

Moderation API Reference

Endpoint

Choose a Workflow

Request Body

Examples

Basic Moderation Request

Python Example

JavaScript Example

Image and Text Moderation

Generated Output Moderation

Multiple Inputs in One Request

Response Format

Response Parameters

Moderation Result Object

Categories

Available Models

Implementation Best Practices

Content Filtering

Category Thresholds

Moderation Boundaries

Handling False Positives

Multiple-Input Processing

Error Handling

Content Policy Considerations

Related Resources