Speech to Text Guide

Speech-to-text turns spoken audio into text for captions, search, summaries, call analytics, support workflows, and accessibility. AvalAI exposes OpenAI-compatible transcription and translation endpoints at https://api.avalai.ir/v1, so OpenAI SDKs work when you set the AvalAI base URL and use AVALAI_API_KEY.

Related docs: Audio API, Audio processing, Processing audio in Chat Completions

Choose a Model

Model	Best for	Notes
gpt-4o-transcribe	High-accuracy file transcription	Supports prompts and json / text output.
gpt-4o-mini-transcribe	Lower-cost transcription	Good default for routine workloads.
gpt-4o-transcribe-diarize	Speaker-labeled transcripts	Use response_format="diarized_json" and chunking_strategy="auto" for longer audio.
whisper-1	Broad compatibility, subtitles, translations	Supports srt, vtt, verbose_json, and word timestamps.
scribe_v2 / scribe_v1	ElevenLabs transcription routes	Useful when you standardize on ElevenLabs audio models.
groq.whisper-large-v3 / groq.whisper-large-v3-turbo	Whisper-compatible Groq routes	Useful for low-latency or provider-specific routing.

Prepare Audio

• Use common formats such as mp3, mp4, mpeg, mpga, m4a, wav, or webm; some OpenAI-compatible reference routes also accept flac or ogg.
• Keep OpenAI-compatible uploads at or below 25 MB; split or compress longer recordings.
• Prefer speech-focused mono audio, remove silence when possible, and avoid cutting chunks mid-sentence.
• Add a short prompt for product names, acronyms, speaker context, or expected spelling when the selected model supports it. OpenAI's diarization model does not support prompt, so keep diarization prompts in your post-processing step instead.
• Use direct input_audio in Chat Completions only when the model must inspect audio itself; otherwise transcribe first.

Language and Reliability Checks

OpenAI's speech-to-text guidance lists Persian and many other languages as supported for Whisper-style transcription and translation, but quality still depends on audio conditions, accents, domain vocabulary, and the selected provider route. In AvalAI, make language handling explicit:

• Set language when the route supports it and the input language is known.
• Keep a product glossary in prompt for names, SKUs, acronyms, and expected spelling; for split files, pass the previous chunk transcript as context when the model supports prompts.
• Use /v1/responses as a post-processing step for punctuation cleanup, glossary correction, classification, extraction, or translation into non-English target languages.
• Add a human review queue for low-confidence segments, diarized speaker-boundary mistakes, or regulated-domain transcripts.
• Evaluate Persian, mixed-language, noisy, and accented samples before choosing a cheaper transcription route.

Basic Transcription

Bash

curl https://api.avalai.ir/v1/audio/transcriptions \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F file="@meeting.mp3" \
  -F model="gpt-4o-transcribe" \
  -F response_format="text" \
  -F prompt="Product names include AvalAI, Qwen, Grok, Claude, and Gemini."

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

with open("meeting.mp3", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="gpt-4o-transcribe",
        file=audio_file,
        response_format="text",
        prompt="Product names include AvalAI, Qwen, Grok, Claude, and Gemini.",
    )

print(transcript)

JavaScript

import fs from "node:fs";
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const transcript = await client.audio.transcriptions.create({
  model: "gpt-4o-transcribe",
  file: fs.createReadStream("meeting.mp3"),
  response_format: "text",
  prompt: "Product names include AvalAI, Qwen, Grok, Claude, and Gemini.",
});

console.log(transcript);

Migrate from Whisper Safely

Migrating a transcription pipeline is a compatibility change, not only a model-name change. Keep the endpoint, input file, and json response format stable first; compare outputs before changing prompts, streaming behavior, or downstream parsing.

Adapted from the official Whisper-to-transcription migration example and openai/openai-cookbook, with AvalAI endpoint, API key, supported-model, and support-boundary changes.

import os

from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)


def transcribe(model: str):
    with open("meeting.wav", "rb") as audio_file:
        return client.audio.transcriptions.create(
            model=model,
            file=audio_file,
            response_format="json",
        )


legacy = transcribe("whisper-1")
candidate = transcribe("gpt-4o-transcribe")

print("Whisper:", legacy.text)
print("Candidate:", candidate.text)

Choose the migration boundary by feature:

Requirement	Migration decision
json file transcription	Evaluate gpt-4o-transcribe; change only the model first, then read plain text from the returned .text field when needed.
srt, vtt, verbose_json, or word timestamps	Keep whisper-1 until the replacement route explicitly supports the required format.
Speech translated into English	Keep /v1/audio/translations with whisper-1 unless AvalAI enables another translation model.
Speaker labels	Use gpt-4o-transcribe-diarize with diarized_json; this is a separate workflow, not a drop-in model swap.
Progressive text from a completed file	Use stream=true only on a compatible GPT-4o transcription route and finalize on transcript.text.done.
Live microphone or call audio	Use Realtime only when AvalAI explicitly enables that route; otherwise send bounded rolling chunks. File-output streaming is not live audio input.

Before rollout, score the Whisper baseline and candidate against the same human-verified reference transcripts. Compare word error rate, exact names and domain terms, accents, background noise, mixed-language speech, final completeness, first-delta and final latency, p95 latency, retry behavior, and current cost/rate limits. Start with a model-only comparison, canary the change, and retain a whisper-1 rollback path for incompatible formats or regressions.

Response Formats

Format	Use when	Model notes
json	You want a structured response with text.	Good default for GPT-4o transcription models.
text	You want a plain text transcript.	Useful for pipelines and scripts.
verbose_json	You need segments or timestamps.	Common with whisper-1.
srt / vtt	You need subtitle files.	Use whisper-1 or compatible Whisper routes.
diarized_json	You need speaker labels.	Use gpt-4o-transcribe-diarize.

For confidence review, request include[]=logprobs only on compatible GPT-4o transcription models with response_format="json". Do not assume logprobs, timestamps, diarization, and prompts can all be combined on the same model; feature support varies by model and provider route.

Speaker Diarization

Use diarization when your downstream workflow needs speaker turns. For recordings longer than 30 seconds, set chunking_strategy to "auto".

If your route supports known-speaker references, you can pass short 2–10 second reference clips as data URLs with known_speaker_names[] and known_speaker_references[]. Keep a fallback path that labels speakers generically (speaker_0, speaker_1) because provider support for named speaker mapping can vary by account and route.

OpenAI's current diarization model is a file/request transcription model, not a Realtime transcription model. For live captions, use a Realtime transcription route only when AvalAI explicitly enables it; for speaker-labeled meeting records, keep the workflow on /v1/audio/transcriptions.

Bash

curl https://api.avalai.ir/v1/audio/transcriptions \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F file="@meeting.wav" \
  -F model="gpt-4o-transcribe-diarize" \
  -F response_format="diarized_json" \
  -F chunking_strategy="auto" \
  -F 'known_speaker_names[]=agent' \
  -F 'known_speaker_references[]=data:audio/wav;base64,AAA...'

Python

import base64
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)


def to_data_url(path: str) -> str:
    with open(path, "rb") as speaker_file:
        encoded = base64.b64encode(speaker_file.read()).decode("utf-8")
    return f"data:audio/wav;base64,{encoded}"


with open("meeting.wav", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="gpt-4o-transcribe-diarize",
        file=audio_file,
        response_format="diarized_json",
        chunking_strategy="auto",
        extra_body={
            "known_speaker_names": ["agent"],
            "known_speaker_references": [to_data_url("agent.wav")],
        },
    )

for segment in transcript.segments:
    print(segment.speaker, segment.start, segment.end, segment.text)

Streaming Transcription

For a completed recording where you want progressive UI updates, set stream=true on compatible GPT-4o transcription models. whisper-1 does not support streamed transcription. For live microphone or call audio, use a realtime architecture only if that route is enabled for your account; otherwise send short rolling chunks to the transcription endpoint.

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

with open("lecture.mp3", "rb") as audio_file:
    stream = client.audio.transcriptions.create(
        model="gpt-4o-mini-transcribe",
        file=audio_file,
        stream=True,
    )

final_text = None

for event in stream:
    if event.type == "transcript.text.delta":
        print(event.delta, end="", flush=True)
    elif event.type == "transcript.text.done":
        final_text = event.text

if final_text is None:
    raise RuntimeError("Transcription stream ended without transcript.text.done")

print(f"\nFinal: {final_text}")

JavaScript

import fs from "node:fs";
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const stream = await client.audio.transcriptions.create({
  model: "gpt-4o-mini-transcribe",
  file: fs.createReadStream("lecture.mp3"),
  stream: true,
});

let finalText = null;

for await (const event of stream) {
  if (event.type === "transcript.text.delta") {
    process.stdout.write(event.delta);
  } else if (event.type === "transcript.text.done") {
    finalText = event.text;
  }
}

if (finalText === null) {
  throw new Error("Transcription stream ended without transcript.text.done");
}

console.log(`\nFinal: ${finalText}`);

Translation to English

Use /v1/audio/translations when you need non-English speech transcribed into English text. Use whisper-1 unless your account has another translation-capable route.

This endpoint is for English output. For translation into another target language, first transcribe the audio, then translate the transcript with /v1/responses using a text model that supports your target language.

Bash

curl https://api.avalai.ir/v1/audio/translations \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F file="@persian.mp3" \
  -F model="whisper-1"

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

with open("persian.mp3", "rb") as audio_file:
    translation = client.audio.translations.create(
        model="whisper-1",
        file=audio_file,
    )

print(translation.text)

Responses Pipeline

Once you have a transcript, use /v1/responses for summarization, extraction, classification, tool calls, or multi-turn state. This is more debuggable than asking an audio model to do every step in one call.

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

with open("support_call.mp3", "rb") as audio_file:
    transcript = client.audio.transcriptions.create(
        model="gpt-4o-transcribe",
        file=audio_file,
        response_format="text",
    )

summary = client.responses.create(
    model="gpt-5.6-luna",
    instructions="Summarize the support call and list action items.",
    input=transcript,
)

print(summary.output_text)

Long Audio

• Split files on silence or speaker turns, not at arbitrary byte boundaries.
• Send the prior chunk transcript as the next chunk's prompt when continuity matters.
• Store chunk start times so you can reconstruct timestamps after transcription.
• Retry transient failures with exponential backoff and idempotent chunk names.

Best Practices

• Use gpt-4o-transcribe for high-value or noisy recordings.
• Use gpt-4o-mini-transcribe or Groq Whisper routes for routine volume.
• Use whisper-1 for translations, subtitles, and timestamp-heavy workflows.
• Use diarization only when speaker labels are required.
• Log model, file size, duration, language, response format, latency, and retry count.
• Treat transcripts as user data; redact or protect sensitive fields before storing.

For a complete diarization, structured extraction, evidence-validation, and human-review workflow, see Speaker-Aware Meeting Intelligence.