Embeddings

Embeddings turn text into vectors so your application can compare meaning, not just keywords. Use them for semantic search, RAG, recommendations, clustering, duplicate detection, classification, and anomaly detection.

Adapted from OpenAI's official Vector embeddings documentation, with AvalAI endpoint, API key, model-availability, and retrieval guidance.

When To Use Embeddings

Use embeddings when you need fast lookup over your own data before calling a generative model. A common AvalAI flow is:

1. Split documents into stable chunks.
2. Embed each chunk with /v1/embeddings.
3. Store vectors with metadata such as document_id, source URL, permissions, language, and update time.
4. Embed the user query with the same model and dimensions.
5. Retrieve nearest chunks with cosine similarity or dot product.
6. Pass the best snippets to /v1/responses or /v1/chat/completions.

Do not mix models or dimensions inside one vector index. Query vectors and stored vectors must use the same embedding model and dimensionality.

Define the index contract before ingestion: model ID, dimensions, distance metric, chunking policy, language strategy, metadata filters, retention rules, and re-embedding trigger. Changing any of these after launch usually requires rebuilding or backfilling the vector index.

Model And Dimension Choices

Current AvalAI embedding models include OpenAI-compatible models such as text-embedding-3-small, text-embedding-3-large, text-embedding-ada-002, plus provider-specific options from Gemini, Cohere, Alibaba, Cloudflare, BAAI, and Nvidia NIM. Check model details for the latest availability.

Use dimensions only when the selected model supports shorter vectors. Smaller vectors reduce storage, memory, and search cost, but can reduce retrieval quality. If your vector store has a maximum dimension size, set dimensions at embedding time rather than truncating vectors later.

For OpenAI third-generation embedding models, the default vector sizes are 1536 for text-embedding-3-small and 3072 for text-embedding-3-large. These embeddings are normalized to length 1, so cosine similarity and dot product typically produce the same ranking. Use cl100k_base when estimating tokens for these models.

Embeddings are not a substitute for fresh data. If a user asks about recent or private facts, embed your current documents and retrieve them; do not rely on the embedding model to know the answer.

OpenAI's reference API requires non-empty input and documents per-model token limits for embedding requests. AvalAI limits can vary by provider, model, route, and account tier, so validate large ingest jobs against model details, rate limits, and a small dry run before starting a bulk backfill.

Cost, Scale, And Freshness

Embedding requests are usually dominated by input-token cost and storage/search cost. Use the usage.prompt_tokens field to log ingest cost, cache vectors for unchanged chunks, and prefer incremental backfills over rebuilding an entire index after every document edit.

For more than a small in-memory corpus, use a vector database or search service for K-nearest-neighbor lookup. Keep metadata filters outside the model call: first enforce tenant, permission, language, product, freshness, and document-type constraints, then rank the remaining candidates by similarity.

Embeddings are a retrieval primitive, not a source of current facts. OpenAI text-embedding-3-* models are useful for semantic similarity, but your application should retrieve current documents and ask a generation model to answer from that context.

Semantic Search Tuning

Semantic search can find relevant text even when the user and document do not share exact keywords. For example, a query like "when did humans reach the moon?" should still retrieve a passage about "the first lunar landing" because the meanings are close.

In an AvalAI app-side retrieval pipeline:

• Rewrite vague queries into concise search phrases, but keep the original user question for the final model call.
• Filter before ranking with metadata such as tenant, language, product, document type, permissions, and freshness.
• Tune top_k and thresholds so low-confidence chunks are dropped instead of being passed to the model as weak evidence.
• Blend semantic and keyword search when exact IDs, product names, legal terms, or Persian/English spelling variants matter.
• Evaluate changes with labeled questions before changing chunk size, overlap, embedding model, dimensions, or ranking logic.

See Retrieval for a complete app-side retrieval pattern and hosted-retrieval migration notes.

Retrieval Quality Evaluation

Before changing an embedding model, dimensions, chunk size, overlap, language strategy, or ranking formula, build a small labeled retrieval set. This turns OpenAI's semantic-search pattern into an AvalAI production gate:

Field	Purpose
query	Natural user question, including Persian/English spelling variants when relevant.
must_include_doc_ids	Chunks or documents that should appear in the top results.
forbidden_doc_ids	Stale, unauthorized, or misleading chunks that must not be retrieved.
filters	Tenant, permission, product, language, or freshness filters expected before ranking.
expected_answer_source	Source passage the generation model should cite or summarize.

Track recall@k, mean reciprocal rank, latency, token cost, and the percentage of answers grounded in retrieved sources. Run the set before and after every index rebuild. For bilingual products, include Persian queries against Persian content, English queries against English content, and mixed-language queries that mirror real support traffic.

Use-Case Checklist

OpenAI's embedding guide frames embeddings as a general text feature representation. In AvalAI projects, common production uses include:

Use case	Practical pattern
Semantic search and RAG	Embed chunks, retrieve relevant context, then answer with /v1/responses or /v1/chat/completions.
Recommendations	Rank items by vector similarity to a source item or user profile.
Duplicate detection	Compare candidate records and flag near-neighbors above a similarity threshold.
Clustering	Group unlabeled tickets, reviews, or support chats before summarization.
Lightweight classification	Embed labels and input text, then choose the nearest label or train a small classifier.
Anomaly detection	Find vectors far from the normal cluster for review or triage.
Code and documentation search	Embed symbols, function summaries, and docs; rank matches against natural-language developer questions.
Diversity measurement	Analyze similarity distributions to find over-represented topics, near-duplicates, or gaps in a corpus.
Feature encoding	Use vectors as free-text features for small ML classifiers or regression models when labels exist.

Quick Example

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.embeddings.create(
    model="text-embedding-3-small",
    input=[
        "AvalAI supports OpenAI-compatible APIs.",
        "Embeddings help retrieve relevant documents.",
    ],
    encoding_format="float",
)

vectors = [item.embedding for item in response.data]
print(len(vectors), len(vectors[0]))

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const response = await client.embeddings.create({
  model: "text-embedding-3-small",
  input: [
    "AvalAI supports OpenAI-compatible APIs.",
    "Embeddings help retrieve relevant documents.",
  ],
  encoding_format: "float",
});

const vectors = response.data.map((item) => item.embedding);
console.log(vectors.length, vectors[0].length);

Bash

curl https://api.avalai.ir/v1/embeddings \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-3-small",
    "input": [
      "AvalAI supports OpenAI-compatible APIs.",
      "Embeddings help retrieve relevant documents."
    ],
    "encoding_format": "float"
  }'

Production Tips

• Cache embeddings for unchanged chunks; re-embedding every request is slow and expensive.
• Keep chunk IDs stable so you can update only changed documents.
• Store permission metadata and filter results before sending context to a model.
• Use an input array for immediate multi-text embedding; use Batch Processing only for offline jobs when the route supports it.
• Count tokens before embedding large inputs; see Token Counting.
• Treat stored vectors as derived user data. They are not directly readable text, but they can still reveal similarity patterns, so apply the same tenant isolation, retention, and deletion policies you use for source documents.

Related Resources

• Embeddings API Reference
• Manual RAG with Embeddings
• Retrieval Guide
• Rate-Limit-Safe Parallel Requests