Fine-tuning Planning via AvalAI

Warning

Feature Not Implemented!

Fine-tuning job creation is currently under development and is not yet available in AvalAI. This page is a planning and migration guide: use it to prepare datasets, evals, and rollout criteria, but do not run the fine-tuning job examples until AvalAI announces the endpoint and supported base models.

Fine-tuning adapts a base model to your examples for a specific task. OpenAI's current supervised fine-tuning guidance emphasizes an eval-first workflow, high-quality JSONL examples, and careful iteration on data before hyperparameters. In AvalAI, apply the same preparation process while treating hosted training as route-, model-, and account-dependent.

Adapted from OpenAI's official fine-tuning, fine-tuning best-practices, Direct Preference Optimization, and Reinforcement Fine-Tuning documentation with AvalAI endpoint, availability, and rollout guidance.

• Higher quality results than prompting alone.
• Training on more examples than fit in a prompt.
• Token savings due to shorter inference prompts.
• Lower latency requests (by using a fine-tuned smaller model).

Models accessed via AvalAI are typically pre-trained on vast datasets. While prompt engineering and few-shot learning are effective, fine-tuning trains the model on many specific examples, tailoring its behavior. Once fine-tuned, you often need simpler prompts at inference time.

Fine-tuning Lifecycle:

1. Prepare and upload training data.
2. Initiate a fine-tuning job via the AvalAI API when the feature is enabled.
3. Evaluate results (e.g., using Evals Guide) and iterate on data if needed.
4. Use your fine-tuned model for inference.

Refer to AvalAI's Pricing page for billing details on fine-tuning training and model usage.

AvalAI Availability Boundary

Fine-tuning availability depends on the upstream provider, model, route, and account. The current data/models.json source does not advertise fine-tunable base models, so do not assume any model can be trained through AvalAI today.

When AvalAI enables hosted fine-tuning, check the Models Overview, Fine-tuning API Reference, and release notes for the exact base model list, supported methods, training limits, file retention, and billing terms.

OpenAI's public fine-tuning docs currently note that OpenAI's hosted fine-tuning platform is winding down for new OpenAI users. Do not copy that availability assumption into AvalAI: treat AvalAI fine-tuning as unavailable until AvalAI publishes a supported route, and treat OpenAI timelines as provider-specific context only.

Choose an Optimization Method

OpenAI's current optimization docs separate model adaptation into supervised fine-tuning, vision fine-tuning, direct preference optimization, and reinforcement fine-tuning. In AvalAI, treat all hosted training methods as future compatibility shapes until the route and base model are announced.

Method	Best for	Do not use when
Supervised fine-tuning (SFT)	Stable tone, format, instruction-following, and cost/latency reduction after prompting works.	You need to teach new factual knowledge; use RAG or tools instead.
Vision fine-tuning	Domain-specific visual recognition when the base model supports image training.	The task is mostly text, general image understanding, or unsupported by the provider route.
Direct preference optimization (DPO)	Aligning subjective preferences such as style, politeness, ranking, or preferred refusal behavior.	You do not have reliable preferred vs. rejected response pairs.
Reinforcement fine-tuning (RFT)	Complex reasoning tasks with measurable rewards and expert graders.	Experts disagree, the base model has zero success, or the task can be solved by lucky guessing.

RFT is especially sensitive to eval design. Prepare graders before training, confirm the model already solves some examples, and keep a held-out validation set that detects reward hacking. If AvalAI later exposes RFT, expect it to be limited to specific reasoning model routes and to require safety screening before deployment.

Optimize Before Fine-tuning

Use fine-tuning only after cheaper interventions have been measured with evals:

1. Model selection: try a stronger or more appropriate model, or increase reasoning effort where supported.
2. Prompt tuning: clarify instructions, constraints, and output contracts.
3. Examples and context: add few-shot examples, retrieved context, or manual RAG with embeddings.
4. Tools: expose deterministic APIs, database lookups, calculators, or guardrail checks through function calling.
5. Accessory models: add small classifier, moderation, evaluator, or routing models around the main call.
6. Fine-tuning: use SFT for labeled demonstrations, DPO for preferred/rejected pairs, or RFT for grader-scored reasoning tasks.

Fine-tuning on a weak prompt can lock in weak behavior. Keep your best production prompt and tool instructions in training examples unless you have eval evidence that removing them is safe.

RFT Readiness Checklist

Before planning reinforcement fine-tuning, confirm all of the following:

• Qualified reviewers agree on what a good answer looks like.
• A grader can score the task without hidden human judgment at training time.
• Baseline eval scores are neither near zero nor already near perfect.
• The base model succeeds on some examples, so reinforcement can improve an existing capability.
• The task is hard to game with lucky guesses or shallow pattern matching.
• Training, validation, and final evaluation sets are separated to avoid leaking eval data into training.

When to Use Fine-tuning

Fine-tuning requires careful investment. First, try optimizing results using:

• Prompt Engineering: Refine instructions and examples. See Prompt Engineering Guide.
• Prompt Chaining: Break complex tasks into sequential prompts.
• Function Calling: Enable the model to use external tools/data. See Function Calling Guide.

Consider fine-tuning when:

• You need to consistently set a specific style, tone, or format.
• You need reliable outputs for complex instructions.
• You need to handle many specific edge cases correctly.
• The desired skill is hard to articulate solely through prompting ("show, not tell").
• You want to potentially reduce cost/latency by using a tuned smaller model to match the quality of a larger one.

Initial prompt engineering work is valuable even if you fine-tune, as the best prompts are often included in the training data.

Preparing Your Dataset

Create a diverse set of demonstration conversations in the format expected by the AvalAI Chat Completions API.

• Format: JSONL file, where each line is a JSON object representing one training example.
• Structure: Each example contains a list of messages, where each message has role (system, user, or assistant) and content.
• Targeting: Include examples that specifically address the areas where the base model fails or doesn't meet requirements. The assistant messages should contain the ideal desired output.

Example Format (Sarcastic Bot)

{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "What's the capital of France?"}, {"role": "assistant", "content": "Paris, as if everyone doesn't know that already."}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "Who wrote 'Romeo and Juliet'?"}, {"role": "assistant", "content": "Oh, just some guy named William Shakespeare. Ever heard of him?"}]}
{"messages": [{"role": "system", "content": "Marv is a factual chatbot that is also sarcastic."}, {"role": "user", "content": "How far is the Moon from Earth?"}, {"role": "assistant", "content": "Around 384,400 kilometers. Give or take a few, like that really matters."}]}

Multi-Turn Examples & Weighting

You can include multiple turns in an example. By default, the model trains on all assistant messages. To skip training on a specific assistant turn, add "weight": 0 to that message object. Use "weight": 1 (or omit the weight key) for turns you want the model to learn from.

{
  "messages": [
    {
      "role": "system",
      "content": "Marv is factual and sarcastic."
    },
    {
      "role": "user",
      "content": "Capital of France?"
    },
    {
      "role": "assistant",
      "content": "Paris",
      "weight": 0
    },
    {
      "role": "user",
      "content": "More sarcastic please."
    },
    {
      "role": "assistant",
      "content": "Paris, like it's a big secret.",
      "weight": 1
    }
  ]
}

Crafting Prompts in Data

Including the best-performing system prompt and user prompts (that you identified during prompt engineering) in every training example generally yields the best results, especially with fewer (<100) examples.

If you shorten or omit instructions in training data to save costs, the model might implicitly learn those instructions, making it harder to override them later at inference time. Teaching the model purely by demonstration (without instructions in the data) might require significantly more examples.

Example Count Recommendations

• Minimum: 10 examples required.
• Recommended Start: 50-100 high-quality examples, then evaluate on a holdout set before adding more data.
• Evaluation: Check if the model shows improvement. Clear improvement suggests adding more data will likely help further. No improvement might mean rethinking the task setup or data structure.

Train/Validation Split

Split your dataset into training and validation sets. Providing a validation file when creating the fine-tuning job allows AvalAI (or the underlying provider) to compute metrics during training, giving you feedback on model improvement. Ensure no overlap between training and validation data.

Create the validation set before training starts and keep it frozen. Use it to compare the base model, each fine-tuned checkpoint, and any prompt/tool alternative. For RFT-style workflows, keep a separate final test set in addition to the validation set because the grader itself can become part of the optimization target.

Token Limits

Training examples are truncated if they exceed the model's maximum context length for training. Check the Models Overview for specific limits of fine-tunable models available via AvalAI. Ensure the total tokens per example (sum of content fields) fit within the limit. Use a tokenizer library (like tiktoken for OpenAI models) to count tokens accurately.

Estimate Costs (Use AvalAI Pricing)

Consult AvalAI's Pricing page for fine-tuning costs. The general formula is:

(AvalAI Base Cost per 1M Training Tokens / 1,000,000) * Total Tokens in Training File * Number of Epochs

Example: A 100k token file trained for 3 epochs on gpt-5.4-mini via AvalAI would have a cost based on AvalAI's specific rate for that model.

Validation tokens are typically not charged.

Check Data Formatting

Before uploading, validate your JSONL file:

• Each line must be a valid JSON object.
• Each object must have a messages key.
• Each message must have role and content.
• Roles must be system, user, or assistant.
• There must be at least one assistant message per example (unless using specific formats like DPO).
• Check token counts per example against model limits.

Data Quality Before Data Quantity

OpenAI's fine-tuning best-practice guidance recommends fixing data quality and distribution before changing hyperparameters or adding large volumes of examples:

• Add examples that directly demonstrate the behavior the base model still misses.
• Remove contradictions, grammar errors, style drift, and unsafe claims from the target assistant messages.
• Match the training distribution to production. If refusals, tool calls, or long answers are overrepresented in data, the fine-tuned model will likely overproduce them.
• Do not let examples teach impossible capabilities, such as saying an action was completed when the application cannot perform that action.
• Keep labeling guidelines consistent across annotators; model quality is capped by human agreement on the desired answer.
• Prefer a smaller, high-quality dataset over a larger noisy dataset. Increase data only after evals show the current dataset is helping.

When data quality is stable, scale quantity deliberately. Fine-tune on the current dataset and on a smaller subset, compare eval results, and use the quality gap to estimate whether adding more examples is likely to help. This is safer than assuming a larger file will compensate for noisy labels.

Upload Training File

When hosted fine-tuning is enabled, upload your validated JSONL file using the AvalAI-compatible Files API and set purpose to fine-tune. Until then, keep this as a migration template and validate your data locally.

# Python Example: Uploading file via AvalAI-compatible endpoint
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",  # AvalAI API endpoint
)

try:
    file_response = client.files.create(
        file=open("my_training_data.jsonl", "rb"), purpose="fine-tune"
    )
    training_file_id = file_response.id
    print(f"File uploaded successfully: {training_file_id}")
except FileNotFoundError:
    print("Error: Training file not found.")
except Exception as e:
    print(f"An error occurred during file upload: {e}")

File processing might take time after upload.

Create a Fine-tuning Job

Start a job via the AvalAI API only after AvalAI announces support for v1/fine_tuning/jobs on your account and model.

# Python Example: Creating Fine-tuning Job via AvalAI
# Assumes 'training_file_id' was obtained from the file upload step
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",  # AvalAI API endpoint
)

try:
    job = client.fine_tuning.jobs.create(
        training_file=training_file_id,
        model="fine-tunable-model-id",  # Use a supported fine-tunable base model ID
        # Optional parameters:
        # validation_file="validation_file_id",
        # hyperparameters={"n_epochs": 3}, # Or {"learning_rate_multiplier": 2, "batch_size": 1}
        # suffix="my-custom-model-name", # Max 64 chars
        # method={"type": "dpo", "dpo": {"hyperparameters": {"beta": 0.1}}} # For DPO
    )
    print(f"Fine-tuning job created: {job.id}")
    print(f"Status: {job.status}")
except Exception as e:
    print(f"An error occurred creating the fine-tuning job: {e}")

• model: Must be a fine-tunable base model identifier available through AvalAI.
• training_file: The ID returned from the file upload.
• validation_file (Optional): ID of your uploaded validation set.
• hyperparameters (Optional): Customize n_epochs, learning_rate_multiplier, batch_size. Defaults are often suitable. See API Reference for details.
• suffix (Optional): Add a custom name component to your fine-tuned model ID (max 64 chars).
• method (Optional): Specify fine-tuning method, e.g., {"type": "dpo", ...} for Preference Fine-tuning. Default is supervised.

Jobs are queued and can take minutes to hours. You might receive an email notification upon completion (provider-dependent).

Job State Playbook

When AvalAI exposes hosted fine-tuning, treat job status as an operational signal, not just a progress label:

Status	What to do
validating / preparing	Confirm files were uploaded with purpose="fine-tune" and that JSONL, token limits, image formats, and method-specific fields are valid.
queued	Record dataset version, model ID, hyperparameters, method, and expected eval suite before training starts.
running	Watch events for loss, validation loss, token accuracy, reward metrics, grader errors, and parse errors depending on the method.
succeeded	Do not deploy immediately. Run the frozen eval suite, safety checks, and side-by-side comparison against the base model.
failed / cancelled	Keep the job ID, event log, file IDs, and request ID for support and postmortem analysis.

For long-running RFT-style jobs, OpenAI's training lifecycle includes pause/resume and checkpoint evaluation patterns. Treat those as conditional AvalAI features: use them only if AvalAI publishes the endpoints for your route, and evaluate every checkpoint on a held-out set before promoting it.

You can manage jobs programmatically:

# Python Example: Managing Fine-tuning Jobs via AvalAI
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",  # AvalAI API endpoint
)

try:
    # List 10 most recent jobs
    recent_jobs = client.fine_tuning.jobs.list(limit=10)
    print("Recent Jobs:", recent_jobs.data)

    # Retrieve a specific job's status
    job_id = "ftjob-xxxxxxxxxxxx"  # Replace with your job ID
    job_status = client.fine_tuning.jobs.retrieve(job_id)
    print(f"Job {job_id} Status:", job_status)
    fine_tuned_model_id = job_status.fine_tuned_model

    # List events for a job
    events = client.fine_tuning.jobs.list_events(job_id=job_id, limit=10)
    print(f"Events for {job_id}:", events.data)

    # Cancel an in-progress job (if needed)
    # cancel_status = client.fine_tuning.jobs.cancel(job_id)
    # print(f"Cancel Status for {job_id}:", cancel_status)

    # Delete a fine-tuned model (requires appropriate permissions)
    if fine_tuned_model_id:
        # delete_status = client.models.delete(fine_tuned_model_id)
        # print(f"Deletion status for {fine_tuned_model_id}:", delete_status)
        pass  # Uncomment delete line if needed

except Exception as e:
    print(f"An error occurred managing fine-tuning jobs: {e}")

Use a Fine-tuned Model

Once a job succeeds, the fine_tuned_model field in the job status will contain the ID of your new model (e.g., ft:gpt-5.4-mini:avalai-org:my-suffix:xxxxxx). Use this ID in the model parameter of your Chat Completions API calls.

# Python Example: Using Fine-tuned Model via AvalAI
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",  # AvalAI API endpoint
)
fine_tuned_model_id = (
    "ft:gpt-5.4-mini:avalai-org:my-suffix:xxxxxx"  # Replace with your model ID
)

try:
    completion = client.chat.completions.create(
        model=fine_tuned_model_id,
        messages=[
            {
                "role": "system",
                "content": "Marv is a factual chatbot that is also sarcastic.",
            },  # System prompt might still be helpful
            {"role": "user", "content": "What's the capital of France?"},
        ],
    )
    print(completion.choices[0].message.content)
except Exception as e:
    print(f"An error occurred using the fine-tuned model: {e}")

Responses API version

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model=fine_tuned_model_id,
    instructions="Marv is a factual chatbot that is also sarcastic.",
    input="What's the capital of France?",
)

print(response.output_text)

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

The model might take a few minutes to become fully available for inference after the job completes.

Analyzing Your Fine-tuned Model

Monitor training progress via:

• Events: The list_events endpoint shows metrics logged during training (loss, accuracy). Look for decreasing loss and increasing accuracy.

{
  "object": "fine_tuning.job.event",
  "id": "ftevent-...",
  "created_at": ...,
  "level": "info",
  "message": "Step 100/200: training loss=0.25, validation loss=0.30",
  "data": {
    "step": 100,
    "train_loss": 0.25,
    "valid_loss": 0.30, // On batch during step

    "train_mean_token_accuracy": 0.91,
    "valid_mean_token_accuracy": 0.89, // On batch during step

    // "full_valid_loss": 0.28, // On full validation set at epoch end

    // "full_valid_mean_token_accuracy": 0.90 // On full validation set at epoch end

  },
  "type": "metrics"
}

• Result Files: After completion, the job object contains result_files. Download these CSV files via the Files API to see detailed step-by-step metrics.
• Manual Evaluation: Generate responses from your fine-tuned model and the base model on a held-out test set and compare quality side-by-side.
• Evals: Use systematic evaluation frameworks (Evals Guide) for quantitative comparison.

Checkpoints and Promotion

If a provider route returns intermediate checkpoints, evaluate them like separate candidate models:

1. Compare the final model, each checkpoint, the base model, and the current prompt-only baseline on the same frozen eval set.
2. Prefer validation metrics over training metrics when choosing a checkpoint; training-only gains can indicate overfitting.
3. For RFT, inspect grader outputs and reward traces, not just aggregate reward. A model can learn to satisfy a weak grader without improving user-visible quality.
4. Promote a checkpoint only after safety checks pass and the rollout plan includes rollback to the previous model ID.

Iterating

• Data Quality: If results are poor, review training data for errors, inconsistencies, lack of necessary information, or imbalance. High-quality data is crucial.
• Data Quantity: If the model shows improvement but isn't perfect, adding more high-quality examples often helps, especially for edge cases. Doubling data can lead to noticeable gains.
• Hyperparameters: Adjust n_epochs (increase if underfitting, decrease if overfitting/losing diversity) or learning_rate_multiplier (increase if not converging) if default settings aren't optimal.

OpenAI's best-practice guidance recommends starting with default hyperparameters and changing them only after eval evidence: increase epochs if the model underfits narrow tasks, decrease epochs if it loses diversity or overfits, and adjust learning rate only when training is not converging.

For RFT-style training, also monitor train_reward_mean, valid_reward_mean, grader-specific reward, sample parse errors, grader errors, and reasoning-token usage. Rising training reward with flat or falling validation reward usually means the model or grader is overfitting. Large parse-error rates usually mean the response schema or grader variables need to be fixed before more training.

Vision Fine-tuning (Model-Dependent)

If the base model available via AvalAI supports vision, you can include images in your fine-tuning data (JSONL format).

• Format: Use the standard Chat Completions message format, including image_url content parts for the user role. Images can be URLs or Base64 data URLs.

{
  "messages": [
    {
      "role": "system",
      "content": "Identify the cheese."
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "What is this?"
        },
        {
          "type": "image_url",
          "image_url": {
            "url": "https://...",

            "detail": "low"
          }
        }
      ]
    },
    {
      "role": "assistant",
      "content": "Danbo"
    }
  ]
}

• Requirements: Check provider documentation via AvalAI for supported image formats (PNG, JPEG, WEBP, non-animated GIF typically), size limits (e.g., <10MB), and content restrictions (no people/faces/children/CAPTCHAs usually).
• Cost: Use detail: "low" for images to significantly reduce token count and training cost.

Preference Fine-tuning (DPO) (Model-Dependent)

Direct Preference Optimization (DPO) fine-tunes based on preferred vs. non-preferred responses.

• Data Format: Each JSONL line needs input (like user messages), preferred_output (ideal assistant message list), and non_preferred_output (suboptimal assistant message list).

{
  "input": {
    "messages": [
      {
        "role": "user",
        "content": "How's SF weather?"
      }
    ]
  },
  "preferred_output": [
    {
      "role": "assistant",
      "content": "Sunny, high 68F."
    }
  ],
  "non_preferred_output": [
    {
      "role": "assistant",
      "content": "It's okay today."
    }
  ]
}

• Method: Specify method={"type": "dpo", ...} when creating the job.
• Beta Hyperparameter: Controls adherence to previous behavior vs. new preferences (0=aggressive, 2=conservative, default="auto").
• Stacking: Often beneficial to first run Supervised Fine-Tuning (SFT) on preferred responses, then run DPO on the resulting SFT model.

Check if DPO is supported for the base model you choose via AvalAI.

Safety and Deployment Gates

Do not deploy a fine-tuned model only because the training job completed. Before release:

• Run the same eval suite against the base model, fine-tuned model, and current prompt-only baseline.
• Review failure slices by language, user segment, document type, refusal type, and tool-use path.
• Run safety checks for policy, privacy, hallucination, prompt-injection, and data-exfiltration scenarios.
• Confirm the tuned model does not claim actions your product cannot perform.
• Keep a rollback path to the base model or previous fine-tuned checkpoint.
• Log model ID, training dataset version, eval run ID, request ID, and production feedback tags for each release.

For regulated or high-impact domains, require human review before training data is uploaded and before the tuned model is enabled for users.