PDF File Inputs

Choosing a PDF path: Use /v1/responses with input_file for one-off PDF understanding, the Files API with purpose="user_data" when you want to upload once and reference a file_id, and manual RAG when you need retrieval over many documents.

Learn how to use PDF files as inputs to the AvalAI API.

AvalAI models with vision capabilities can accept PDF files as input_file items in the Responses API. Provide PDFs as an external URL (file_url) or Base64-encoded data (file_data) when file storage is not enabled; when file storage is enabled for your account and endpoint, you can also upload once through the Files API and reference the returned file ID (file_id).

How it works

File processing depends on the file type:

• PDF files: for vision-capable models, AvalAI follows the OpenAI-compatible pattern of sending both extracted text and page images into the model context. This helps when diagrams, forms, or charts contain information that is not available in plain text.
• Non-PDF documents and text files: text is extracted, but embedded images or charts are not added to the model context. Convert those files to PDF first when visual fidelity matters.
• Spreadsheets: spreadsheet-like files should be treated as structured data, not long prose. For small worksheets, pass the file directly; for joins, aggregations, or large sheets, extract the data in your application and send a compact summary or use a retrieval workflow.
• Large corpora: do not stuff every document into one request. Use manual RAG with embeddings or file search patterns when you need retrieval over many files.

Use case	Recommended input style
Public or temporary PDF	file_url in the Responses request
Private PDF reused across requests	Upload to /v1/files with purpose="user_data", then pass file_id
Small local PDF without persistence	file_data with a Base64 data URL
Many PDFs or recurring knowledge base queries	Chunk, embed, store, retrieve, then answer with /v1/responses

File URLs

Use a file_url when the PDF is already available at an HTTPS URL and you do not need to persist it in AvalAI file storage.

Send a URL-backed PDF to Responses

Bash

curl "https://api.avalai.ir/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
    "model": "gpt-5.5",
    "input": [
      {
        "role": "user",
        "content": [
          {
            "type": "input_text",
            "text": "Summarize the main obligations in this PDF."
          },
          {
            "type": "input_file",
            "file_url": "https://example.com/contract.pdf"
          }
        ]
      }
    ]
  }'

JavaScript

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const response = await client.responses.create({
  model: "gpt-5.5",
  input: [
    {
      role: "user",
      content: [
        {
          type: "input_text",
          text: "Summarize the main obligations in this PDF.",
        },
        {
          type: "input_file",
          file_url: "https://example.com/contract.pdf",
        },
      ],
    },
  ],
});

console.log(response.output_text);

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"],
    base_url="https://api.avalai.ir/v1",
)

response = client.responses.create(
    model="gpt-5.5",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_text",
                    "text": "Summarize the main obligations in this PDF.",
                },
                {
                    "type": "input_file",
                    "file_url": "https://example.com/contract.pdf",
                },
            ],
        }
    ],
)

print(response.output_text)

Uploading files

In the example below, we first upload a PDF using the Files API, then reference its file ID in an API request to the model.

Upload a file to use in a response

Bash

curl https://api.avalai.ir/v1/files \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -F purpose="user_data" \
  -F file="@draconomicon.pdf"

curl "https://api.avalai.ir/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
    "model": "gpt-5.5",
    "input": [
      {
        "role": "user",
        "content": [
          {
            "type": "input_file",
            "file_id": "file-6F2ksmvXxt4VdoqmHRw6kL"
          },
          {
            "type": "input_text",
            "text": "What is the first dragon in the book?"
          }
        ]
      }
    ]
  }'

JavaScript

import fs from "fs";
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const file = await client.files.create({
  file: fs.createReadStream("draconomicon.pdf"),
  purpose: "user_data",
});

const response = await client.responses.create({
  model: "gpt-5.5",
  input: [
    {
      role: "user",
      content: [
        {
          type: "input_file",
          file_id: file.id,
        },
        {
          type: "input_text",
          text: "What is the first dragon in the book?",
        },
      ],
    },
  ],
});

console.log(response.output_text);

Python

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"], base_url="https://api.avalai.ir/v1"
)

file = client.files.create(file=open("draconomicon.pdf", "rb"), purpose="user_data")

response = client.responses.create(
    model="gpt-5.5",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_file",
                    "file_id": file.id,
                },
                {
                    "type": "input_text",
                    "text": "What is the first dragon in the book?",
                },
            ],
        }
    ],
)

print(response.output_text)

Base64-encoded files

You can send PDF file inputs as Base64-encoded inputs as well.

Base64 encode a file to use in a response

Bash

PDF_BASE64=$(base64 -i draconomicon.pdf) # Use -w 0 on Linux for no line breaks

curl "https://api.avalai.ir/v1/responses" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
 "model": "gpt-5.5",
 "input": [
 {
 "role": "user",
 "content": [
 {
 "type": "input_file",
 "filename": "draconomicon.pdf",
 "file_data": "data:application/pdf;base64,'"$PDF_BASE64"'"
 },
 {
 "type": "input_text",
 "text": "What is the first dragon in the book?"
 }
 ]
 }
 ]
 }'

JavaScript

import fs from "fs";
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir/v1",
});

const data = fs.readFileSync("draconomicon.pdf");
const base64String = data.toString("base64");

const response = await client.responses.create({
  model: "gpt-5.5",
  input: [
    {
      role: "user",
      content: [
        {
          type: "input_file",
          filename: "draconomicon.pdf",
          file_data: `data:application/pdf;base64,${base64String}`,
        },
        {
          type: "input_text",
          text: "What is the first dragon in the book?",
        },
      ],
    },
  ],
});

console.log(response.output_text);

Python

import base64
import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["AVALAI_API_KEY"], base_url="https://api.avalai.ir/v1"
)

with open("draconomicon.pdf", "rb") as f:
    data = f.read()

base64_string = base64.b64encode(data).decode("utf-8")

response = client.responses.create(
    model="gpt-5.5",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_file",
                    "filename": "draconomicon.pdf",
                    "file_data": f"data:application/pdf;base64,{base64_string}",
                },
                {
                    "type": "input_text",
                    "text": "What is the first dragon in the book?",
                },
            ],
        },
    ],
)

print(response.output_text)

Usage considerations

Below are a few considerations to keep in mind while using PDF inputs.

Token usage

Vision-capable Responses routes can place both extracted text and an image of each PDF page into the model context, even when a page is mostly text. Before deploying at scale, test representative documents, log usage.input_tokens, and review the pricing implications of using PDFs as input. More on pricing.

File size limitations

For Responses-style file inputs, keep each file under 50 MB and keep the combined file payload in a single request under 50 MB. Some upstream providers may enforce stricter limits for specific models or endpoints; split large PDFs or use retrieval when you hit those limits.

Supported models

PDF parsing with page images requires a model that supports both text and image input, such as current OpenAI vision-capable models. Check model features here.

File upload purpose

You can upload these files to the Files API with any purpose, but we recommend using the user_data purpose for files you plan to use as model inputs.

Accuracy guardrails

Ask for page numbers, visible quotes, table names, or section headings when the answer must be audited. For contracts, invoices, medical documents, and financial reports, verify sampled pages outside the model before acting on extracted values.

Related Resources

• Files API Reference
• Responses API Reference
• Vision Guide
• Pricing

OCR Processing with Mistral OCR 4

AvalAI supports mistral-ocr-4-0 for layout-aware OCR and document understanding. It extracts Markdown together with bounding boxes, typed block classifications, and confidence information across 170 languages. mistral-ocr-latest now resolves to mistral-ocr-4-0 and uses OCR 4 pricing; use the versioned ID when reproducibility matters.

Warning

Note: Mistral OCR accepts valid HTTP/HTTPS document URLs and Base64-encoded document data. Use v1/files only when the selected AvalAI route supports file IDs for that workflow; otherwise pass a URL or data URL directly to the OCR endpoint.

Key Features

• Extracts text while preserving document structure and hierarchy as Markdown
• Returns bounding boxes for highlighting, citations, and redaction workflows
• Classifies blocks such as titles, tables, equations, and signatures
• Provides confidence information for validation and human review
• Handles complex layouts including multi-column text and mixed content
• Supports 170 languages across 10 language groups
• Costs $0.004 per OCR page and $0.005 per annotated page

OCR with PDF URL

You can process a PDF document by providing its URL:

Python

import os
from mistralai import Mistral

client = Mistral(
    server_url="https://api.avalai.ir", api_key=os.environ["AVALAI_API_KEY"]
)

document_param = {
    "type": "document_url",
    "document_url": "https://arxiv.org/pdf/1805.04770",
}

ocr_response = client.ocr.process(
    model="mistral-ocr-4-0",
    document=document_param,
    pages=list(range(0, 100)),  # Process up to 100 pages
)

print(ocr_response)

JavaScript

import { Mistral } from "mistralai";

const client = new Mistral({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir",
});

const documentParam = {
  type: "document_url",
  document_url: "https://arxiv.org/pdf/1805.04770",
};

const ocrResponse = await client.ocr.process({
  model: "mistral-ocr-4-0",
  document: documentParam,
  pages: Array.from({ length: 100 }, (_, i) => i), // Process up to 100 pages
});

console.log(ocrResponse);

Bash

curl https://api.avalai.ir/v1/ocr \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
"model": "mistral-ocr-4-0",
"document": {
"type": "document_url",
"document_url": "https://arxiv.org/pdf/1805.04770"
},
"include_image_base64": true
}' -o ocr_output.json

OCR with Base64-encoded PDF

When you do not need reusable file storage, use Base64 encoding to process a local PDF directly:

Python

import base64
import os
from mistralai import Mistral

# Read and encode the PDF file
with open("document.pdf", "rb") as f:
    pdf_data = f.read()

base64_pdf = base64.b64encode(pdf_data).decode("utf-8")
document_url = f"data:application/pdf;base64,{base64_pdf}"

# Process the encoded PDF
client = Mistral(
    server_url="https://api.avalai.ir", api_key=os.environ["AVALAI_API_KEY"]
)

document_param = {"type": "document_url", "document_url": document_url}

ocr_response = client.ocr.process(
    model="mistral-ocr-4-0",
    document=document_param,
    pages=list(range(0, 100)),  # Process up to 100 pages
)

print(ocr_response)

JavaScript

import fs from "fs";
import { Mistral } from "mistralai";

// Read and encode the PDF file
const pdfData = fs.readFileSync("document.pdf");
const base64Pdf = pdfData.toString("base64");
const documentUrl = `data:application/pdf;base64,${base64Pdf}`;

// Process the encoded PDF
const client = new Mistral({
  apiKey: process.env.AVALAI_API_KEY,
  baseURL: "https://api.avalai.ir",
});

const documentParam = {
  type: "document_url",
  document_url: documentUrl,
};

const ocrResponse = await client.ocr.process({
  model: "mistral-ocr-4-0",
  document: documentParam,
  pages: Array.from({ length: 100 }, (_, i) => i),
});

console.log(ocrResponse);

Bash

# Convert PDF to base64
PDF_BASE64=$(base64 -i document.pdf) # Use -w 0 on Linux for no line breaks

# Process the encoded PDF
curl https://api.avalai.ir/v1/ocr \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
"model": "mistral-ocr-4-0",
"document": {
"type": "document_url",
"document_url": "data:application/pdf;base64,'"$PDF_BASE64"'"
},
"include_image_base64": true
}' -o ocr_output.json

OCR with Images

Mistral OCR can process images in two ways:

Using a direct image URL:

curl https://api.avalai.ir/v1/ocr \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
"model": "mistral-ocr-4-0",
"document": {
"type": "image_url",
"image_url": "https://raw.githubusercontent.com/mistralai/cookbook/refs/heads/main/mistral/ocr/receipt.png"
}
}' -o ocr_output.json

Using base64-encoded images:

Python

import base64
import os
from mistralai import Mistral

# Read and encode the image file
with open("document.jpg", "rb") as f:
    image_data = f.read()

base64_image = base64.b64encode(image_data).decode("utf-8")
image_url = f"data:image/jpeg;base64,{base64_image}"

# Process the encoded image
client = Mistral(
    server_url="https://api.avalai.ir", api_key=os.environ["AVALAI_API_KEY"]
)

document_param = {"type": "image_url", "image_url": image_url}

ocr_response = client.ocr.process(model="mistral-ocr-4-0", document=document_param)

print(ocr_response)

Bash

# Convert image to base64
IMAGE_BASE64=$(base64 -i document.jpg) # Use -w 0 on Linux for no line breaks

# Process the encoded image
curl https://api.avalai.ir/v1/ocr \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
"model": "mistral-ocr-4-0",
"document": {
"type": "image_url",
"image_url": "data:image/jpeg;base64,'"$IMAGE_BASE64"'"
}
}' -o ocr_output.json

Example Output

The OCR API returns both the extracted text content in markdown format and metadata about the document structure:

{
  "pages": [
    {
      "index": 1,
      "markdown": "# LEVERAGING UNLABELED DATA TO PREDICT OUT-OF-DISTRIBUTION PERFORMANCE \n\nSaurabh Garg*<br>Carnegie Mellon University<br>sgarg2@andrew.cmu.edu<br>Sivaraman Balakrishnan<br>Carnegie Mellon University<br>sbalakri@andrew.cmu.edu<br>Zachary C. Lipton<br>Carnegie Mellon University<br>zlipton@andrew.cmu.edu\n\n## Behnam Neyshabur\n\nGoogle Research, Blueshift team\nneyshabur@google.com\n\nHanie Sedghi<br>Google Research, Brain team<br>hsedghi@google.com\n\n\n#### Abstract\n\nReal-world machine learning deployments are characterized by mismatches between the source (training) and target (test) distributions that may cause performance drops...",
      "images": [],
      "dimensions": {
        "dpi": 200,
        "height": 2200,
        "width": 1700
      }
    }
    // Additional pages...

  ],
  "model": "mistral-ocr-4-0",
  "usage_info": {
    "pages_processed": 3,
    "doc_size_bytes": null
  }
}

Document Understanding

You can combine Mistral OCR with language models to enable natural language interaction with document content. This allows you to extract information and insights from documents by asking questions in natural language:

curl https://api.avalai.ir/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '{
  "model": "mistral-small-latest",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "what is the last sentence in the document"
        },
        {
          "type": "document_url",
          "document_url": "https://arxiv.org/pdf/1805.04770"
        }
      ]
    }
  ]
}'

Responses API version This version uses `gpt-5.5` because `mistral-small-latest` may not be enabled for `/v1/responses` in the current AvalAI model data.

Use this version when the selected model supports /v1/responses. messages moves to input, and the final text is read from response.output_text.

curl https://api.avalai.ir/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $AVALAI_API_KEY" \
  -d '
  {
    "model": "gpt-5.5",
    "input": [
      {
        "role": "user",
        "content": [
          {
            "type": "input_text",
            "text": "Summarize the uploaded file."
          },
          {
            "type": "input_file",
            "file_id": "file_abc123"
          }
        ]
      }
    ]
  }'

• messages → input
• system message → instructions or a developer item
• choices[0].message.content → response.output_text
• for tools and multimodal output, inspect response.output by item type.

Usage Considerations

• File Size Limits: Uploaded document files must not exceed 50 MB in size and should be no longer than 1,000 pages.
• Supported Image Formats: PNG (.png), JPEG (.jpeg and .jpg), WEBP (.webp), and non-animated GIF with only one frame (.gif).
• Pricing: mistral-ocr-4-0 costs $0.004 per OCR page. Document or image annotation costs $0.005 per annotated page. The mistral-ocr-latest alias uses the same OCR 4 pricing.

Related Resources

• Mistral AI Models
• OCR 4 Announcement