Connect Hermes Agent to AvalAI

Hermes Agent can use AvalAI as a named OpenAI-compatible model provider while keeping its tools, sessions, gateways, and auxiliary services under separate control. This guide starts with the safest broad-compatibility path, verifies plain chat before tools, and then explains which capabilities need another route or configuration.

Validated: The configuration and upstream labels on this page were checked against public official sources on 2026-08-06. Model availability can change; confirm the selected model in the AvalAI model catalog before setup.

What this integration is

The request flow is:

You → Hermes agent loop and tools → AvalAI OpenAI-compatible endpoint → selected model

AvalAI authenticates model requests. Hermes independently controls what the agent can read, write, execute, reach over the network, and expose through a gateway. A valid API key does not make an unsafe tool or filesystem policy safe.

Use this integration when you want an autonomous terminal or messaging agent with resumable sessions and a current AvalAI model. Use a simpler chat client when you do not need agent tools or long-running sessions.

What you need

• Linux, macOS, or Windows through WSL2, following the current Hermes installation requirements.
• A recent Hermes Agent release.
• An AvalAI API key stored outside source control.
• A model with at least 64,000 input tokens. Hermes rejects smaller contexts for agent use with tools.
• A model that supports every capability you intend to test. In this guide, gpt-5.4-mini is the pilot because the current catalog lists Chat Completions, Responses, streaming, tools, structured output, vision, and 272,000 input tokens.

Create a separate AvalAI key for Hermes so you can revoke it and attribute usage without affecting other applications.

Preflight AvalAI

Confirm these values before changing Hermes:

Check	Required value
OpenAI-compatible base URL	https://api.avalai.ir/v1
Broad compatibility route	/v1/chat/completions
Optional Responses route	/v1/responses
Pilot model	gpt-5.4-mini
Minimum Hermes context	64,000 input tokens

The model ID, route, and capabilities are separate checks. A model listed by /v1/models may not support every endpoint or tool behavior.

AvalAI does not currently support the gpt-transcribe or gpt-live-transcribe model identifiers. Do not paste upstream examples using those IDs into Hermes.

Install Hermes

Follow the current Hermes installation guide. After installation, confirm that the CLI and configuration directory are available:

hermes --version
hermes doctor

Do not put provider secrets in a repository-local shell script. Hermes stores secrets in ~/.hermes/.env and non-secret settings in ~/.hermes/config.yaml.

Connect AvalAI

Recommended interactive setup

Run hermes model outside an active chat session, then choose Custom endpoint (self-hosted / VLLM / etc.). Enter:

Hermes field	Value
API base URL	https://api.avalai.ir/v1
API key	Your dedicated AvalAI key
Model	gpt-5.4-mini
API mode / transport	chat_completions
Context length	272000

The wizard persists the provider configuration. Use chat_completions first because it is the broad compatibility path.

Auditable named-provider configuration

Store only the secret in ~/.hermes/.env:

AVALAI_API_KEY=replace-with-your-avalai-key

Then define providers.avalai in ~/.hermes/config.yaml:

providers:
  avalai:
    api: https://api.avalai.ir/v1
    key_env: AVALAI_API_KEY
    transport: chat_completions
    default_model: gpt-5.4-mini
    models:
      gpt-5.4-mini:
        context_length: 272000
        supports_vision: true

key_env tells Hermes which environment entry contains the secret. Avoid an inline api_key in config.yaml, backups, screenshots, or shared diagnostics.

Optional Responses transport

Use a separate named provider when you intentionally want Hermes to send Responses requests. Change the transport only after confirming the model lists /v1/responses in the current AvalAI catalog:

providers:
  avalai-responses:
    api: https://api.avalai.ir/v1
    key_env: AVALAI_API_KEY
    transport: codex_responses
    default_model: gpt-5.4-mini
    models:
      gpt-5.4-mini:
        context_length: 272000
        supports_vision: true

codex_responses is a Hermes transport name; it does not mean every model or OpenAI-compatible server implements Responses.

Verify the first flow

Verify one layer at a time:

1. Run the diagnostic and resolve configuration errors first.
2. Start Hermes and confirm the startup banner shows the intended provider and model.
3. Send a plain prompt, then a follow-up that depends on the first answer.
4. If the selected model supports function calling, ask for one safe, read-only tool action such as listing the current directory. Do not begin with a write or shell mutation.
5. Exit and resume the session.
6. Test in-session switching only after the default path works.

hermes doctor
hermes
hermes --continue

Inside a running session, /model custom:avalai:gpt-5.4-mini selects the named provider. Use terminal-level hermes model to add or change providers; /model only switches among providers that are already configured.

Supported capabilities

The status describes the Hermes-to-AvalAI path, not a promise about every model.

Status	Capability	What to expect
Direct	Chat Completions	The custom provider can send /v1/chat/completions requests. Start here.
Model/route dependent	Responses	Use codex_responses only with a model verified for /v1/responses.
Model/route dependent	Streaming	Hermes and the selected model must both preserve the expected stream events.
Model/route dependent	System messages and sampling	Unsupported parameters can still be rejected or ignored by a model route.
Model/route dependent	Tools and structured output	Requires model function calling, compatible tool schemas, and Hermes tool permission.
Model/route dependent	Vision/image input	Set supports_vision: true only for a catalog-verified vision model.
Unsupported or unvalidated	Embeddings and RAG through the main provider	The main Hermes inference provider is not an embeddings configuration surface.
Separate configuration	Image generation	Hermes image tools or Tool Gateway use their own backend and credentials.
Separate configuration	Speech-to-text and text-to-speech	Voice and transcription tools are auxiliary services, not implied by the chat provider.
Separate configuration	Web search and browser automation	Hermes tool backends and permissions remain separate from AvalAI model authentication.
Unsupported or unvalidated	Realtime audio and video	No source-backed mapping from the named model provider was validated for these routes.

Auxiliary models may default to the main provider, but vision analysis, web summarization, image generation, voice, browser automation, memory services, and messaging gateways can have separate configuration and billing. Check their active provider before assuming an AvalAI key covers them.

Run with Docker Compose

Hermes publishes an official Dockerfile and docker-compose.yml. Use those assets from a reviewed tag instead of copying an unofficial image definition. The validated tag for this guide is v2026.8.3:

git clone https://github.com/NousResearch/hermes-agent.git
cd hermes-agent
git checkout v2026.8.3
HERMES_UID="$(id -u)" HERMES_GID="$(id -g)" docker compose up -d --build
docker compose exec gateway hermes doctor
docker compose logs --tail=100 gateway dashboard

The upstream Compose file:

• builds the reviewed source into the local hermes-agent image;
• mounts host ~/.hermes at /opt/data for durable configuration and sessions;
• maps the container service user to HERMES_UID and HERMES_GID;
• binds the dashboard to 127.0.0.1 through host networking;
• leaves the OpenAI-compatible API server off unless both API_SERVER_HOST and API_SERVER_KEY are configured.

Do not expose the dashboard with --insecure --host 0.0.0.0. For remote administration, prefer an SSH tunnel or an existing authenticated TLS ingress. Do not enable a messaging gateway until its user allowlist is explicit.

Operate safely

• Mount only workspaces the agent must access. Hermes can execute tools and modify files independently of model authentication.
• Keep ~/.hermes/.env, auth stores, session data, and backups readable only by the service owner.
• Keep the dashboard and gateway private. If you enable the API server, generate a separate strong API_SERVER_KEY.
• Review terminal, browser, network, and messaging permissions separately. Disable tools you do not need.
• Redact authorization headers, prompts, responses, file contents, and personal data from diagnostic bundles.
• Watch AvalAI usage and rate limits; retries and auxiliary model calls can add requests beyond the visible turn.

Before an upgrade, stop the services, back up the complete Hermes data directory, and record the current source revision:

docker compose stop
git rev-parse HEAD
tar -czf hermes-data-backup.tgz -C "$HOME" .hermes
docker compose start

To upgrade, check out a reviewed tag and rebuild. To roll back, stop the services, return to the recorded revision, rebuild, and restore the backup only if a data migration requires it. Test restoration on a separate copy before relying on the backup.

Troubleshooting

Symptom	Check this layer first	Safe next action
401 or 403	AVALAI_API_KEY, file ownership, or account access	Re-run hermes doctor; verify key_env without printing the secret.
404	Base URL or transport	Keep /v1; use chat_completions unless the model supports Responses.
Model not found	Exact ID and selected named provider	Recheck /v1/models and /model custom:avalai:....
Context error at startup	Context metadata below 64K	Select a larger-context model or correct verified context_length.
Tool call appears as text	Model/tool schema compatibility	Confirm function calling and reduce to one read-only tool.
Plain chat works but vision fails	supports_vision, input shape, auxiliary routing	Verify the model catalog and active vision provider.
Empty or broken stream	Chat-vs-Responses mismatch or proxy buffering	Return to non-streaming plain chat, then re-enable one layer at a time.
Session does not resume	Data path or volume ownership	Confirm ~/.hermes is mounted at /opt/data and writable by the mapped UID.
Container-created files are root-owned	UID/GID mapping or bypassed entrypoint	Use the upstream Compose flow and do not replace its /init chain.

Related AvalAI guides and official sources

AvalAI:

• API introduction
• Model catalog
• Model selection
• Function calling
• Vision
• Rate limits
• Production best practices

Hermes Agent:

• Official repository
• Quickstart
• Installation
• AI providers and custom endpoints

Validation boundary

This guide was source-reviewed on 2026-08-06. Its Markdown, configuration parity, links, and static-site behavior can be validated locally. This work did not use an AvalAI credential, install Hermes, build or start the containers, send a paid request, expose a gateway, restore a backup, or test a production network. Verify the current upstream release and model catalog before applying the commands to an operational system.