Hermes Agent models and providers

Choose a Hermes Agent model and provider

Hermes Agent ships no model. It calls whichever provider you configure, so this choice determines quality, cost, and data flow all at once.

The decision in three steps:

1. Authentication. Pick something you can log into or hold a key for today. One working provider beats three half-configured ones.
2. Capability. The model must handle tool calling and at least 64,000 tokens of context. Below that, agent runs fail in confusing ways.
3. Data flow. A hosted API receives your prompts and tool output. A local endpoint does not leave your machine.

Provider categories and requirements

Four categories, and the difference between them is what you must supply.

• Aggregated subscription. Nous Portal authenticates through OAuth and reaches 300+ models plus a hosted tool gateway, with no per-provider keys to manage. hermes setup --portal wires it in one command.
• Direct vendor APIs. Anthropic, OpenAI, Google Gemini, xAI, DeepSeek, Mistral and others take an API key in ~/.hermes/.env or an OAuth login. You manage each key separately.
• Routers and gateways. OpenRouter and similar services put one key in front of many models, with their own routing and fallback behaviour.
• Local and self-hosted endpoints. Ollama, vLLM, SGLang, llama.cpp, and LM Studio are configured as a custom provider with a base_url. Nothing leaves your machine, and you own the context and tool-calling flags.

Cloud platform paths — AWS Bedrock, Google Vertex AI, Azure AI Foundry — sit inside the direct-vendor category but authenticate through their platform's own credential chain.

Compare by task quality, tool use, context, cost, and privacy

Configure provider credentials

Keys belong in ~/.hermes/.env, never in config.yaml. hermes config set routes recognised secret keys there automatically:

Documented environment-variable names differ per provider — for example OPENROUTER_API_KEY, ANTHROPIC_API_KEY, OPENAI_API_KEY, GOOGLE_API_KEY or GEMINI_API_KEY, DEEPSEEK_API_KEY, XAI_API_KEY. OAuth-based providers store their credentials in ~/.hermes/auth.json instead.

Restrict the file, and treat key rotation as routine. How those credentials are then exposed to containers, MCP subprocesses, and skills is covered under provider credential security.

Select and switch models

• Persistently: set model.provider and model.name in config.yaml, or re-run hermes model.
• For one run: pass the model on the command line.
• Mid-session: the /model slash command switches between providers you have already configured. It cannot add a new one — exit and run hermes model for that.

Two cost-relevant behaviours are worth knowing. Switching model inside a live conversation means the next message re-reads the whole conversation at full input price. And auxiliary tasks can be pointed at a cheaper model under the auxiliary: block rather than paying main-model rates for title generation and compression.

Which of those choices actually moves your bill is covered by provider cost components.

Use local or custom endpoints

Any OpenAI-compatible endpoint works through the custom provider:

model:
  provider: custom
  base_url: http://localhost:8000/v1
  name: <model-id>
  context_length: 64000

The two failure modes are always the same, and both are on the server side:

• Context too small. Ollama in particular defaults well below the requirement; the documented fix is to start it with OLLAMA_CONTEXT_LENGTH=64000, or set the equivalent flag for your server (--max-model-len on vLLM, --context-length on SGLang, -c 64000 on llama.cpp).
• Tool calling not enabled. vLLM needs --enable-auto-tool-choice with a --tool-call-parser; llama.cpp needs --jinja. Without those, the agent looks broken while the model is simply answering in prose.

Set fallbacks and limits

A single provider outage should not end a run. Fallback providers are declared as an ordered list and tried when the primary model errors:

fallback_providers:
  - provider: openrouter
    model: <model-id>
  - provider: anthropic
    model: <model-id>

Related controls: providers.<name>.request_timeout_seconds for slow endpoints, agent.api_max_retries for how many attempts happen before falling back, and agent.run_budget_seconds to cap a run by wall-clock time.

Verify model and tool behavior

A greeting proves nothing. Verify with all four:

• hermes config get model returns the provider and model you intended.
• A one-shot prompt returns a response without an authentication error.
• A task that requires a tool actually shows a tool call in the interface.
• A long input does not immediately fail with a context error, which would mean the context limit is lower than the agent needs.

The end-to-end version of this check — a bounded task with a checkable artifact — is owned by verify first model response.

Fix provider errors