LLM Provider Configuration

Overview

OrcBot supports multiple LLM providers with automatic fallback and provider-specific model persistence. You can configure providers via environment variables or YAML configuration files.

Quick Start

llmProvider: openai
modelName: gpt-4o

providerModelNames:
  openai: gpt-4o
  google: gemini-2.0-flash-exp
  anthropic: claude-3-5-sonnet-20241022

OPENAI_API_KEY=sk-your-key-here
GOOGLE_API_KEY=your-google-key
ANTHROPIC_API_KEY=sk-ant-your-key

Provider Selection

enum

Explicit LLM provider selection. Supported values:

openai - OpenAI (GPT-4, GPT-3.5, o1)
google - Google Gemini
anthropic - Anthropic Claude
bedrock - AWS Bedrock
openrouter - OpenRouter (200+ models)
nvidia - NVIDIA NIM
ollama - Local Ollama models

The provider can also be inferred from the model name prefix (e.g., claude- → anthropic).

string

default:"gpt-4o"

Primary model to use. The provider is auto-detected from the model name if not explicitly set.

object

Per-provider model preferences. When you switch providers, OrcBot automatically restores the last model used for that provider.

providerModelNames:
  openai: gpt-4o
  google: gemini-2.0-flash-exp
  anthropic: claude-3-5-sonnet-20241022
  openrouter: anthropic/claude-3.5-sonnet

object

Fallback models when primary provider fails:

fallbackModelNames:
  openai: gpt-4o-mini
  google: gemini-1.5-flash

string

Fast model for lightweight operations like summarization and classification.Examples:

gpt-4o-mini
gemini-1.5-flash
claude-3-5-haiku-20241022

OpenAI

Configuration

string

required

OpenAI API key (starts with sk-).Policy: APPROVAL - API keys are sensitive and require approval for changes.Environment variable: OPENAI_API_KEY

Supported Models

GPT-4o (Recommended)

Model ID: gpt-4o
Context: 128K tokens
Strengths: Best balance of intelligence, speed, and cost
Use cases: General tasks, coding, reasoning

llmProvider: openai
modelName: gpt-4o

GPT-4o Mini

Model ID: gpt-4o-mini
Context: 128K tokens
Strengths: Fast and affordable
Use cases: Quick responses, summarization, classification

llmProvider: openai
modelName: gpt-4o-mini
fastModelName: gpt-4o-mini

GPT-4 Turbo

Model ID: gpt-4-turbo
Context: 128K tokens
Strengths: Previous generation flagship
Use cases: Complex reasoning, analysis

llmProvider: openai
modelName: gpt-4-turbo

o1 Series (Reasoning)

Model IDs: o1, o1-mini, o1-preview
Context: 128K tokens (o1), 128K tokens (o1-mini)
Strengths: Advanced reasoning, complex problem-solving
Limitations: No function calling, streaming only
Use cases: Deep analysis, strategic planning

o1 models do not support tool calling. Use with deep_reason skill only.

llmProvider: openai
modelName: o1

Example Configuration

orcbot.config.yaml

# OpenAI Configuration
llmProvider: openai
modelName: gpt-4o
fastModelName: gpt-4o-mini
openaiApiKey: sk-your-key-here  # Or set via environment variable

# Provider-specific models
providerModelNames:
  openai: gpt-4o

fallbackModelNames:
  openai: gpt-4o-mini

Google Gemini

Configuration

string

required

Google AI API key.Policy: APPROVAL - API keys are sensitive and require approval for changes.Environment variable: GOOGLE_API_KEYGet your API key at: https://makersuite.google.com/app/apikey

Supported Models

Gemini 2.0 Flash (Recommended)

Model ID: gemini-2.0-flash-exp
Context: 1M tokens
Strengths: Multimodal, fast, large context
Use cases: General tasks, image analysis, long documents

llmProvider: google
modelName: gemini-2.0-flash-exp

Gemini 1.5 Pro

Model ID: gemini-1.5-pro
Context: 2M tokens
Strengths: Massive context window, high intelligence
Use cases: Extremely long documents, complex analysis

llmProvider: google
modelName: gemini-1.5-pro

Gemini 1.5 Flash

Model ID: gemini-1.5-flash
Context: 1M tokens
Strengths: Fast, affordable, good quality
Use cases: Quick responses, summarization

llmProvider: google
modelName: gemini-1.5-flash

Computer Use (Experimental)

boolean

default:false

Enable Gemini’s computer use capabilities for screen control.

string

default:"gemini-2.5-computer-use-preview-10-2025"

Model to use for computer use features.

Example Configuration

orcbot.config.yaml

# Google Gemini Configuration
llmProvider: google
modelName: gemini-2.0-flash-exp
fastModelName: gemini-1.5-flash
googleApiKey: your-google-key  # Or set via environment variable

# Provider-specific models
providerModelNames:
  google: gemini-2.0-flash-exp

fallbackModelNames:
  google: gemini-1.5-flash

Anthropic Claude

Configuration

string

required

Anthropic API key (starts with sk-ant-).Policy: APPROVAL - API keys are sensitive and require approval for changes.Environment variable: ANTHROPIC_API_KEY

Supported Models

Claude 3.5 Sonnet (Recommended)

Model ID: claude-3-5-sonnet-20241022
Context: 200K tokens
Strengths: Best reasoning, coding, analysis
Use cases: Complex tasks, code generation, writing

llmProvider: anthropic
modelName: claude-3-5-sonnet-20241022

Claude 3.5 Haiku

Model ID: claude-3-5-haiku-20241022
Context: 200K tokens
Strengths: Fast, affordable, intelligent
Use cases: Quick responses, summarization

llmProvider: anthropic
modelName: claude-3-5-haiku-20241022

Claude 3 Opus

Model ID: claude-3-opus-20240229
Context: 200K tokens
Strengths: Highest intelligence, complex reasoning
Use cases: Most demanding tasks, strategic planning

llmProvider: anthropic
modelName: claude-3-opus-20240229

Example Configuration

orcbot.config.yaml

# Anthropic Claude Configuration
llmProvider: anthropic
modelName: claude-3-5-sonnet-20241022
fastModelName: claude-3-5-haiku-20241022
anthropicApiKey: sk-ant-your-key  # Or set via environment variable

# Provider-specific models
providerModelNames:
  anthropic: claude-3-5-sonnet-20241022

fallbackModelNames:
  anthropic: claude-3-5-haiku-20241022

AWS Bedrock

Configuration

string

AWS region for Bedrock service (e.g., us-east-1, us-west-2).Environment variable: BEDROCK_REGION or AWS_REGION

string

AWS access key ID.Policy: LOCKED - Critical authentication credential.Environment variable: BEDROCK_ACCESS_KEY_ID or AWS_ACCESS_KEY_ID

string

AWS secret access key.Policy: LOCKED - Critical authentication credential.Environment variable: BEDROCK_SECRET_ACCESS_KEY or AWS_SECRET_ACCESS_KEY

string

AWS session token (for temporary credentials).Environment variable: BEDROCK_SESSION_TOKEN or AWS_SESSION_TOKEN

Supported Models

Bedrock provides access to multiple model families:

Anthropic Claude: anthropic.claude-3-sonnet-20240229-v1:0
Amazon Titan: amazon.titan-text-premier-v1:0
Meta Llama: meta.llama3-70b-instruct-v1:0
Cohere Command: cohere.command-r-plus-v1:0
Mistral AI: mistral.mistral-large-2407-v1:0

Example Configuration

orcbot.config.yaml

# AWS Bedrock Configuration
llmProvider: bedrock
modelName: anthropic.claude-3-5-sonnet-20241022-v2:0
bedrockRegion: us-east-1

# Credentials via environment variables recommended
# bedrockAccessKeyId: AKIA...
# bedrockSecretAccessKey: ...

.env

BEDROCK_REGION=us-east-1
BEDROCK_ACCESS_KEY_ID=AKIA...
BEDROCK_SECRET_ACCESS_KEY=...

OpenRouter

Configuration

string

required

OpenRouter API key.Policy: APPROVAL - API keys are sensitive and require approval for changes.Environment variable: OPENROUTER_API_KEYGet your API key at: https://openrouter.ai/keys

string

default:"https://openrouter.ai/api/v1"

OpenRouter API base URL.Environment variable: OPENROUTER_BASE_URL

string

HTTP Referer header for OpenRouter requests (optional, for ranking).Environment variable: OPENROUTER_REFERER

string

Application name for OpenRouter (optional, for ranking).Environment variable: OPENROUTER_APP_NAME

Supported Models

OpenRouter provides access to 200+ models from multiple providers:

Popular Models

anthropic/claude-3.5-sonnet - Claude 3.5 Sonnet
openai/gpt-4o - GPT-4o
google/gemini-2.0-flash-exp:free - Gemini 2.0 Flash (free)
meta-llama/llama-3.3-70b-instruct - Llama 3.3 70B
mistralai/mistral-large - Mistral Large
deepseek/deepseek-chat - DeepSeek V3

Free Models

OpenRouter offers several free models:

google/gemini-2.0-flash-exp:free
meta-llama/llama-3.2-3b-instruct:free
microsoft/phi-3-mini-128k-instruct:free

Free models have rate limits and may have lower availability.

Example Configuration

orcbot.config.yaml

# OpenRouter Configuration
llmProvider: openrouter
modelName: anthropic/claude-3.5-sonnet
openrouterApiKey: sk-or-your-key
openrouterReferer: https://github.com/yourusername/orcbot
openrouterAppName: MyOrcBot

# Provider-specific models
providerModelNames:
  openrouter: anthropic/claude-3.5-sonnet

fallbackModelNames:
  openrouter: google/gemini-2.0-flash-exp:free

Browse all models: https://openrouter.ai/docs#models

Ollama (Local Models)

Configuration

boolean

default:false

Enable local Ollama integration for running models locally.

string

default:"http://localhost:11434"

Ollama API endpoint URL.Policy: SAFE - Non-sensitive endpoint configuration.

Supported Models

Ollama supports running models locally:

Llama 3.3: llama3.3:70b, llama3.3:8b
Qwen 2.5: qwen2.5:72b, qwen2.5:32b
Mistral: mistral:7b, mistral-nemo:12b
DeepSeek: deepseek-r1:70b, deepseek-r1:8b
Phi-4: phi4:14b
Gemma 2: gemma2:27b, gemma2:9b

TUI Management

OrcBot’s TUI provides built-in Ollama management:

orcbot ui

Auto-start server - Automatically starts Ollama if not running
Pull models - Download models from the Ollama library
Model selection - Switch between installed models
OpenAI-compatible - Native tool calling support

Example Configuration

orcbot.config.yaml

# Ollama Local Configuration
llmProvider: ollama
modelName: llama3.3:70b
ollamaEnabled: true
ollamaApiUrl: http://localhost:11434

# Provider-specific models
providerModelNames:
  ollama: llama3.3:70b

fallbackModelNames:
  ollama: llama3.3:8b

Installation

Install Ollama

Download from https://ollama.com or install via package manager:

# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

Pull a model

ollama pull llama3.3:70b

Start Ollama service

ollama serve

Configure OrcBot

ollamaEnabled: true
llmProvider: ollama
modelName: llama3.3:70b

Additional Providers

OrcBot also supports these providers via direct API integration:

NVIDIA NIM

string

NVIDIA NIM API key.Policy: APPROVAL - API keys are sensitive.Environment variable: NVIDIA_API_KEY

llmProvider: nvidia
modelName: meta/llama-3.1-405b-instruct
nvidiaApiKey: nvapi-your-key

Groq

string

Groq API key for ultra-fast inference.Environment variable: GROQ_API_KEY

modelName: groq/llama-3.3-70b
groqApiKey: gsk_your-key

Mistral AI

string

Mistral AI API key.Environment variable: MISTRAL_API_KEY

modelName: mistral-large-latest
mistralApiKey: your-mistral-key

Cerebras

string

Cerebras API key for fastest inference.Environment variable: CEREBRAS_API_KEY

modelName: cerebras/llama-3.3-70b
cerebrasApiKey: your-cerebras-key

xAI

string

xAI (Grok) API key.Environment variable: XAI_API_KEY

modelName: grok-2
xaiApiKey: your-xai-key

Multi-Provider Strategy

Configure multiple providers for automatic fallback:

orcbot.config.yaml

# Primary provider
llmProvider: openai
modelName: gpt-4o

# Per-provider models (auto-restored on provider switch)
providerModelNames:
  openai: gpt-4o
  google: gemini-2.0-flash-exp
  anthropic: claude-3-5-sonnet-20241022
  openrouter: anthropic/claude-3.5-sonnet
  ollama: llama3.3:70b

# Fallback models
fallbackModelNames:
  openai: gpt-4o-mini
  google: gemini-1.5-flash
  anthropic: claude-3-5-haiku-20241022

# Fast model for lightweight tasks
fastModelName: gpt-4o-mini

When a provider fails, OrcBot automatically tries the fallback model, then switches to an alternative provider if configured.

Provider Auto-Detection

You can omit llmProvider and OrcBot will infer it from the model name:

# Auto-detects provider from model prefix
modelName: claude-3-5-sonnet-20241022  # → anthropic
modelName: gpt-4o                      # → openai
modelName: gemini-2.0-flash-exp        # → google
modelName: llama3.3:70b                # → ollama

Best Practices

Use Environment Variables

Store API keys in .env files or environment variables, not in YAML configuration.

Configure Fallbacks

Set fallback models to ensure continuity when primary provider fails.

Choose Fast Models

Configure fastModelName for lightweight operations to reduce latency and cost.

Test Locally First

Use Ollama for development and testing before using paid APIs.

​Overview

​Quick Start

​Provider Selection

​OpenAI

​Configuration

​Supported Models

​Example Configuration

​Google Gemini

​Configuration

​Supported Models

​Computer Use (Experimental)

​Example Configuration

​Anthropic Claude

​Configuration

​Supported Models

​Example Configuration

​AWS Bedrock

​Configuration

​Supported Models

​Example Configuration

​OpenRouter

​Configuration

​Supported Models

​Example Configuration

​Ollama (Local Models)

​Configuration

​Supported Models

​TUI Management

​Example Configuration

​Installation

​Additional Providers

​NVIDIA NIM

​Groq

​Mistral AI

​Cerebras

​xAI

​Multi-Provider Strategy

​Provider Auto-Detection

​Best Practices

Use Environment Variables

Configure Fallbacks

Choose Fast Models

Test Locally First

Overview

Quick Start

Provider Selection

OpenAI

Configuration

Supported Models

Example Configuration

Google Gemini

Configuration

Supported Models

Computer Use (Experimental)

Example Configuration

Anthropic Claude

Configuration

Supported Models

Example Configuration

AWS Bedrock

Configuration

Supported Models

Example Configuration

OpenRouter

Configuration

Supported Models

Example Configuration

Ollama (Local Models)

Configuration

Supported Models

TUI Management

Example Configuration

Installation

Additional Providers

NVIDIA NIM

Groq

Mistral AI

Cerebras

xAI

Multi-Provider Strategy

Provider Auto-Detection

Best Practices