> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orcbot.buzzchat.site/llms.txt
> Use this file to discover all available pages before exploring further.

# LLM Provider Configuration

> Configure OpenAI, Google Gemini, AWS Bedrock, OpenRouter, and Ollama

## Overview

OrcBot supports multiple LLM providers with automatic fallback and provider-specific model persistence. You can configure providers via environment variables or YAML configuration files.

## Quick Start

<CodeGroup>
  ```yaml orcbot.config.yaml theme={null}
  llmProvider: openai
  modelName: gpt-4o

  providerModelNames:
    openai: gpt-4o
    google: gemini-2.0-flash-exp
    anthropic: claude-3-5-sonnet-20241022
  ```

  ```bash .env theme={null}
  OPENAI_API_KEY=sk-your-key-here
  GOOGLE_API_KEY=your-google-key
  ANTHROPIC_API_KEY=sk-ant-your-key
  ```
</CodeGroup>

## Provider Selection

<ParamField path="llmProvider" type="enum">
  Explicit LLM provider selection. Supported values:

  * `openai` - OpenAI (GPT-4, GPT-3.5, o1)
  * `google` - Google Gemini
  * `anthropic` - Anthropic Claude
  * `bedrock` - AWS Bedrock
  * `openrouter` - OpenRouter (200+ models)
  * `nvidia` - NVIDIA NIM
  * `ollama` - Local Ollama models

  <Note>
    The provider can also be inferred from the model name prefix (e.g., `claude-` → anthropic).
  </Note>
</ParamField>

<ParamField path="modelName" type="string" default="gpt-4o">
  Primary model to use. The provider is auto-detected from the model name if not explicitly set.
</ParamField>

<ParamField path="providerModelNames" type="object">
  Per-provider model preferences. When you switch providers, OrcBot automatically restores the last model used for that provider.

  ```yaml theme={null}
  providerModelNames:
    openai: gpt-4o
    google: gemini-2.0-flash-exp
    anthropic: claude-3-5-sonnet-20241022
    openrouter: anthropic/claude-3.5-sonnet
  ```
</ParamField>

<ParamField path="fallbackModelNames" type="object">
  Fallback models when primary provider fails:

  ```yaml theme={null}
  fallbackModelNames:
    openai: gpt-4o-mini
    google: gemini-1.5-flash
  ```
</ParamField>

<ParamField path="fastModelName" type="string">
  Fast model for lightweight operations like summarization and classification.

  Examples:

  * `gpt-4o-mini`
  * `gemini-1.5-flash`
  * `claude-3-5-haiku-20241022`
</ParamField>

## OpenAI

### Configuration

<ParamField path="openaiApiKey" type="string" required>
  OpenAI API key (starts with `sk-`).

  **Policy:** APPROVAL - API keys are sensitive and require approval for changes.

  **Environment variable:** `OPENAI_API_KEY`
</ParamField>

### Supported Models

<AccordionGroup>
  <Accordion title="GPT-4o (Recommended)">
    * **Model ID:** `gpt-4o`
    * **Context:** 128K tokens
    * **Strengths:** Best balance of intelligence, speed, and cost
    * **Use cases:** General tasks, coding, reasoning

    ```yaml theme={null}
    llmProvider: openai
    modelName: gpt-4o
    ```
  </Accordion>

  <Accordion title="GPT-4o Mini">
    * **Model ID:** `gpt-4o-mini`
    * **Context:** 128K tokens
    * **Strengths:** Fast and affordable
    * **Use cases:** Quick responses, summarization, classification

    ```yaml theme={null}
    llmProvider: openai
    modelName: gpt-4o-mini
    fastModelName: gpt-4o-mini
    ```
  </Accordion>

  <Accordion title="GPT-4 Turbo">
    * **Model ID:** `gpt-4-turbo`
    * **Context:** 128K tokens
    * **Strengths:** Previous generation flagship
    * **Use cases:** Complex reasoning, analysis

    ```yaml theme={null}
    llmProvider: openai
    modelName: gpt-4-turbo
    ```
  </Accordion>

  <Accordion title="o1 Series (Reasoning)">
    * **Model IDs:** `o1`, `o1-mini`, `o1-preview`
    * **Context:** 128K tokens (o1), 128K tokens (o1-mini)
    * **Strengths:** Advanced reasoning, complex problem-solving
    * **Limitations:** No function calling, streaming only
    * **Use cases:** Deep analysis, strategic planning

    <Warning>
      o1 models do not support tool calling. Use with `deep_reason` skill only.
    </Warning>

    ```yaml theme={null}
    llmProvider: openai
    modelName: o1
    ```
  </Accordion>
</AccordionGroup>

### Example Configuration

```yaml orcbot.config.yaml theme={null}
# OpenAI Configuration
llmProvider: openai
modelName: gpt-4o
fastModelName: gpt-4o-mini
openaiApiKey: sk-your-key-here  # Or set via environment variable

# Provider-specific models
providerModelNames:
  openai: gpt-4o

fallbackModelNames:
  openai: gpt-4o-mini
```

## Google Gemini

### Configuration

<ParamField path="googleApiKey" type="string" required>
  Google AI API key.

  **Policy:** APPROVAL - API keys are sensitive and require approval for changes.

  **Environment variable:** `GOOGLE_API_KEY`

  Get your API key at: [https://makersuite.google.com/app/apikey](https://makersuite.google.com/app/apikey)
</ParamField>

### Supported Models

<AccordionGroup>
  <Accordion title="Gemini 2.0 Flash (Recommended)">
    * **Model ID:** `gemini-2.0-flash-exp`
    * **Context:** 1M tokens
    * **Strengths:** Multimodal, fast, large context
    * **Use cases:** General tasks, image analysis, long documents

    ```yaml theme={null}
    llmProvider: google
    modelName: gemini-2.0-flash-exp
    ```
  </Accordion>

  <Accordion title="Gemini 1.5 Pro">
    * **Model ID:** `gemini-1.5-pro`
    * **Context:** 2M tokens
    * **Strengths:** Massive context window, high intelligence
    * **Use cases:** Extremely long documents, complex analysis

    ```yaml theme={null}
    llmProvider: google
    modelName: gemini-1.5-pro
    ```
  </Accordion>

  <Accordion title="Gemini 1.5 Flash">
    * **Model ID:** `gemini-1.5-flash`
    * **Context:** 1M tokens
    * **Strengths:** Fast, affordable, good quality
    * **Use cases:** Quick responses, summarization

    ```yaml theme={null}
    llmProvider: google
    modelName: gemini-1.5-flash
    ```
  </Accordion>
</AccordionGroup>

### Computer Use (Experimental)

<ParamField path="googleComputerUseEnabled" type="boolean" default={false}>
  Enable Gemini's computer use capabilities for screen control.
</ParamField>

<ParamField path="googleComputerUseModel" type="string" default="gemini-2.5-computer-use-preview-10-2025">
  Model to use for computer use features.
</ParamField>

### Example Configuration

```yaml orcbot.config.yaml theme={null}
# Google Gemini Configuration
llmProvider: google
modelName: gemini-2.0-flash-exp
fastModelName: gemini-1.5-flash
googleApiKey: your-google-key  # Or set via environment variable

# Provider-specific models
providerModelNames:
  google: gemini-2.0-flash-exp

fallbackModelNames:
  google: gemini-1.5-flash
```

## Anthropic Claude

### Configuration

<ParamField path="anthropicApiKey" type="string" required>
  Anthropic API key (starts with `sk-ant-`).

  **Policy:** APPROVAL - API keys are sensitive and require approval for changes.

  **Environment variable:** `ANTHROPIC_API_KEY`
</ParamField>

### Supported Models

<AccordionGroup>
  <Accordion title="Claude 3.5 Sonnet (Recommended)">
    * **Model ID:** `claude-3-5-sonnet-20241022`
    * **Context:** 200K tokens
    * **Strengths:** Best reasoning, coding, analysis
    * **Use cases:** Complex tasks, code generation, writing

    ```yaml theme={null}
    llmProvider: anthropic
    modelName: claude-3-5-sonnet-20241022
    ```
  </Accordion>

  <Accordion title="Claude 3.5 Haiku">
    * **Model ID:** `claude-3-5-haiku-20241022`
    * **Context:** 200K tokens
    * **Strengths:** Fast, affordable, intelligent
    * **Use cases:** Quick responses, summarization

    ```yaml theme={null}
    llmProvider: anthropic
    modelName: claude-3-5-haiku-20241022
    ```
  </Accordion>

  <Accordion title="Claude 3 Opus">
    * **Model ID:** `claude-3-opus-20240229`
    * **Context:** 200K tokens
    * **Strengths:** Highest intelligence, complex reasoning
    * **Use cases:** Most demanding tasks, strategic planning

    ```yaml theme={null}
    llmProvider: anthropic
    modelName: claude-3-opus-20240229
    ```
  </Accordion>
</AccordionGroup>

### Example Configuration

```yaml orcbot.config.yaml theme={null}
# Anthropic Claude Configuration
llmProvider: anthropic
modelName: claude-3-5-sonnet-20241022
fastModelName: claude-3-5-haiku-20241022
anthropicApiKey: sk-ant-your-key  # Or set via environment variable

# Provider-specific models
providerModelNames:
  anthropic: claude-3-5-sonnet-20241022

fallbackModelNames:
  anthropic: claude-3-5-haiku-20241022
```

## AWS Bedrock

### Configuration

<ParamField path="bedrockRegion" type="string">
  AWS region for Bedrock service (e.g., `us-east-1`, `us-west-2`).

  **Environment variable:** `BEDROCK_REGION` or `AWS_REGION`
</ParamField>

<ParamField path="bedrockAccessKeyId" type="string">
  AWS access key ID.

  **Policy:** LOCKED - Critical authentication credential.

  **Environment variable:** `BEDROCK_ACCESS_KEY_ID` or `AWS_ACCESS_KEY_ID`
</ParamField>

<ParamField path="bedrockSecretAccessKey" type="string">
  AWS secret access key.

  **Policy:** LOCKED - Critical authentication credential.

  **Environment variable:** `BEDROCK_SECRET_ACCESS_KEY` or `AWS_SECRET_ACCESS_KEY`
</ParamField>

<ParamField path="bedrockSessionToken" type="string">
  AWS session token (for temporary credentials).

  **Environment variable:** `BEDROCK_SESSION_TOKEN` or `AWS_SESSION_TOKEN`
</ParamField>

### Supported Models

Bedrock provides access to multiple model families:

* **Anthropic Claude:** `anthropic.claude-3-sonnet-20240229-v1:0`
* **Amazon Titan:** `amazon.titan-text-premier-v1:0`
* **Meta Llama:** `meta.llama3-70b-instruct-v1:0`
* **Cohere Command:** `cohere.command-r-plus-v1:0`
* **Mistral AI:** `mistral.mistral-large-2407-v1:0`

### Example Configuration

```yaml orcbot.config.yaml theme={null}
# AWS Bedrock Configuration
llmProvider: bedrock
modelName: anthropic.claude-3-5-sonnet-20241022-v2:0
bedrockRegion: us-east-1

# Credentials via environment variables recommended
# bedrockAccessKeyId: AKIA...
# bedrockSecretAccessKey: ...
```

```bash .env theme={null}
BEDROCK_REGION=us-east-1
BEDROCK_ACCESS_KEY_ID=AKIA...
BEDROCK_SECRET_ACCESS_KEY=...
```

## OpenRouter

### Configuration

<ParamField path="openrouterApiKey" type="string" required>
  OpenRouter API key.

  **Policy:** APPROVAL - API keys are sensitive and require approval for changes.

  **Environment variable:** `OPENROUTER_API_KEY`

  Get your API key at: [https://openrouter.ai/keys](https://openrouter.ai/keys)
</ParamField>

<ParamField path="openrouterBaseUrl" type="string" default="https://openrouter.ai/api/v1">
  OpenRouter API base URL.

  **Environment variable:** `OPENROUTER_BASE_URL`
</ParamField>

<ParamField path="openrouterReferer" type="string">
  HTTP Referer header for OpenRouter requests (optional, for ranking).

  **Environment variable:** `OPENROUTER_REFERER`
</ParamField>

<ParamField path="openrouterAppName" type="string">
  Application name for OpenRouter (optional, for ranking).

  **Environment variable:** `OPENROUTER_APP_NAME`
</ParamField>

### Supported Models

OpenRouter provides access to 200+ models from multiple providers:

<AccordionGroup>
  <Accordion title="Popular Models">
    * `anthropic/claude-3.5-sonnet` - Claude 3.5 Sonnet
    * `openai/gpt-4o` - GPT-4o
    * `google/gemini-2.0-flash-exp:free` - Gemini 2.0 Flash (free)
    * `meta-llama/llama-3.3-70b-instruct` - Llama 3.3 70B
    * `mistralai/mistral-large` - Mistral Large
    * `deepseek/deepseek-chat` - DeepSeek V3
  </Accordion>

  <Accordion title="Free Models">
    OpenRouter offers several free models:

    * `google/gemini-2.0-flash-exp:free`
    * `meta-llama/llama-3.2-3b-instruct:free`
    * `microsoft/phi-3-mini-128k-instruct:free`

    <Note>
      Free models have rate limits and may have lower availability.
    </Note>
  </Accordion>
</AccordionGroup>

### Example Configuration

```yaml orcbot.config.yaml theme={null}
# OpenRouter Configuration
llmProvider: openrouter
modelName: anthropic/claude-3.5-sonnet
openrouterApiKey: sk-or-your-key
openrouterReferer: https://github.com/yourusername/orcbot
openrouterAppName: MyOrcBot

# Provider-specific models
providerModelNames:
  openrouter: anthropic/claude-3.5-sonnet

fallbackModelNames:
  openrouter: google/gemini-2.0-flash-exp:free
```

Browse all models: [https://openrouter.ai/docs#models](https://openrouter.ai/docs#models)

## Ollama (Local Models)

### Configuration

<ParamField path="ollamaEnabled" type="boolean" default={false}>
  Enable local Ollama integration for running models locally.
</ParamField>

<ParamField path="ollamaApiUrl" type="string" default="http://localhost:11434">
  Ollama API endpoint URL.

  **Policy:** SAFE - Non-sensitive endpoint configuration.
</ParamField>

### Supported Models

Ollama supports running models locally:

* **Llama 3.3:** `llama3.3:70b`, `llama3.3:8b`
* **Qwen 2.5:** `qwen2.5:72b`, `qwen2.5:32b`
* **Mistral:** `mistral:7b`, `mistral-nemo:12b`
* **DeepSeek:** `deepseek-r1:70b`, `deepseek-r1:8b`
* **Phi-4:** `phi4:14b`
* **Gemma 2:** `gemma2:27b`, `gemma2:9b`

### TUI Management

OrcBot's TUI provides built-in Ollama management:

```bash theme={null}
orcbot ui
```

* **Auto-start server** - Automatically starts Ollama if not running
* **Pull models** - Download models from the Ollama library
* **Model selection** - Switch between installed models
* **OpenAI-compatible** - Native tool calling support

### Example Configuration

```yaml orcbot.config.yaml theme={null}
# Ollama Local Configuration
llmProvider: ollama
modelName: llama3.3:70b
ollamaEnabled: true
ollamaApiUrl: http://localhost:11434

# Provider-specific models
providerModelNames:
  ollama: llama3.3:70b

fallbackModelNames:
  ollama: llama3.3:8b
```

### Installation

<Steps>
  <Step title="Install Ollama">
    Download from [https://ollama.com](https://ollama.com) or install via package manager:

    ```bash theme={null}
    # macOS
    brew install ollama

    # Linux
    curl -fsSL https://ollama.com/install.sh | sh
    ```
  </Step>

  <Step title="Pull a model">
    ```bash theme={null}
    ollama pull llama3.3:70b
    ```
  </Step>

  <Step title="Start Ollama service">
    ```bash theme={null}
    ollama serve
    ```
  </Step>

  <Step title="Configure OrcBot">
    ```yaml theme={null}
    ollamaEnabled: true
    llmProvider: ollama
    modelName: llama3.3:70b
    ```
  </Step>
</Steps>

## Additional Providers

OrcBot also supports these providers via direct API integration:

### NVIDIA NIM

<ParamField path="nvidiaApiKey" type="string">
  NVIDIA NIM API key.

  **Policy:** APPROVAL - API keys are sensitive.

  **Environment variable:** `NVIDIA_API_KEY`
</ParamField>

```yaml theme={null}
llmProvider: nvidia
modelName: meta/llama-3.1-405b-instruct
nvidiaApiKey: nvapi-your-key
```

### Groq

<ParamField path="groqApiKey" type="string">
  Groq API key for ultra-fast inference.

  **Environment variable:** `GROQ_API_KEY`
</ParamField>

```yaml theme={null}
modelName: groq/llama-3.3-70b
groqApiKey: gsk_your-key
```

### Mistral AI

<ParamField path="mistralApiKey" type="string">
  Mistral AI API key.

  **Environment variable:** `MISTRAL_API_KEY`
</ParamField>

```yaml theme={null}
modelName: mistral-large-latest
mistralApiKey: your-mistral-key
```

### Cerebras

<ParamField path="cerebrasApiKey" type="string">
  Cerebras API key for fastest inference.

  **Environment variable:** `CEREBRAS_API_KEY`
</ParamField>

```yaml theme={null}
modelName: cerebras/llama-3.3-70b
cerebrasApiKey: your-cerebras-key
```

### xAI

<ParamField path="xaiApiKey" type="string">
  xAI (Grok) API key.

  **Environment variable:** `XAI_API_KEY`
</ParamField>

```yaml theme={null}
modelName: grok-2
xaiApiKey: your-xai-key
```

## Multi-Provider Strategy

Configure multiple providers for automatic fallback:

```yaml orcbot.config.yaml theme={null}
# Primary provider
llmProvider: openai
modelName: gpt-4o

# Per-provider models (auto-restored on provider switch)
providerModelNames:
  openai: gpt-4o
  google: gemini-2.0-flash-exp
  anthropic: claude-3-5-sonnet-20241022
  openrouter: anthropic/claude-3.5-sonnet
  ollama: llama3.3:70b

# Fallback models
fallbackModelNames:
  openai: gpt-4o-mini
  google: gemini-1.5-flash
  anthropic: claude-3-5-haiku-20241022

# Fast model for lightweight tasks
fastModelName: gpt-4o-mini
```

<Note>
  When a provider fails, OrcBot automatically tries the fallback model, then switches to an alternative provider if configured.
</Note>

## Provider Auto-Detection

You can omit `llmProvider` and OrcBot will infer it from the model name:

```yaml theme={null}
# Auto-detects provider from model prefix
modelName: claude-3-5-sonnet-20241022  # → anthropic
modelName: gpt-4o                      # → openai
modelName: gemini-2.0-flash-exp        # → google
modelName: llama3.3:70b                # → ollama
```

## Best Practices

<CardGroup cols={2}>
  <Card title="Use Environment Variables" icon="lock">
    Store API keys in `.env` files or environment variables, not in YAML configuration.
  </Card>

  <Card title="Configure Fallbacks" icon="rotate">
    Set fallback models to ensure continuity when primary provider fails.
  </Card>

  <Card title="Choose Fast Models" icon="bolt">
    Configure `fastModelName` for lightweight operations to reduce latency and cost.
  </Card>

  <Card title="Test Locally First" icon="server">
    Use Ollama for development and testing before using paid APIs.
  </Card>
</CardGroup>