System Architecture

OrcBot v2.1 is a production-ready autonomous AI agent built on a multi-layered architecture that separates concerns across channels, memory, reasoning, and execution.

Architecture Diagram

The full system architecture from the README shows how all components interact:

Core Components

1. Channels Layer

Purpose: Handle inbound/outbound communication with users across platforms. Key files:

src/channels/TelegramChannel.ts — Telegram integration via Telegraf
src/channels/WhatsAppChannel.ts — WhatsApp integration via Baileys
src/channels/DiscordChannel.ts — Discord integration via discord.js
src/channels/Gateway.ts — REST API + WebSocket gateway

Flow:

User sends message on a channel (e.g., Telegram)
Channel writes message to short-term memory
Channel pushes task to ActionQueue with metadata (source, chatId, userId)

2. Action Queue

Purpose: Durable priority queue for tasks with retry, TTL, and chaining support. Key file: src/memory/ActionQueue.ts Features:

Priority-based execution (0-10, higher = more urgent)
Automatic retry with exponential backoff
Task dependencies (dependsOn field)
TTL for completed/failed actions
Atomic disk persistence via JSONAdapter

interface Action {
    id: string;
    payload: {
        description: string;
        source: string;  // 'telegram' | 'whatsapp' | 'autonomy' | etc.
        sourceId?: string;
        chatId?: string;
        userId?: string;
        isAdmin?: boolean;
        isHeartbeat?: boolean;
    };
    priority: number;
    status: 'pending' | 'running' | 'completed' | 'failed';
    dependsOn?: string[];  // IDs of prerequisite actions
}

3. Agent Core (Action Loop)

Purpose: Orchestrate the ReAct reasoning loop (see agent-loop.mdx). Key file: src/core/Agent.ts (lines 67-797) Responsibilities:

Pop next action from queue
Run SimulationEngine (pre-task planning)
Call DecisionEngine in a loop (reason → tool calls → observations)
Execute skills via SkillsManager
Write observations to memory
Terminate when task is complete

4. Decision Engine

Purpose: Assemble prompts and call LLM with retry logic and guardrails (see decision-pipeline.mdx). Key file: src/core/DecisionEngine.ts Components:

PromptRouter — Selectively activates 8 modular prompt helpers based on task intent
DecisionPipeline — Applies guardrails (deduplication, loop detection, safety checks)
ParserLayer — Normalizes LLM output to structured JSON with 3-tier fallback
ContextCompactor — Automatically compacts prompt when context overflow occurs

5. Memory System

Purpose: Multi-tier memory storage with consolidation and semantic search (see memory-system.mdx). Key file: src/memory/MemoryManager.ts Memory Types:

Short-term — Recent step observations (last ~20 entries)
Episodic — LLM-summarized conversation batches
Long-term — Persistent markdown files (JOURNAL.md, LEARNING.md, USER.md)
Vector — Semantic embeddings for full-history search

6. Skills System

Purpose: Extensible tool registry with hot-reloadable plugins (see skills-system.mdx). Key file: src/core/SkillsManager.ts Features:

Core skills registered at startup (web_search, browser_navigate, run_command, etc.)
Dynamic TypeScript/JavaScript plugins loaded from ~/.orcbot/plugins/
SKILL.md agent skills (agentskills.io format) with progressive disclosure
Intent-based skill routing
Admin-only elevated skills

7. Multi-LLM Provider

Purpose: Abstract LLM provider routing with automatic fallback. Key file: src/core/MultiLLM.ts Supported Providers:

OpenAI (GPT-4o, o1)
Google Gemini
AWS Bedrock
OpenRouter (200+ models)
NVIDIA NIM
Local Ollama

Features:

Automatic provider selection based on model prefix (e.g., gemini-2.0-flash → Google)
Fallback chain on rate limits or errors
Native tool calling for OpenAI and Google
Token tracking and cost estimation

Data Flow Example

Here’s a complete flow for a user request:

1. User: "Search for latest AI news" (via Telegram)
   ↓
2. TelegramChannel receives message
   ↓
3. TelegramChannel.saveMemory({ type: 'short', content: 'User: Search for latest AI news', metadata: { source: 'telegram', chatId: 123 } })
   ↓
4. TelegramChannel.pushTask("Search for latest AI news", priority: 5, { source: 'telegram', chatId: 123 })
   ↓
5. ActionQueue persists action to disk
   ↓
6. Agent.runActionLoop() pops action
   ↓
7. SimulationEngine creates pre-task plan: "I'll use web_search to find news, then summarize"
   ↓
8. DecisionEngine.decide() — Step 1
   - Assembles prompt (task + short memory + episodic + skills list)
   - Calls MultiLLM with tool definitions
   - LLM responds: {"tool": "web_search", "query": "latest AI news 2025"}
   ↓
9. DecisionPipeline validates tool call (no loops, dedup pass)
   ↓
10. Agent executes web_search skill
    ↓
11. web_search calls WebBrowser → Serper API → returns news results
    ↓
12. Agent.saveMemory({ type: 'short', content: 'Observation: Tool web_search returned 10 articles...', metadata: { actionId, step: 1 } })
    ↓
13. DecisionEngine.decide() — Step 2
    - Sees search results in memory
    - LLM responds: {"tool": "send_telegram", "chatId": 123, "message": "Here are the latest AI news..."}
    ↓
14. Agent executes send_telegram skill
    ↓
15. TelegramChannel.sendMessage(123, "Here are the latest AI news...")
    ↓
16. Agent.saveMemory({ type: 'short', content: 'Assistant sent Telegram message to 123', metadata: { role: 'assistant' } })
    ↓
17. DecisionEngine.decide() — Step 3
    - LLM responds: {"completed": true, "summary": "Successfully searched and delivered news"}
    ↓
18. Agent marks action as completed
    ↓
19. MemoryManager.consolidate() (if threshold reached)
    - Summarizes old short-term memories into episodic entry

Configuration

All configuration is centralized in ConfigManager (src/config/ConfigManager.ts), which loads from:

Environment variables
Local ./orcbot.config.yaml
Home ~/orcbot.config.yaml
Global ~/.orcbot/orcbot.config.yaml

Key settings:

# LLM Provider
modelName: gpt-4o
llmProvider: openai  # or 'google', 'openrouter', 'bedrock'
openaiApiKey: sk-...

# Channels
telegramToken: 1234567890:ABC...
whatsappEnabled: true
discordToken: MTk4...

# Memory Limits
memoryContextLimit: 20
memoryEpisodicLimit: 5
memoryConsolidationThreshold: 30

# Action Loop
maxStepsPerAction: 15
maxMessagesPerAction: 10

# Autonomy
autonomyEnabled: true
autonomyInterval: 3600000  # 1 hour in ms
autonomyAllowedChannels: ['telegram']

# Skills
safeMode: false  # Disable run_command and file write if true
pluginAllowList: []  # Empty = all plugins allowed
pluginDenyList: []   # Block specific plugins

Security Architecture

Local-First Design:

All memory, logs, and profiles stored in ~/.orcbot/
No hidden uploads or telemetry
Secrets loaded from config (never hardcoded)

Access Control:

Admin-only elevated skills (run_command, write_file, manage_config)
Cross-channel send blocking (non-admin tasks can’t send to other channels)
Plugin allow/deny lists
Safe mode to disable dangerous operations

Information Boundaries:

Non-admin tasks don’t see journal/learning/episodic context (prevents cross-user leakage)
Session scoping (per-channel-peer by default, configurable to main/per-peer)
Memory deduplication prevents repeated storage of identical events

Performance Optimizations

Token Reduction:

Compact skills prompt (names + usage only after step 1)
Step history compaction (preserve first N + last M, summarize middle)
Aggressive pruning of large tool outputs in history
Per-action prompt caching (core instructions cached per action)

Latency Reduction:

Parallel async retrieval (semantic recall + episodic + RAG run concurrently)
Context assembly caching (recent/episodic/extended cached for 5-30s)
Vector memory background indexing (runs every 5 minutes)

Reliability:

Atomic disk writes with .bak backups (JSONAdapter)
Action queue persistence (survives crashes)
LLM retry with exponential backoff
Automatic context compaction on overflow
Circuit breaker pattern in browser operations

Extension Points

Adding a New Channel:

Implement BaseChannel interface in src/channels/
Call agent.pushTask() for inbound messages
Register channel skills (e.g., send_slack, react_slack)
Add channel detection logic to skills-system

Adding a New Skill:

Register via agent.skills.registerSkill() (for core skills)
Or drop a .ts file in ~/.orcbot/plugins/ (for dynamic plugins)
Skill interface:

{
  name: 'my_skill',
  description: 'What it does and when to use it',
  usage: 'my_skill(arg1, arg2)',
  handler: async (args) => { /* impl */ }
}

Adding a New LLM Provider:

Add provider logic in MultiLLM.ts
Implement callWithTools() for native tool calling (optional)
Add model prefix detection (e.g., claude- → Anthropic)
Update config schema

Get Started

Core Concepts

Guides

Advanced

System Architecture

Architecture Diagram

Core Components

1. Channels Layer

2. Action Queue

3. Agent Core (Action Loop)

4. Decision Engine

5. Memory System

6. Skills System

7. Multi-LLM Provider

Data Flow Example

Configuration

Security Architecture

Performance Optimizations

Extension Points

Further Reading

​Architecture Diagram

​Core Components

​1. Channels Layer

​2. Action Queue

​3. Agent Core (Action Loop)

​4. Decision Engine

​5. Memory System

​6. Skills System

​7. Multi-LLM Provider

​Data Flow Example

​Configuration

​Security Architecture

​Performance Optimizations

​Extension Points

​Further Reading

Architecture Diagram

Core Components

1. Channels Layer

2. Action Queue

3. Agent Core (Action Loop)

4. Decision Engine

5. Memory System

6. Skills System

7. Multi-LLM Provider

Data Flow Example

Configuration

Security Architecture

Performance Optimizations

Extension Points

Further Reading