Skip to content

CODEBASE.md

Latest commit

 

History

History
411 lines (344 loc) · 18.8 KB

CODEBASE.md

File metadata and controls

411 lines (344 loc) · 18.8 KB

CODEBASE.md - Complete Codebase Index for AI Agents

Auto-generated codebase map. Use this to understand file locations, module responsibilities, exports, and dependency flow before making changes.

Quick Reference

Area Entry Point Key Files
CLI src/cli/index.ts init.ts, start.ts, skill.ts, pairing.ts
Agent src/agents/agent.ts config/agent.config.ts
Telegram src/services/TelegramService.ts agents/telegram-agent/
Config src/config/index.ts lib/configWriter.ts
Tools src/tools/index.ts shellTools.ts, fileSystemTools.ts, scheduleTools.ts
Memory src/memory/index.ts (uses @iqai/adk MemoryService)
Skills src/skills/SkillService.ts types/skills.ts
Pairing src/lib/pairing.ts cli/pairing.ts
Types src/types/index.ts workspace.ts, skills.ts

Architecture Overview

CLI Commands (src/cli/)
    │
    ├── init → creates .adk-claw/config.json + workspace/
    ├── start → TelegramService → createClawdAgent() → MCP sampling loop
    ├── skill → SkillService (install/list/remove from GitHub)
    └── pairing → lib/pairing.ts (approve/deny Telegram users)

Agent (src/agents/agent.ts)
    │
    ├── Reads config via src/config/index.ts
    ├── Loads workspace files via src/agents/config/agent.config.ts
    ├── Creates MemoryService via src/memory/index.ts
    ├── Loads tools: memory, schedule, shell, filesystem, private
    └── Returns { runner, sessionService, session, memoryService }

Telegram Flow:
    TelegramService.startTelegramBot()
        → createClawdAgent({ channel: "telegram" })
        → createSamplingHandler (handles incoming messages)
        → getTelegramAgent (MCP tools for sending)
        → initScheduler (cron jobs)

File-by-File Index

src/index.ts (Main Entry - Dev/Test)

  • Purpose: Direct entry point for dev testing (not used in production CLI)
  • Exports: None (runs main())
  • Calls: createClawdAgent({ channel: "cli" }), runner.ask()

src/cli/index.ts (CLI Router)

  • Purpose: CLI entry point (#!/usr/bin/env node), routes commands
  • Commands: init (default), start, skill, skills, pairing, help, version
  • Imports: ./init.js, lazy-imports ./start.js, ./skill.js, ./pairing.js

src/cli/init.ts (Setup Wizard)

  • Purpose: Interactive setup wizard using @clack/prompts
  • Exports: init()
  • Creates:
    • .adk-claw/config.json — full config with auth, channels, tools, skills, memory, cron
    • .adk-claw/workspace/ — copies templates with placeholder replacement
    • Git repo in workspace
  • Key constants: TEMPLATES_DIR resolves differently for dev vs built mode (lines 16-18)
  • Config interface: Defined locally (lines 20-70) — mirrors configFileSchema in config/index.ts

src/cli/start.ts (Start Command)

  • Purpose: Launches Telegram bot
  • Exports: start()
  • Calls: configExists(), startTelegramBot()

src/cli/skill.ts (Skill Management CLI)

  • Purpose: CLI for adk-claw skill add|list|remove
  • Exports: skillCommand(args)
  • Imports: SkillService (fetchSkillFromRepo, installSkill, parseSkillMd, etc.), config/index (addInstalledSkill, removeInstalledSkill)

src/cli/pairing.ts (Pairing Management CLI)

  • Purpose: CLI for adk-claw pairing list|approve|deny|users|remove
  • Exports: pairingCommand(args)
  • Imports: All pairing functions from lib/pairing.ts

src/config/index.ts (Configuration)

  • Purpose: Load, validate, and cache .adk-claw/config.json
  • Exports:
    • getConfig(): AppConfig — cached runtime config (flat interface)
    • getRawConfig(): ConfigFile — raw parsed config (nested, matches file structure)
    • configExists(): boolean
    • addInstalledSkill(name) / removeInstalledSkill(name) / getInstalledSkills()
    • type ConfigFile, type CronJobConfig, type AppConfig
  • Schema: configFileSchema (Zod) validates the JSON file structure
  • Transform: toAppConfig() flattens nested config into flat AppConfig
  • Key fields in AppConfig: apiKey, model, provider, agentName, workspacePath, telegramEnabled, telegramBotToken, discordEnabled, discordUserToken, githubEnabled, githubToken, memoryEnabled, skillsEnabled, cronEnabled, cronJobs

src/lib/configWriter.ts (Config Mutation)

  • Purpose: Read-modify-write helper for config.json
  • Exports: updateConfig(updater: (config: ConfigFile) => void)
  • Used by: scheduleTools.ts to persist cron job changes

src/agents/agent.ts (Main Agent Factory)

  • Purpose: Creates the main Clawd agent with all tools and memory
  • Exports: createClawdAgent(options?: CreateAgentOptions)
  • Returns: { runner, sessionService, session, memoryService } (from AgentBuilder.build())
  • Flow:
    1. Validates workspace (SOUL.md, AGENTS.md, USER.md must exist)
    2. Loads workspace config (SOUL.md, AGENTS.md, USER.md parsed with gray-matter)
    3. Creates MemoryService (FileStorageProvider + OpenRouter embeddings)
    4. Builds system prompt from workspace files + skills
    5. Loads tools: RecallMemory, WriteMemory, ForgetMemory, PreloadMemory, schedule, shell, filesystem, private
    6. Builds agent via AgentBuilder
  • Key constant: PRELOAD_MEMORY_COUNT = 5

src/agents/config/agent.config.ts (System Prompt Builder)

  • Purpose: Load workspace markdown files, build the full system prompt
  • Exports:
    • loadSoul(workspacePath) → SoulConfig
    • loadAgentsConfig(workspacePath) → AgentsConfig
    • loadUserConfig(workspacePath) → UserConfig
    • loadWorkspaceConfig(workspacePath) → WorkspaceConfig
    • buildSystemPrompt(config, runtime?) → string
    • buildSystemPromptFromParams(params) → string
    • buildSystemPromptWithSkills(config) → string (async)
    • validateWorkspace(workspacePath) → { valid, missing }
  • Prompt sections: Tools, ToolCallStyle, Memory, Safety, Heartbeat, SilentReply, Reactions (Telegram-only), Runtime
  • Key constant: MAX_BOOTSTRAP_CHARS = 65536 (truncation threshold)
  • Dependencies: gray-matter for frontmatter parsing, lib/tokens.ts for special tokens

src/agents/telegram-agent/agent.ts (Telegram Sub-Agent)

  • Purpose: Creates a Telegram-specific LlmAgent with MCP tools
  • Exports: getTelegramAgent(samplingHandler) → LlmAgent
  • Uses: @iqai/adk LlmAgent, OpenRouter model

src/agents/telegram-agent/tool.ts (Telegram MCP Tools)

  • Purpose: Initializes Telegram MCP toolset
  • Exports: getTelegramMcpTools(samplingHandler) → tools array
  • Uses: @iqai/adk McpTelegram with bot token and SAMPLING_IGNORE_COMMANDS: ""

src/services/TelegramService.ts (Telegram Bot Core)

  • Purpose: Main Telegram bot - message handling, command routing, MCP sampling
  • Exports: startTelegramBot()
  • Flow:
    1. Creates agent with createClawdAgent({ channel: "telegram" })
    2. Creates sampling handler that:
      • Parses incoming MCP messages (parseTelegramMessage())
      • Routes /start, /new, /reset, /help commands
      • Checks pairing for non-command messages
      • Forwards to runner.ask() for AI responses
    3. Initializes Telegram agent (MCP tools)
    4. Registers bot commands with Telegram API
    5. Initializes scheduler
  • Message format: Key-value format from MCP (user_id, chat_id, content fields)
  • Bot commands: /start, /new (save & reset), /reset (clear), /help

src/services/SchedulerService.ts (Cron Jobs)

  • Purpose: Manages scheduled/recurring tasks via @iqai/adk AgentScheduler
  • Exports:
    • initScheduler(runner, sessionService, appName) → AgentScheduler
    • stopScheduler()
    • broadcastToTelegram(jobId, responseText) — sends to all paired users
    • wrapScheduledInput(input) — prefixes with [SCHEDULED TASK] context
    • extractResponseText(events) — extracts text from ADK events
  • Key: Jobs are configured in config.json cron section, results broadcast to all paired Telegram users

src/services/index.ts (Services Barrel)

  • Re-exports: initScheduler, stopScheduler, startTelegramBot

src/tools/index.ts (Tools Barrel)

  • Re-exports: getFileSystemTools, loadPrivateTools, scheduleTools, setSchedulerDeps, execShellTool, shellTools

src/tools/shellTools.ts (Shell Execution)

  • Purpose: Allowlist-based shell command execution
  • Exports: execShellTool (createTool), shellTools array
  • Tool name: exec_shell
  • Security: Allowlist matching (configurable), metacharacter blocking, execFile (no shell)
  • Defaults: cd, ls, pwd, whoami, df, free, ps, uptime, date, cat, head, tail, git status/log/diff/branch, gh
  • Limits: 30s timeout, 1MB max output

src/tools/fileSystemTools.ts (Workspace File Access)

  • Purpose: MCP-based filesystem access to workspace directory
  • Exports: getFileSystemTools() → tools array
  • Uses: @modelcontextprotocol/server-filesystem via MCP stdio transport
  • Tools provided: read_file, write_file, list_directory, search_files, get_file_info, create_directory, move_file

src/tools/scheduleTools.ts (Dynamic Scheduling)

  • Purpose: Agent tools for managing cron jobs at runtime
  • Exports: scheduleTools array, setSchedulerDeps(deps)
  • Tools:
    • add_schedule — create new scheduled job (cron or intervalMs)
    • remove_schedule — cancel a job
    • list_schedules — list all jobs with status
    • toggle_schedule — pause/resume a job
  • Key: Persists changes to config.json AND live scheduler
  • Limit: MAX_JOBS = 50

src/tools/private-loader.ts (Dynamic Tool Loading)

  • Purpose: Discovers and loads tools from src/tools/private/ (gitignored)
  • Exports: loadPrivateTools() → BaseTool[]
  • Convention: Files export *Tools arrays (e.g., discordTools)
  • Scans: private/ directory relative to this file

src/tools/private/desktopTools.ts (Desktop Control - Private)

  • Purpose: Mouse, keyboard, screenshot, window management via mcp-desktop-pro
  • Exports: loadTools() — async factory
  • Uses: @iqai/adk McpGeneric

src/tools/private/discordTools.ts (Discord - Private)

  • Purpose: Discord channel listing, message fetching, search via user API
  • Exports: discordTools array (3 tools)
  • Tools: list_discord_channels, fetch_discord_messages, search_discord_messages
  • Security: Channel allowlist via discord-channels.json in workspace
  • Rate limiting: Auto-retry on 429

src/memory/index.ts (Memory Service Factory)

  • Purpose: Creates ADK MemoryService with file storage and OpenRouter embeddings
  • Exports: createMemoryService(options) → MemoryService
  • Components:
    • FileStorageProvider — stores in workspace/memory/ as markdown
    • OpenRouterEmbeddingProvider — uses openai/text-embedding-3-small
    • LlmSummaryProvider — uses configured model via OpenRouter

src/skills/SkillService.ts (Skill Management Core)

  • Purpose: Fetch, parse, install, remove, and list skills from GitHub
  • Exports:
    • parseGitHubInput(input, skillName?) → { owner, repo, skillName }
    • fetchSkillFromRepo(owner, repo, skillName) → { content, url }
    • fetchSkill(url) → string (legacy)
    • parseSkillMd(content) → { metadata, content }
    • installSkill(workspacePath, content, metadata, source) → InstalledSkill
    • listSkills(workspacePath) → InstalledSkill[]
    • removeSkill(workspacePath, skillName) → boolean
    • getSkill(workspacePath, skillName) → InstalledSkill | null
    • buildSkillsPrompt(workspacePath) → string | null
  • Skill format: SKILL.md with YAML frontmatter (name, description, license, metadata)
  • Install location: workspace/skills/{name}/SKILL.md + .skill-meta.json

src/skills/index.ts (Skills Barrel)

  • Re-exports: Everything from SkillService.ts

src/lib/open-router.ts (OpenRouter Client)

  • Purpose: Creates OpenRouter AI SDK provider instance
  • Exports: openrouter (createOpenRouter instance)
  • Side effect: Sets process.env.OPENROUTER_API_KEY from config on import

src/lib/logger.ts (Logging)

  • Purpose: File-based logger with daily rotation to .adk-claw/logs/YYYY-MM-DD.log
  • Exports: createLogger(context?) → Logger, logger (default), getLogsDir()
  • Levels: debug (console only if ADK_DEBUG=true), info, warn, error
  • File logging: info+ level written to log files

src/lib/pairing.ts (Multi-Channel Pairing)

  • Purpose: Code-based user pairing with CLI approval for Telegram (extensible to other channels)
  • Exports:
    • createPendingPair(channel, userId, username, chatId) → code | null
    • consumePendingPair(channel, code) → PendingPair | null (approve)
    • removePendingPair(channel, code) → boolean (deny)
    • isAllowed(channel, userId) → boolean
    • addToAllowlist(channel, userId, username?)
    • removeFromAllowlist(channel, userId) → boolean
    • getAllowlist(channel) → AllowedUser[]
    • getPendingPairs(channel) → Record<string, PendingPair>
    • isValidChannel(channel) → type guard
    • formatTimeRemaining(expiresAt) → string
    • Types: Channel, PendingPair, AllowedUser
  • Storage: JSON files in .adk-claw/credentials/ ({channel}-pairing.json, {channel}-allowFrom.json)
  • Code format: 8 chars from ABCDEFGHJKLMNPQRSTUVWXYZ23456789, expires in 1 hour, max 3 pending

src/lib/tokens.ts (Special Tokens)

  • Purpose: Constants for special agent response tokens
  • Exports: SILENT_REPLY_TOKEN = "___", HEARTBEAT_OK = "HEARTBEAT_OK"

src/lib/configWriter.ts (Config Writer)

  • Purpose: Read-modify-write utility for config.json
  • Exports: updateConfig(updater) — applies mutation and writes back

src/types/index.ts (Types Barrel)

  • Re-exports: Everything from skills.ts and workspace.ts

src/types/workspace.ts (Workspace Types)

  • Types: SoulConfig, AgentsConfig, UserConfig, WorkspaceConfig, MemoryFile, RuntimeInfo, SystemPromptParams

src/types/skills.ts (Skill Types)

  • Types: SkillMetadata, InstalledSkill, SkillSource

Templates (templates/)

File Purpose Placeholders
templates/config.json Default config template N/A
templates/workspace/AGENTS.md Workspace context (frontmatter + markdown) {{AGENT_NAME}}, {{CREATED_DATE}}
templates/workspace/SOUL.md Agent personality definition {{AGENT_NAME}}, {{CREATED_DATE}}
templates/workspace/USER.md User profile {{USER_NAME}}, {{CREATED_DATE}}
templates/workspace/.gitignore Workspace gitignore None
templates/workspace/skills/.gitkeep Skills directory placeholder None

Runtime File Structure (.adk-claw/)

.adk-claw/                          # Created by `adk-claw init`, gitignored
├── config.json                     # Main config (auth, channels, tools, etc.)
├── credentials/                    # Pairing data (created at runtime)
│   ├── telegram-pairing.json       # Pending pairing requests
│   └── telegram-allowFrom.json     # Approved users
├── logs/                           # Daily log files (created at runtime)
│   └── YYYY-MM-DD.log
└── workspace/                      # Agent workspace
    ├── .git/                       # Git repo (initialized by init)
    ├── .gitignore
    ├── AGENTS.md                   # Workspace context config
    ├── SOUL.md                     # Agent personality
    ├── USER.md                     # User profile
    ├── memory/                     # Memory storage (managed by ADK MemoryService)
    ├── skills/                     # Installed skills
    │   └── {skill-name}/
    │       ├── SKILL.md
    │       └── .skill-meta.json
    └── discord-channels.json       # Discord channel allowlist (if Discord enabled)

Dependency Graph (Internal Imports)

src/cli/index.ts
  ├── cli/init.ts
  ├── cli/start.ts → services/TelegramService.ts
  ├── cli/skill.ts → skills/SkillService.ts, config/index.ts
  └── cli/pairing.ts → lib/pairing.ts

src/services/TelegramService.ts
  ├── agents/agent.ts (createClawdAgent)
  ├── agents/telegram-agent/agent.ts (getTelegramAgent)
  ├── config/index.ts
  ├── lib/logger.ts, lib/pairing.ts
  ├── tools/scheduleTools.ts (setSchedulerDeps)
  ├── services/SchedulerService.ts

src/agents/agent.ts
  ├── config/index.ts (getConfig)
  ├── agents/config/agent.config.ts (buildSystemPrompt, loadWorkspaceConfig, validateWorkspace)
  ├── lib/open-router.ts (openrouter provider)
  ├── memory/index.ts (createMemoryService)
  ├── skills/SkillService.ts (buildSkillsPrompt)
  ├── tools/index.ts (all tool groups)
  └── types/index.ts

src/agents/config/agent.config.ts
  ├── config/index.ts
  ├── lib/tokens.ts
  ├── skills/SkillService.ts
  └── types/index.ts

src/tools/scheduleTools.ts
  ├── lib/configWriter.ts
  ├── lib/logger.ts
  └── services/SchedulerService.ts

src/services/SchedulerService.ts
  ├── config/index.ts
  ├── lib/logger.ts
  └── lib/pairing.ts

Key Patterns & Conventions

Adding a New Tool

  1. Create file in src/tools/ (or src/tools/private/ if gitignored)
  2. Use createTool() from @iqai/adk with Zod schema
  3. Export as somethingTools array
  4. For public tools: add to src/tools/index.ts barrel and wire into src/agents/agent.ts
  5. For private tools: export as *Tools array — auto-discovered by private-loader.ts

Adding a New CLI Command

  1. Create src/cli/{command}.ts with exported handler function
  2. Add case to switch in src/cli/index.ts
  3. Update printHelp() in src/cli/index.ts

Adding a New Channel

  1. Add to Channel type in src/lib/pairing.ts
  2. Create service in src/services/
  3. Add config schema fields in src/config/index.ts (both configFileSchema and AppConfig)
  4. Add CLI init prompts in src/cli/init.ts

Config Changes

  • Schema: src/config/index.ts — configFileSchema (Zod) + AppConfig interface
  • Init defaults: src/cli/init.ts — collectConfig() return value
  • Runtime mutation: Use updateConfig() from src/lib/configWriter.ts

External Dependencies

  • @iqai/adk — Core agent framework (AgentBuilder, LlmAgent, McpTelegram, McpToolset, McpGeneric, MemoryService, AgentScheduler, tools)
  • @openrouter/ai-sdk-provider — OpenRouter AI SDK integration
  • @clack/prompts — Interactive CLI prompts
  • gray-matter — YAML frontmatter parsing for .md files
  • picocolors — Terminal colors
  • zod — Schema validation
  • dotenv — Environment variable loading