diff --git a/docs/deployment/cloud-deployment.mdx b/docs/deployment/cloud-deployment.mdx index 83a4412..bf52cb7 100644 --- a/docs/deployment/cloud-deployment.mdx +++ b/docs/deployment/cloud-deployment.mdx @@ -3,217 +3,285 @@ title: 'Cloud Deployment' description: 'Deploy agents to RunAgent cloud infrastructure' --- - - Cloud deployment is coming soon. This guide covers the planned features and deployment process. - +Deploy agents to RunAgent cloud infrastructure for production use with automatic scaling, monitoring, and global availability. -## Overview +## Prerequisites -RunAgent's cloud deployment provides a serverless, scalable infrastructure for running your agents in production. With automatic scaling, monitoring, and global distribution, you can focus on building agents while we handle the operations. +Before deploying to the cloud, you need: -## Deployment Process +1. **RunAgent Account** - Sign up at [run-agent.ai](https://run-agent.ai) +2. **API Key** - Get your API key from the dashboard +3. **Configured CLI** - Authenticate your CLI with your API key -### Quick Deploy +## Authentication Setup + +Configure your CLI with your API key: ```bash -# Deploy to cloud -runagent deploy . - -# Deploy with custom name -runagent deploy . --name production-agent - -# Deploy to specific environment -runagent deploy . --env staging -``` - -### Deployment Steps - - - - RunAgent validates your configuration and code - - - Creates optimized container with dependencies - - - Securely uploads to RunAgent infrastructure - - - Starts agent with auto-scaling enabled - - - Verifies agent is responding correctly - - - -## Environments - -### Development +# Setup authentication +runagent setup --api-key + +# Verify configuration +runagent setup --api-key +``` + +The setup command will: +- Validate your API key with the RunAgent middleware +- Store credentials securely in `~/.runagent/config.json` +- Display your account information and tier +- Show middleware sync status + +### Teardown Configuration + +Remove stored credentials: + ```bash -runagent deploy . --env development +runagent teardown ``` -- Lower cost tier -- Detailed logging -- Relaxed rate limits -### Staging +## Deployment Methods + +### Method 1: Quick Deploy (Upload + Start) + +Deploy your agent in a single command: + ```bash -runagent deploy . --env staging +# Deploy to cloud (combines upload + start) +runagent deploy --folder + +# Example +runagent deploy --folder ./my-agent +``` + +This command will: +1. Validate your agent configuration +2. Upload agent code and metadata to the cloud +3. Start the agent automatically +4. Return the deployment endpoint + +**Output:** +``` +βœ… Full deployment successful! +πŸ†” Agent ID: abc-123-def-456 +🌐 Endpoint: https://api.run-agent.ai/api/v1/agents/abc-123-def-456 ``` -- Production-like environment -- Full monitoring -- Same limits as production -### Production +### Method 2: Two-Step Deploy (Upload Then Start) + +For more control, upload and start separately: + +#### Step 1: Upload Agent + ```bash -runagent deploy . --env production +# Upload agent to cloud +runagent upload --folder + +# Specify framework explicitly (optional) +runagent upload --folder --framework langgraph ``` -- High availability -- Auto-scaling -- Full monitoring and alerting -## Configuration +**Upload Process:** +1. Agent validation (checks for required files) +2. Fingerprint generation (for duplicate detection) +3. Metadata upload (config + entrypoints) +4. Source code packaging and upload +5. Local database tracking -### Deployment Settings +**Output:** +``` +πŸ“€ Uploading agent from: ./my-agent +πŸ” Validating agent... +βœ… Agent validation passed +πŸ“‹ Agent config loaded successfully +πŸ” Agent fingerprint: a1b2c3d4... + +πŸ“¦ Uploading to: https://api.run-agent.ai/api/v1 +🌐 Uploading agent metadata... + Metadata uploaded successfully +πŸ“¦ Creating upload package... + Created upload package: agent_abc12345.zip + Uploading agent_abc12345.zip (1,234,567 bytes)... + Processing upload... + Upload completed! + +βœ… Upload successful! +πŸ†” Agent ID: abc-123-def-456 +🌐 Server: https://api.run-agent.ai/api/v1 +πŸ” Fingerprint: a1b2c3d4... + +πŸ’‘ Next step: runagent start --id abc-123-def-456 +``` -```json -{ - "deployment": { - "memory": "1GB", - "timeout": 60, - "min_instances": 1, - "max_instances": 10, - "regions": ["us-east-1", "eu-west-1"], - "environment_variables": { - "LOG_LEVEL": "INFO" - } - } -} +#### Step 2: Start Agent + +```bash +# Start uploaded agent +runagent start --id + +# With custom configuration +runagent start --id --config '{"env": "production"}' +``` + +**Output:** +``` +Starting agent: abc-123-def-456 + +βœ… Agent started successfully! +πŸ†” Agent ID: abc-123-def-456 +🌐 Endpoint: https://api.run-agent.ai/api/v1/agents/abc-123-def-456/execute ``` -### Scaling Configuration +## Agent Validation +Before upload, RunAgent validates your agent: + +### Required Files +- `runagent.config.json` - Agent configuration +- `main.py` or `agent.py` - Entry point file +- `requirements.txt` - Python dependencies (optional) + +### Validation Checks +- βœ… Configuration file exists and is valid JSON +- βœ… Entry point file exists +- βœ… Entrypoints are properly defined +- βœ… Framework is supported +- βœ… No syntax errors in configuration + +**Example Valid Configuration:** ```json { - "scaling": { - "metric": "concurrent_requests", - "target": 10, - "scale_up_cooldown": 60, - "scale_down_cooldown": 300 - } + "agent_name": "my-agent", + "version": "1.0.0", + "framework": "langgraph", + "entrypoints": [ + { + "tag": "chat", + "file": "main.py", + "module": "agent.chat" + } + ] } ``` -## Monitoring +## Duplicate Detection -### Metrics Available +RunAgent uses fingerprinting to detect duplicate agents: -- Request count and latency -- Error rates and types -- Token usage -- Memory and CPU utilization -- Concurrent executions +### How It Works +1. **Fingerprint Generation**: Creates a unique hash of your agent's content +2. **Duplicate Check**: Compares with existing agents in your account +3. **User Prompt**: If duplicate found, asks whether to overwrite or create new -### Accessing Metrics +**Example Scenarios:** -```bash -# View real-time metrics -runagent metrics +**Scenario 1: Identical Content** +``` +⚠️ Agent with identical content already exists! +πŸ†” Existing Agent ID: abc-123-def-456 +πŸ“Š Status: deployed +πŸ“ Type: Remote -# Get specific metric -runagent metrics --metric latency +Do you want to overwrite the existing agent? [y/N]: +``` -# Export metrics -runagent metrics --export csv +**Scenario 2: Modified Content** +``` +⚠️ Agent content has changed! +πŸ†” Existing Agent ID: abc-123-def-456 +πŸ“Š Status: deployed +πŸ“ Type: Remote +πŸ” Content fingerprint changed (modified files detected) + +What would you like to do? + [overwrite] - Overwrite existing agent (coming soon) + [new] - Create new agent + [cancel] - Cancel upload ``` -## Security +## Running Cloud Agents -### Data Encryption -- TLS 1.3 for all API traffic -- Encryption at rest for stored data -- Encrypted environment variables +Execute your deployed agents: -### Network Security -- Private VPC per deployment -- No direct internet access -- API Gateway with rate limiting +### Basic Execution -### Compliance -- SOC 2 Type II (coming soon) -- GDPR compliant -- HIPAA ready (Enterprise) +```bash +# Run with entrypoint tag +runagent run --id --tag --param1=value1 --param2=value2 -## Cost Management +# Examples +runagent run --id abc-123 --tag chat --message="Hello" +runagent run --id abc-123 --tag process --data='{"key": "value"}' +``` -### Pricing Model -- Pay per request -- No charge for idle time -- Volume discounts available +### Using Input Files -### Cost Optimization +```bash +# Run with JSON input file +runagent run --id --tag --input config.json +``` +**Example input.json:** ```json { - "optimization": { - "cache_responses": true, - "compress_payloads": true, - "batch_requests": true + "message": "Hello, world!", + "config": { + "temperature": 0.7, + "max_tokens": 100 } } ``` -### Monitoring Costs +### Streaming Execution -```bash -# View current usage -runagent usage +For streaming entrypoints (must end with `_stream`): -# Set spending alerts -runagent alerts create --threshold 100 --type cost +```bash +# Run streaming endpoint +runagent run --id --tag chat_stream --message="Tell me a story" ``` -## Advanced Features +## Cloud vs Local Deployment -### Multi-Region Deployment +| Feature | Local | Cloud | +|---------|-------|-------| +| **Setup** | `runagent serve` | `runagent deploy` | +| **Infrastructure** | Your machine | RunAgent cloud | +| **Scaling** | Manual | Automatic | +| **Uptime** | While running | 24/7 | +| **Monitoring** | Local logs only | Full monitoring dashboard | +| **URL** | localhost:port | Global endpoint | +| **Authentication** | None | API key required | +| **Cost** | Free | Usage-based pricing | -```json -{ - "deployment": { - "regions": ["us-east-1", "eu-west-1", "ap-southeast-1"], - "routing": "latency" // or "geolocation" - } -} -``` +## Deployment Workflow -### Custom Domains +### Complete Example ```bash -# Add custom domain -runagent domain add api.mycompany.com --deployment +# 1. Create and test agent locally +runagent init my-agent --framework langgraph --template default +cd my-agent -# Configure SSL -runagent domain ssl api.mycompany.com --cert-arn -``` +# 2. Test locally +runagent serve -### Webhook Integration +# 3. Authenticate (if not already done) +runagent setup --api-key -```json -{ - "webhooks": { - "on_success": "https://api.mycompany.com/success", - "on_error": "https://api.mycompany.com/error", - "on_timeout": "https://api.mycompany.com/timeout" - } -} +# 4. Deploy to cloud +runagent deploy --folder . + +# 5. Test cloud deployment +runagent run --id --tag chat --message="Hello" ``` -## CI/CD Integration +### CI/CD Integration -### GitHub Actions +#### GitHub Actions ```yaml -name: Deploy to RunAgent +name: Deploy to RunAgent Cloud + on: push: branches: [main] @@ -221,6 +289,7 @@ on: jobs: deploy: runs-on: ubuntu-latest + steps: - uses: actions/checkout@v3 @@ -232,140 +301,231 @@ jobs: - name: Install RunAgent run: pip install runagent - - name: Deploy + - name: Deploy Agent env: RUNAGENT_API_KEY: ${{ secrets.RUNAGENT_API_KEY }} run: | - runagent deploy . --env production --name my-agent + runagent setup --api-key $RUNAGENT_API_KEY + runagent deploy --folder . ``` -### GitLab CI +#### GitLab CI ```yaml deploy: stage: deploy + image: python:3.9 + script: - pip install runagent - - runagent deploy . --env $CI_COMMIT_BRANCH + - runagent setup --api-key $RUNAGENT_API_KEY + - runagent deploy --folder . + only: - main - - staging ``` -## Rollback and Versioning +## Monitoring and Management -### Version Management +### View Agent Status ```bash -# List versions -runagent versions +# Check if agent is running +runagent start --id +``` -# Deploy specific version -runagent deploy . --version 1.2.3 +### Local Deployment Info -# Rollback to previous -runagent rollback +Deployment information is saved locally in `.deployments/`: -# Rollback to specific version -runagent rollback --version 1.2.0 +```json +{ + "agent_id": "abc-123-def-456", + "remote": true, + "base_url": "https://api.run-agent.ai/api/v1", + "fingerprint": "a1b2c3d4...", + "source_folder": "/path/to/my-agent", + "saved_at": "2025-01-15 10:30:00", + "client_version": "1.0" +} ``` -### Blue-Green Deployment +## Troubleshooting -```bash -# Deploy new version without switching -runagent deploy . --strategy blue-green --no-switch +### Authentication Errors + +**Error:** `Not authenticated. Run 'runagent setup --api-key ' first` -# Test new version -runagent test --version new +**Solution:** +```bash +# Setup authentication +runagent setup --api-key -# Switch traffic -runagent switch --to new +# Verify setup worked +runagent setup --api-key ``` -## Troubleshooting Deployments +### Upload Failures + +**Error:** `Agent validation failed` + +**Solution:** +1. Check that `runagent.config.json` exists and is valid +2. Verify entry point file exists (`main.py` or `agent.py`) +3. Ensure all required dependencies are in `requirements.txt` +4. Run `runagent serve` locally to test first + +**Error:** `Failed to upload agent: HTTP 413` + +**Solution:** Your agent folder is too large. Remove unnecessary files: +- Remove `__pycache__` directories +- Remove `.pyc` files +- Remove large data files (use cloud storage instead) +- Add `.gitignore` patterns to exclude files -### Deployment Failures +### Deployment Fails But Upload Succeeds +**Error:** `Upload succeeded but start failed` + +**Solution:** ```bash -# Check deployment status -runagent status +# Agent is uploaded, just needs to be started +runagent start --id +``` + +### Connection Issues + +**Error:** `Cannot connect to middleware server` + +**Solution:** +1. Check your internet connection +2. Verify the base URL is correct +3. Check if middleware server is accessible +4. Try with explicit base URL: `runagent setup --api-key --base-url https://api.run-agent.ai` + +## Best Practices + +### 1. Test Locally First -# View deployment logs -runagent logs --deployment +Always test your agent locally before deploying: + +```bash +# Test locally +runagent serve +runagent run --id --tag chat --local --message="test" -# Get detailed error -runagent describe +# Deploy after successful local testing +runagent deploy --folder . ``` -### Performance Issues +### 2. Version Control + +Include deployment info in version control: ```bash -# View performance metrics -runagent performance +# Add to .gitignore +.deployments/*.json +.runagent/ + +# But track your agent code +git add runagent.config.json main.py requirements.txt +git commit -m "feat: add chat agent" +``` -# Enable profiling -runagent config --set profiling=true +### 3. Environment Variables -# Download profile -runagent profile --download +Use environment variables for sensitive data: + +```json +{ + "env_vars": { + "OPENAI_API_KEY": "${OPENAI_API_KEY}", + "DATABASE_URL": "${DATABASE_URL}" + } +} ``` -## Best Practices +Set them before deploying: +```bash +export OPENAI_API_KEY="sk-..." +runagent deploy --folder . +``` -1. **Test Thoroughly** - - Use staging environment - - Run integration tests - - Monitor initial deployment +### 4. Incremental Updates -2. **Resource Optimization** - - Right-size memory allocation - - Set appropriate timeouts - - Use caching where possible +For agent updates: -3. **Security** - - Rotate API keys regularly - - Use least privilege principle - - Enable audit logging +```bash +# Upload new version +runagent upload --folder . -4. **Monitoring** - - Set up alerts - - Track key metrics - - Regular performance reviews +# Test before switching +runagent run --id --tag chat --message="test" -## Migration Guide +# If good, can use new agent ID +``` -### From Local to Cloud +### 5. Monitor Performance + +After deployment: +- Test all entrypoints +- Check response times +- Verify outputs are correct +- Monitor error rates + +## Security -1. Test in staging first -2. Update environment variables -3. Verify all dependencies -4. Deploy with same configuration -5. Monitor closely after deployment +### API Key Management -### From Other Platforms +**Best Practices:** +- βœ… Store API keys in environment variables +- βœ… Use different keys for development and production +- βœ… Rotate keys regularly +- βœ… Never commit API keys to version control +- ❌ Don't share API keys +- ❌ Don't hardcode keys in agent code + +### Secure Deployment ```bash -# Import from existing service -runagent import --from heroku --app my-app +# Use environment variables +export RUNAGENT_API_KEY="your-key" +runagent deploy --folder . + +# Or use secure config file +runagent setup --api-key $RUNAGENT_API_KEY +``` -# Validate imported configuration -runagent validate . +## Limits and Quotas -# Deploy -runagent deploy . +### Free Tier +- 5 local agents +- Cloud deployment available +- Standard execution limits + +### Enhanced Limits +Contact sales for: +- Unlimited local agents +- Higher execution limits +- Priority support +- Custom SLAs + +Check your current limits: +```bash +runagent db status --capacity ``` -## Coming Soon +## Next Steps -- Auto-scaling based on custom metrics -- A/B testing support -- Edge deployment options -- Advanced caching strategies -- WebSocket support +- [Local Development](/deployment/local-development) - Test agents locally +- [SDK Overview](/sdk/overview) - Use Python SDK for deployment +- [API Reference](/api-reference) - Direct API access +- [Monitoring Guide](/monitoring) - Track agent performance -## See Also +## Support -- [Local Development](/deployment/local-development) - Test locally first -- [Monitoring Guide](/api-reference/webhooks) - Set up monitoring -- [Security Best Practices](/resources/troubleshooting) - Secure your deployment \ No newline at end of file +Need help with cloud deployment? +- πŸ“– [Documentation](https://docs.run-agent.ai) +- πŸ’¬ [Discord Community](https://discord.gg/runagent) +- πŸ“§ [Email Support](mailto:support@run-agent.ai) +- πŸ› [Report Issues](https://github.com/runagent/runagent/issues) \ No newline at end of file diff --git a/docs/deployment/local-development.mdx b/docs/deployment/local-development.mdx index 88d7391..8f23a14 100644 --- a/docs/deployment/local-development.mdx +++ b/docs/deployment/local-development.mdx @@ -1,7 +1,6 @@ --- title: 'Local Development' description: 'Running and testing agents locally' -icon: 'local' --- ## Overview diff --git a/docs/docs.json b/docs/docs.json index 065c998..0f69864 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -75,6 +75,9 @@ "how-to/frameworks/crewai", "how-to/frameworks/letta", "how-to/frameworks/agno", + "how-to/frameworks/ag2", + "how-to/frameworks/llamaindex", + "how-to/frameworks/parlant", "how-to/frameworks/custom" ] }, diff --git a/docs/explanation/feature-overview.mdx b/docs/explanation/feature-overview.mdx index 119dbac..da4dffd 100644 --- a/docs/explanation/feature-overview.mdx +++ b/docs/explanation/feature-overview.mdx @@ -6,7 +6,7 @@ icon: 'rocket' import NeedHelp from '/snippets/need-help.mdx'; -> See what RunAgent supports β€” from sandbox microVM deployment to streaming SDKs, security, observability, and agent workflows. +> See what RunAgent supports β€” from serverless deployment to streaming SDKs, security, observability, and agent workflows. ## Start Small, Scale Big @@ -18,7 +18,7 @@ import NeedHelp from '/snippets/need-help.mdx'; Use LangGraph, CrewAI, or your own custom agent. - Full SDKs in Python, JS, Go, Rust; REST/WS APIs with streaming. + Full SDKs in Python, JS, Go, Rust; REST/WebSocket APIs with streaming. @@ -28,34 +28,53 @@ import NeedHelp from '/snippets/need-help.mdx'; | Feature | What It Enables | |---|---| -| **Sandboxed MicroVMs** | Each agent runs in isolation (Firecracker-based), for security and containment | -| **Autoscaling & Concurrency** | RunAgent scales agents automatically under load, handles cold starts | -| **Streaming Token Responses** | Clients receive tokens as they are generated β€” ideal for chat UIs | -| **Cross-Language Access** | Call the same agent from JS, Go, Python, Rust without extra infra | -| **State & Memory** *(coming soon)* | Agents can optionally hold memory across calls for richer behavior | -| **Security & Auth** *(in progress)* | Agent-scoped API keys, role scopes, sandbox limits | -| **Telemetry & Observability** *(future)* | Logs, metrics, traces to monitor latency, errors, usage | -| **Agent-to-Agent (A2A)** *(vision)* | Agents can call and coordinate with other agents | -| **Versioning, Rollbacks, Rate Limits** *(roadmap)* | Control agent versions, limit usage, revert if needed | +| **Secure Isolation** | Each agent runs in its own isolated environment for security and resource containment | +| **Automatic Scaling** | RunAgent scales agents automatically under load with fast cold starts | +| **Real-Time Streaming** | Clients receive tokens as they are generated β€” ideal for chat UIs and interactive applications | +| **Cross-Language Access** | Call the same agent from JavaScript, Go, Python, Rust without extra infrastructure | +| **State & Memory** *(coming soon)* | Agents can optionally hold memory across calls for richer, contextual behavior | +| **Security & Auth** *(in progress)* | Agent-scoped API keys, role-based access, sandbox limits | +| **Telemetry & Observability** *(future)* | Logs, metrics, traces to monitor latency, errors, and usage patterns | +| **Agent-to-Agent (A2A)** *(vision)* | Agents can call and coordinate with other agents for complex workflows | +| **Versioning, Rollbacks, Rate Limits** *(roadmap)* | Control agent versions, limit usage, and revert deployments if needed | --- ## How It Works (High Level) -1. **Write an agent** (one Python function with entrypoint configuration) -2. **Deploy to microVMs** β€” RunAgent handles sandboxing + autoscaling -3. **Call via SDK / REST / WS** β€” streaming, sync calls, multi-language -4. **Monitor & scale** β€” logs, metrics, retries, rate limits -5. **Iterate & evolve** β€” new versions, memory, agent communication +1. **Write an agent** - Create a Python function with entrypoint configuration +2. **Deploy serverlessly** - RunAgent handles isolation, sandboxing, and automatic scaling +3. **Call via SDK / REST / WebSocket** - Use streaming, synchronous calls, across multiple languages +4. **Monitor & scale** - Track logs, metrics, retries, and rate limits +5. **Iterate & evolve** - Deploy new versions, add memory, enable agent communication + +--- + +## Production-Ready Features + + + + Deploy agents to production in seconds with `runagent deploy .` - no infrastructure configuration needed. + + + Works seamlessly with LangGraph, CrewAI, AutoGen, AG2, Letta, and custom frameworks. + + + Python, JavaScript/TypeScript, Go, and Rust SDKs with idiomatic APIs. C++ and C# coming soon. + + + WebSocket-based streaming works natively across all SDKs with minimal code changes. + + --- ## Next Steps -- πŸ‘‰ Start with **Deploy your first agent** -- 🧭 Explore **Frameworks & How-tos** -- πŸ“– Dive into **SDK / API Reference** -- πŸ”’ Or read about **Architecture & Concepts** +- Start with **[Deploy your first agent](/tutorials/deploy-your-first-agent)** +- Explore **[Frameworks & How-tos](/explanation/frameworks-overview)** +- Dive into **[SDK / API Reference](/reference/sdk/overview)** +- Or read about **[Architecture & Concepts](/explanation/core-concepts)** --- @@ -63,4 +82,4 @@ import NeedHelp from '/snippets/need-help.mdx'; This page is a **landing overview**. For detailed instructions, go to the linked pages above. - + \ No newline at end of file diff --git a/docs/explanation/introduction.mdx b/docs/explanation/introduction.mdx index 13e03dc..5debaee 100644 --- a/docs/explanation/introduction.mdx +++ b/docs/explanation/introduction.mdx @@ -10,20 +10,20 @@ RunAgent lets you deploy Python-based AI agents and access them from any languag ## Key Benefits -- πŸš€ **Deploy agents in minutes** (serverless, sandboxed microVMs) -- 🌐 **Access from Python, JS, Go, Rust** with native-feeling SDKs -- ⚑ **Built-in autoscaling + token streaming** for real-time applications +- **Deploy agents in seconds** - Serverless infrastructure with automatic scaling +- **Access from Python, JS, Go, Rust and coming more** - Native-feeling SDKs for every language +- **Built-in streaming** - Real-time token streaming for responsive applications ## Quick Actions - + Get started with a complete tutorial - + Integrate with popular AI frameworks - + Access agents from any language @@ -45,7 +45,7 @@ Suppose you've built an incredible AI agent in Python. It uses **LangGraph for c ### Then reality hits: -Your whole team is excited to use it! But the **frontend team** needs to access it in JavaScript, your **mobile app team** wants it in Kotlin, your **Unity team** wants it in good old C#, your **systems team** requires it in Rust. +Your whole team is excited to use it! But the **frontend team** needs to access it in JavaScript, your **mobile app team** wants it in Kotlin, your **Unity team** wants it in C#, your **systems team** requires it in Rust. ### The traditional approach? Build separate implementations, REST APIs, WebSocket handlers... @@ -69,24 +69,36 @@ The fastest way to experience the magic is with **RunAgent CLI**: ``` - + - Let's start with our minimal Agent example. + Let's start with a minimal Agent example. ```bash runagent init my_agent cd my_agent ``` + + This creates a new directory with the basic structure you need to get started. - - Somewhere in your Agent codebase (in this case `main.py`) - ```python main.py icon="python" + + + In your agent codebase (for example, `main.py`), define a function that will serve as your agent's entrypoint: + + ```python main.py def mock_response(message, role="user"): """Test the mock agent with non-streaming responses""" - ... + # Your agent logic here + # Process the message and generate a response return response.content ``` - This `mock_response` function is one of the invocation functions for our agent, so we will add this in `runagent.config.json` file: + + This `mock_response` function is one of the invocation functions for our agent. It takes a message and role as parameters and returns the agent's response. + + + + + Open the `runagent.config.json` file and add your function as an entrypoint: + ```json "entrypoints": [ { @@ -96,66 +108,82 @@ def mock_response(message, role="user"): } ] ``` + + The `file` specifies where your function lives, `module` is the function name, and `tag` is a label you'll use to call this specific entrypoint. - + + + Start your agent locally to test it: + ```bash runagent serve . ``` -And you will see output similar to: +You will see output similar to: + ```bash -πŸ€– Agent Details: +Agent Details: - Agent ID: f7066c98-0eb2-488c-bb37-a869a93d51ce - Host: 127.0.0.1 - Port: 8451 - Framework: default - Status: ready -🌐 Server running at: http://127.0.0.1:8451 -πŸ“– API Documentation: http://127.0.0.1:8451/docs +Server running at: http://127.0.0.1:8451 +API Documentation: http://127.0.0.1:8451/docs -πŸ“ Available endpoints: +Available endpoints: - POST /api/v1/agents/.../execute/minimal - Run your agent INFO: Uvicorn running on http://127.0.0.1:8451 (Press CTRL+C to quit) ``` - That's it, your agent is running, and is accessible through standard REST-api as well as all RunAgent SDKs. + + That's it! Your agent is now running and accessible through standard REST API as well as all RunAgent SDKs. - -Using the RunAgent SDKs, you can use your agent in your application, only using the agent ID and the entrypoint tag. Your agentic entrypoint (`mock_response` function) now becomes accessible in: - + + +Using the RunAgent SDKs, you can access your agent from any supported language. You only need the agent ID and the entrypoint tag. Your `mock_response` function now becomes accessible in multiple languages: + ```python Python from runagent import RunAgentClient +# Create a client pointing to your agent ra = RunAgentClient( agent_id="f7066c98-0eb2-488c-bb37-a869a93d51ce", entrypoint_tag="minimal", local=True ) +# Call your agent agent_result = ra.run( role="user", message="Analyze the benefits of remote work for software teams" ) + +print(agent_result) ``` ```javascript JavaScript import { RunAgentClient } from 'runagent'; +// Create a client pointing to your agent const ra = new RunAgentClient({ agentId: 'f7066c98-0eb2-488c-bb37-a869a93d51ce', entrypointTag: 'minimal', local: true, }); -const analysis = await client.run({ +// Call your agent +const analysis = await ra.run({ role: "user", message: "Analyze the benefits of remote work for software teams" }); + +console.log(analysis); ``` ```rust Rust @@ -164,8 +192,14 @@ use serde_json::json; #[tokio::main] async fn main() -> Result<(), Box> { - let client = RunAgentClient::new("f7066c98-0eb2-488c-bb37-a869a93d51ce", "minimal", true).await?; + // Create a client pointing to your agent + let client = RunAgentClient::new( + "f7066c98-0eb2-488c-bb37-a869a93d51ce", + "minimal", + true + ).await?; + // Call your agent let response = client.run(&[ ("role", json!("user")), ("message", json!("Analyze the benefits of remote work for software teams")) @@ -189,6 +223,7 @@ import ( ) func main() { + // Create a client pointing to your agent agentClient, err := client.NewWithAddress( "f7066c98-0eb2-488c-bb37-a869a93d51ce", "minimal", @@ -204,6 +239,7 @@ func main() { ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute) defer cancel() + // Call your agent result, err := agentClient.Run(ctx, map[string]interface{}{ "message": "Analyze the benefits of remote work for software teams", "role": "user", @@ -217,14 +253,54 @@ func main() { ``` +Notice how the same agent logic is accessible from all languages with idiomatic syntax for each. + -...and you're ready to **deploy your AI agents** & **use them in the futuristic software** you are building. +That's the core workflow. You're now ready to **deploy your AI agents** and **integrate them into your applications**. + +## From Local Testing to Production in Seconds + +Once you've tested your agent locally and confirmed everything works, deploying to production is just one command away. + +**RunAgent Cloud** is the fastest AI agent deployment platform, designed to take your local agent from development to production instantly. No complex infrastructure setup, no deployment headaches. + +### Deploy to Production + +```bash +runagent deploy . +``` + +That's it. Your agent is now live and accessible globally with: + +- **Automatic scaling** - Handle one request or one million +- **Global edge network** - Low latency worldwide +- **Zero infrastructure management** - Focus on your agent, not servers +- **Production-ready security** - Enterprise-grade isolation and security + +Your deployed agent gets a production URL and agent ID. Simply update your SDK clients to use the production endpoint: + +```python +# Switch from local to production +ra = RunAgentClient( + agent_id="your-production-agent-id", + entrypoint_tag="minimal", + local=False # Now pointing to RunAgent Cloud +) +``` + +The same code you tested locally now runs in a fully managed production environment. + + + Explore advanced deployment options, environment variables, and monitoring capabilities. + -We will ❀️ to hear about it. Hit us up on [Discord](https://discord.gg/Q9P9AdHVHz). - ## Agent Framework Support @@ -233,42 +309,36 @@ RunAgent works with any Python-based AI agent framework: Deploy your LangGraph agents with built-in state management and workflow execution. Deploy multi-agent CrewAI systems with coordinated execution and monitoring. Deploy Microsoft AutoGen multi-agent conversations with step-by-step execution. Deploy AG2 conversational agents with fact-checking and multi-turn interactions. Deploy Agno AI agents with print response capabilities and streaming support. Deploy Letta memory-enabled agents with persistent conversations and tool integration. @@ -276,7 +346,6 @@ RunAgent works with any Python-based AI agent framework: @@ -324,28 +393,37 @@ RunAgent provides native-like access to your deployed agents across multiple lan href="/reference/sdk/overview" horizontal > - We're actively developing SDKs for additional languages including C#, Java, and PHP. Want to contribute or request a specific language? Join our Discord community. + We're actively developing SDKs for additional languages including C++, C#, Java, and PHP. Want to contribute or request a specific language? Join our Discord community. ## Real-Time Streaming Across Languages -In addition to REST API like responses, you can also stream your agent response super-easily through our SDKs. +In addition to standard REST API responses, you can also stream your agent responses seamlessly through our SDKs. -When your targeted entrypoint streams response, RunAgent makes it feel native in every language SDK: +When your entrypoint streams a response, RunAgent makes it feel native in every language SDK: - + - Somewhere an Iterator in your Agent codebase (in this case `main.py`) + Create a function in your agent codebase that returns an Iterator (in this case in `main.py`): - ```python main.py icon="python" + ```python main.py def mock_response_stream(message, role="user") -> Iterator[str]: """Test the mock agent with streaming responses""" - ... - return response.content + # Your streaming agent logic here + # Yield chunks of the response as they're generated + for chunk in generate_response_chunks(message): + yield chunk ``` - This `mock_response_stream` function will return an Iterator, and to stream this response, we will add this in `runagent.config.json` file, as another entrypoint: + + This `mock_response_stream` function will return an Iterator that yields response chunks as they're generated. + + + + + Add this function to your `runagent.config.json` file as another entrypoint: + ```json "entrypoints": [ { @@ -355,54 +433,64 @@ def mock_response_stream(message, role="user") -> Iterator[str]: } ] ``` - The tag for a Streaming Entrypoint must end with a `_stream` suffix. That is how RunAgent identifies it for streaming. + + The tag for a streaming entrypoint must end with a `_stream` suffix. This is how RunAgent identifies it as a streaming endpoint. - - Spin up the agent just like we did before. But now we have an additional streaming entrypoint. + + + Spin up the agent just like before. Now you have an additional streaming entrypoint available. + ```bash runagent serve . ``` - -Using the RunAgent SDKs, you can use your agent in your application, only using the agent ID and the entrypoint tag. Your agentic entrypoint (`mock_response` function) now becomes accessible in: + + +Using the RunAgent SDKs, you can stream responses from your agent. The streaming experience feels natural in each language: ```python Python from runagent import RunAgentClient +# Create a client with the streaming tag ra = RunAgentClient( agent_id="f7066c98-0eb2-488c-bb37-a869a93d51ce", entrypoint_tag="minimal_stream", local=True ) +# Stream the response chunk by chunk for chunk in ra.run( - role = "user", - content = "Analyze the benefits of remote work for software teams" + role="user", + message="Analyze the benefits of remote work for software teams" ): - print(chunk) - + print(chunk, end='', flush=True) ``` - ```javascript JavaScript import { RunAgentClient } from 'runagent'; +// Create a client with the streaming tag const client = new RunAgentClient({ agentId: 'f7066c98-0eb2-488c-bb37-a869a93d51ce', - entrypointTag: 'report_gen_streaming', + entrypointTag: 'minimal_stream', local: true }); await client.initialize(); -const stream = await client.runStream({topic: "AI Market Trends"}); +// Stream the response chunk by chunk +const stream = await client.runStream({ + role: "user", + message: "Analyze the benefits of remote work for software teams" +}); + for await (const chunk of stream) { - document.getElementById('report').innerHTML += chunk; + document.getElementById('output').innerHTML += chunk; } ``` @@ -413,9 +501,19 @@ use futures::StreamExt; #[tokio::main] async fn main() -> Result<(), Box> { - let client = RunAgentClient::new("f7066c98-0eb2-488c-bb37-a869a93d51ce", "report_gen_streaming", true).await?; + // Create a client with the streaming tag + let client = RunAgentClient::new( + "f7066c98-0eb2-488c-bb37-a869a93d51ce", + "minimal_stream", + true + ).await?; + + // Stream the response chunk by chunk + let mut stream = client.run_stream(&[ + ("role", json!("user")), + ("message", json!("Analyze the benefits of remote work for software teams")) + ]).await?; - let mut stream = client.run_stream(&[("topic", json!("AI Market"))]).await?; while let Some(chunk) = stream.next().await { print!("{}", chunk?); } @@ -435,26 +533,28 @@ import ( ) func main() { + // Create a client with the streaming tag client := runagent.NewRunAgentClient(runagent.Config{ AgentID: "f7066c98-0eb2-488c-bb37-a869a93d51ce", - EntrypointTag: "report_gen_streaming", + EntrypointTag: "minimal_stream", Local: true, }) ctx := context.Background() client.Initialize(ctx) - s, err := client.RunStream(ctx, map[string]interface{}{ - "role": "user", - "message": "Analyze the benefits of remote work for software teams", + // Stream the response chunk by chunk + stream, err := client.RunStream(ctx, map[string]interface{}{ + "role": "user", + "message": "Analyze the benefits of remote work for software teams", }) if err != nil { log.Fatal(err) } - defer s.Close() + defer stream.Close() for { - data, hasMore, err := s.Next(ctx) + data, hasMore, err := stream.Next(ctx) if err != nil { log.Fatal(err) } @@ -469,13 +569,14 @@ func main() { - -**What's happening**: WebSocket connections, real-time data flow, and native iteration patternsβ€”all handled automatically by RunAgent. +**What's happening behind the scenes**: WebSocket connections, real-time data flow, and native iteration patterns are all handled automatically by RunAgent's infrastructure. + +## Next Steps -- Head to the [tutorials](/tutorials/deploy-your-first-agent) guide to see the full magic +- Head to the [tutorials](/tutorials/deploy-your-first-agent) guide to see the full capabilities - Browse our [framework guides](/how-to/frameworks/langgraph) for LangGraph, CrewAI, and more -- Check out [core concepts](/explanation/core-concepts) to understand how it works +- Check out [core concepts](/explanation/core-concepts) to understand how it all works - + \ No newline at end of file diff --git a/docs/explanation/key-concepts-at-a-glance.mdx b/docs/explanation/key-concepts-at-a-glance.mdx index c0c54b4..5e4b7c2 100644 --- a/docs/explanation/key-concepts-at-a-glance.mdx +++ b/docs/explanation/key-concepts-at-a-glance.mdx @@ -11,17 +11,17 @@ import NeedHelp from '/snippets/need-help.mdx'; ## The Magic in 4 Bullets - - **Sandboxed isolation** - Each agent runs in its own secure container, preventing resource conflicts and security issues. + + **Sandboxed execution** - Each agent runs in its own secure, isolated environment, preventing resource conflicts and security issues. - - **Serverless style** - Automatically scales from 0 to thousands of requests without manual configuration. + + **Serverless architecture** - Automatically scales from zero to thousands of requests without manual configuration. - + **Native APIs** - Call the same agent from Python, JavaScript, Go, Rust with language-idiomatic interfaces. - - **Real-time responses** - Built-in token streaming for interactive applications and long-form generation. + + **Built-in streaming** - Token-by-token streaming for interactive applications and long-form generation. @@ -30,7 +30,7 @@ import NeedHelp from '/snippets/need-help.mdx'; ## Traditional Approach vs RunAgent - + - Spin up FastAPI server - Build WebSocket handlers for streaming - Set up authentication and rate limiting @@ -41,11 +41,11 @@ import NeedHelp from '/snippets/need-help.mdx'; - **Result**: Weeks of infrastructure work before you can use your agent - + - Write Python function - Run `runagent serve .` - Call from any language immediately - - **Result**: Agent is live and accessible in minutes + - **Result**: Agent is live and accessible in seconds @@ -58,7 +58,7 @@ import NeedHelp from '/snippets/need-help.mdx'; | **Entrypoints** | Python functions exposed as API endpoints | Your business logic becomes accessible from any language | | **Tags** | Nicknames for your functions (e.g., "chat", "analyze") | Clean, memorable API names instead of function names | | **Streaming** | Real-time token delivery as they are generated | Interactive UIs, progress indicators, better UX | -| **MicroVMs** | Lightweight, isolated containers for each agent | Security, resource isolation, fast cold starts | +| **Isolation** | Secure, containerized execution environments | Security, resource isolation, fast cold starts | | **SDKs** | Language-native clients that feel like local functions | No need to learn REST APIs or WebSocket protocols | | **Config** | `runagent.config.json` defines your agent's interface | Single source of truth for what your agent can do | @@ -101,16 +101,16 @@ import NeedHelp from '/snippets/need-help.mdx'; ## What Makes This Different - + Built specifically for AI agents, not generic microservices. Handles streaming, state, and agent patterns out of the box. - + No Docker, Kubernetes, or cloud configuration needed. Focus on your agent logic, not deployment. - + Security, scaling, monitoring, and error handling built-in from day one. - + Your Python agent becomes a first-class citizen in every supported language ecosystem. @@ -120,15 +120,15 @@ import NeedHelp from '/snippets/need-help.mdx'; ## Ready to Dive Deeper? - + See the magic in action with a 5-minute tutorial - + Understand entrypoints, streaming, and configuration in detail - + Learn how RunAgent works under the hood - + \ No newline at end of file diff --git a/docs/how-to/call-from-rust.mdx b/docs/how-to/call-from-rust.mdx index b3d9381..934d6fc 100644 --- a/docs/how-to/call-from-rust.mdx +++ b/docs/how-to/call-from-rust.mdx @@ -4,17 +4,26 @@ description: 'Use RunAgent agents from Rust applications' icon: 'rust' --- - - **Prerequisites**: Rust 1.70+ and completed [Deploy Your First Agent](/tutorials/deploy-your-first-agent) tutorial - + +Use RunAgent agents from Rust applications with high-performance, type-safe execution. + +## Prerequisites + +- Rust 1.70 or higher +- Completed [Deploy Your First Agent](/tutorials/deploy-your-first-agent) tutorial ## Overview -The Rust SDK provides high-performance access to RunAgent agents with zero-cost abstractions, futures-based async operations, and compile-time type safety. It's designed to work seamlessly with Rust's ownership model and error handling. +The Rust SDK provides production-ready access to RunAgent agents with: + +- **Zero-cost abstractions** for optimal performance +- **Futures-based async operations** powered by Tokio +- **Compile-time type safety** with comprehensive error handling +- **Seamless integration** with Rust's ownership model ## Installation -Add to your `Cargo.toml`: +Add the following dependencies to your `Cargo.toml`: ```toml [dependencies] @@ -26,7 +35,9 @@ serde_json = "1.0" ## Basic Usage -### Synchronous Calls +### Synchronous Execution + +Execute an agent and wait for the complete response: ```rust use runagent::client::RunAgentClient; @@ -34,14 +45,14 @@ use serde_json::json; #[tokio::main] async fn main() -> Result<(), Box> { - // Create client + // Create client connection let client = RunAgentClient::new( "your_agent_id_here", "main", - true, // local + true, // local deployment ).await?; - // Call your agent + // Execute agent with parameters let result = client.run(&[ ("message", json!("Hello, how are you?")), ("userId", json!("rust_user")), @@ -52,7 +63,9 @@ async fn main() -> Result<(), Box> { } ``` -### Asynchronous Calls +### Asynchronous Execution + +Process multiple requests concurrently for improved throughput: ```rust use runagent::client::RunAgentClient; @@ -67,7 +80,7 @@ async fn main() -> Result<(), Box> { true, ).await?; - // Run multiple requests concurrently + // Create concurrent requests let futures = (0..5).map(|i| { let client = client.clone(); async move { @@ -78,6 +91,7 @@ async fn main() -> Result<(), Box> { } }); + // Execute all requests concurrently let results = join_all(futures).await; for (i, result) in results.into_iter().enumerate() { @@ -93,7 +107,9 @@ async fn main() -> Result<(), Box> { ## Advanced Features -### 1. Streaming Responses +### Streaming Responses + +Process responses as they arrive for real-time applications: ```rust use runagent::client::RunAgentClient; @@ -104,29 +120,38 @@ use futures::StreamExt; async fn main() -> Result<(), Box> { let client = RunAgentClient::new( "your_agent_id_here", - "streaming", // Note: tag ends with _stream + "streaming", // Entrypoint tag must end with _stream true, ).await?; - // Stream responses + // Create streaming connection let mut stream = client.run_stream(&[ ("message", json!("Tell me a story")), ("userId", json!("rust_user")), ]).await?; + // Process chunks as they arrive while let Some(chunk) = stream.next().await { - print!("{}", chunk?); + match chunk { + Ok(data) => print!("{}", data), + Err(e) => { + eprintln!("Stream error: {}", e); + break; + } + } } Ok(()) } ``` -### 2. Error Handling +### Error Handling + +Handle different error types with Rust's type-safe error handling: ```rust use runagent::client::RunAgentClient; -use runagent::errors::*; +use runagent::types::{RunAgentError, RunAgentResult}; use serde_json::json; #[tokio::main] @@ -142,23 +167,25 @@ async fn main() -> Result<(), Box> { ]).await; match result { - Ok(response) => println!("Success: {}", response), + Ok(response) => { + println!("Success: {}", response); + } Err(e) => { match e { - RunAgentError::AuthenticationError => { + RunAgentError::Authentication { .. } => { eprintln!("Authentication failed. Check your API key."); } RunAgentError::AgentNotFoundError => { - eprintln!("Agent not found. Check your agent ID."); + eprintln!("Agent not found. Verify the agent ID."); } - RunAgentError::ValidationError(msg) => { - eprintln!("Validation error: {}", msg); + RunAgentError::Validation { message } => { + eprintln!("Validation error: {}", message); } RunAgentError::RateLimitError => { - eprintln!("Rate limit exceeded. Please wait and try again."); + eprintln!("Rate limit exceeded. Please retry later."); } RunAgentError::TimeoutError => { - eprintln!("Request timed out. Please try again."); + eprintln!("Request timed out."); } RunAgentError::NetworkError => { eprintln!("Network error. Check your connection."); @@ -174,7 +201,9 @@ async fn main() -> Result<(), Box> { } ``` -### 3. Custom Headers and Metadata +### Custom Configuration + +Configure the client with custom headers and timeouts: ```rust use runagent::client::RunAgentClient; @@ -183,10 +212,12 @@ use std::collections::HashMap; #[tokio::main] async fn main() -> Result<(), Box> { + // Prepare custom headers let mut headers = HashMap::new(); headers.insert("X-Request-ID".to_string(), "unique-request-id".to_string()); headers.insert("X-User-ID".to_string(), "rust_user".to_string()); + // Create client with configuration let client = RunAgentClient::new_with_config( "your_agent_id_here", "main", @@ -212,14 +243,16 @@ async fn main() -> Result<(), Box> { ### Environment Variables +Configure the SDK using environment variables: + ```bash -# Set API key +# API authentication export RUNAGENT_API_KEY="your-api-key" -# Set API URL +# API endpoint export RUNAGENT_API_URL="https://api.run-agent.ai" -# Set default agent ID +# Default agent export RUNAGENT_AGENT_ID="your-agent-id" ``` @@ -239,6 +272,8 @@ Create `~/.runagent/config.json`: ### Programmatic Configuration +Configure the client directly in code: + ```rust use runagent::client::RunAgentClient; use std::collections::HashMap; @@ -253,17 +288,18 @@ async fn main() -> Result<(), Box> { "main", true, Some(headers), - Some(30), + Some(30), // timeout in seconds ).await?; - // Use client... Ok(()) } ``` ## Best Practices -### 1. **Connection Management** +### Connection Management + +Implement a connection manager for efficient resource usage: ```rust use runagent::client::RunAgentClient; @@ -276,20 +312,27 @@ struct AgentManager { } impl AgentManager { - async fn new(agent_id: &str, entrypoint_tag: &str) -> Result> { + async fn new( + agent_id: &str, + entrypoint_tag: &str + ) -> Result> { let client = RunAgentClient::new(agent_id, entrypoint_tag, true).await?; Ok(Self { client: Arc::new(Mutex::new(client)), }) } - async fn run(&self, message: &str, options: &[(&str, serde_json::Value)]) -> Result> { + async fn run( + &self, + message: &str, + options: &[(&str, serde_json::Value)] + ) -> Result> { let client = self.client.lock().await; let mut params = vec![("message", json!(message))]; params.extend(options.iter().map(|(k, v)| (*k, v.clone()))); let result = client.run(¶ms).await?; - Ok(result) + Ok(result.to_string()) } } @@ -302,7 +345,9 @@ async fn main() -> Result<(), Box> { } ``` -### 2. **Retry Logic** +### Retry Logic + +Implement exponential backoff for resilient operations: ```rust use runagent::client::RunAgentClient; @@ -318,11 +363,10 @@ async fn run_with_retry( for attempt in 0..max_retries { match client.run(&[("message", json!(message))]).await { - Ok(result) => return Ok(result), + Ok(result) => return Ok(result.to_string()), Err(e) => { last_error = Some(e); if attempt < max_retries - 1 { - // Exponential backoff let backoff = Duration::from_secs(2_u64.pow(attempt as u32)); sleep(backoff).await; } @@ -342,20 +386,25 @@ async fn main() -> Result<(), Box> { } ``` -### 3. **Logging and Monitoring** +### Logging and Monitoring + +Add observability to your agent interactions: ```rust use runagent::client::RunAgentClient; use serde_json::json; use std::time::Instant; -use tracing::{info, error, warn}; +use tracing::{info, error}; struct MonitoredAgent { client: RunAgentClient, } impl MonitoredAgent { - async fn new(agent_id: &str, entrypoint_tag: &str) -> Result> { + async fn new( + agent_id: &str, + entrypoint_tag: &str + ) -> Result> { let client = RunAgentClient::new(agent_id, entrypoint_tag, true).await?; Ok(Self { client }) } @@ -368,7 +417,7 @@ impl MonitoredAgent { Ok(result) => { let duration = start_time.elapsed(); info!("Request completed in {:?}", duration); - Ok(result) + Ok(result.to_string()) } Err(e) => { let duration = start_time.elapsed(); @@ -392,7 +441,9 @@ async fn main() -> Result<(), Box> { ## Common Patterns -### 1. **Agent Factory Pattern** +### Agent Factory Pattern + +Create agents with consistent configuration: ```rust use runagent::client::RunAgentClient; @@ -415,7 +466,11 @@ impl AgentFactory { Self { base_config } } - async fn get_agent(&self, agent_id: &str, entrypoint_tag: &str) -> Result> { + async fn get_agent( + &self, + agent_id: &str, + entrypoint_tag: &str + ) -> Result> { RunAgentClient::new_with_config( agent_id, entrypoint_tag, @@ -425,11 +480,17 @@ impl AgentFactory { ).await.map_err(|e| e.into()) } - async fn get_chat_agent(&self, agent_id: &str) -> Result> { + async fn get_chat_agent( + &self, + agent_id: &str + ) -> Result> { self.get_agent(agent_id, "chat").await } - async fn get_analysis_agent(&self, agent_id: &str) -> Result> { + async fn get_analysis_agent( + &self, + agent_id: &str + ) -> Result> { self.get_agent(agent_id, "analyze").await } } @@ -449,11 +510,14 @@ async fn main() -> Result<(), Box> { } ``` -### 2. **Agent Wrapper Pattern** +### Agent Wrapper Pattern + +Encapsulate agent functionality with a clean interface: ```rust use runagent::client::RunAgentClient; use serde_json::json; +use futures::StreamExt; struct AgentWrapper { client: RunAgentClient, @@ -462,7 +526,10 @@ struct AgentWrapper { } impl AgentWrapper { - async fn new(agent_id: &str, entrypoint_tag: &str) -> Result> { + async fn new( + agent_id: &str, + entrypoint_tag: &str + ) -> Result> { let client = RunAgentClient::new(agent_id, entrypoint_tag, true).await?; Ok(Self { client, @@ -471,32 +538,45 @@ impl AgentWrapper { }) } - async fn call(&self, params: &[(&str, serde_json::Value)]) -> Result> { - self.client.run(params).await.map_err(|e| e.into()) + async fn call( + &self, + params: &[(&str, serde_json::Value)] + ) -> Result> { + self.client.run(params).await.map(|v| v.to_string()).map_err(|e| e.into()) } - async fn chat(&self, message: &str, user_id: &str) -> Result> { + async fn chat( + &self, + message: &str, + user_id: &str + ) -> Result> { let result = self.client.run(&[ ("message", json!(message)), ("userId", json!(user_id)), ]).await?; - Ok(result) + Ok(result.to_string()) } - async fn analyze(&self, data: &str, analysis_type: &str) -> Result> { + async fn analyze( + &self, + data: &str, + analysis_type: &str + ) -> Result> { let result = self.client.run(&[ ("data", json!(data)), ("analysisType", json!(analysis_type)), ]).await?; - Ok(result) + Ok(result.to_string()) } async fn stream(&self, message: &str) -> Result> { - let mut stream = self.client.run_stream(&[("message", json!(message))]).await?; - let mut response = String::new(); + let mut stream = self.client.run_stream(&[ + ("message", json!(message)) + ]).await?; + let mut response = String::new(); while let Some(chunk) = stream.next().await { - response.push_str(&chunk?); + response.push_str(&chunk?.to_string()); } Ok(response) @@ -512,7 +592,9 @@ async fn main() -> Result<(), Box> { } ``` -### 3. **Agent Pool Pattern** +### Agent Pool Pattern + +Distribute load across multiple agent instances: ```rust use runagent::client::RunAgentClient; @@ -526,7 +608,9 @@ struct AgentPool { } impl AgentPool { - async fn new(agent_configs: Vec) -> Result> { + async fn new( + agent_configs: Vec + ) -> Result> { let mut agents = Vec::new(); for config in agent_configs { @@ -553,10 +637,13 @@ impl AgentPool { agent } - async fn call(&self, params: &[(&str, serde_json::Value)]) -> Result> { + async fn call( + &self, + params: &[(&str, serde_json::Value)] + ) -> Result> { let agent = self.get_agent().await; let client = agent.lock().await; - client.run(params).await.map_err(|e| e.into()) + client.run(params).await.map(|v| v.to_string()).map_err(|e| e.into()) } } @@ -586,13 +673,6 @@ async fn main() -> Result<(), Box> { headers: None, timeout: None, }, - AgentConfig { - agent_id: "agent3".to_string(), - entrypoint_tag: "main".to_string(), - local: true, - headers: None, - timeout: None, - }, ]).await?; let result = pool.call(&[("message", json!("Hello from pool"))]).await?; @@ -601,56 +681,11 @@ async fn main() -> Result<(), Box> { } ``` -## Error Handling - -### Common Error Types - -```rust -use runagent::client::RunAgentClient; -use runagent::errors::*; -use serde_json::json; - -async fn handle_errors(client: &RunAgentClient, message: &str) -> Result<(), Box> { - let result = client.run(&[("message", json!(message))]).await; - - match result { - Ok(response) => { - println!("Success: {}", response); - } - Err(e) => { - match e { - RunAgentError::AuthenticationError => { - eprintln!("Authentication failed. Check your API key."); - } - RunAgentError::AgentNotFoundError => { - eprintln!("Agent not found. Check your agent ID."); - } - RunAgentError::ValidationError(msg) => { - eprintln!("Validation error: {}", msg); - } - RunAgentError::RateLimitError => { - eprintln!("Rate limit exceeded. Please wait and try again."); - } - RunAgentError::TimeoutError => { - eprintln!("Request timed out. Please try again."); - } - RunAgentError::NetworkError => { - eprintln!("Network error. Check your connection."); - } - _ => { - eprintln!("Unexpected error: {}", e); - } - } - } - } - - Ok(()) -} -``` - ## Performance Optimization -### 1. **Connection Pooling** +### Connection Pooling + +Reuse connections for improved performance: ```rust use runagent::client::RunAgentClient; @@ -664,7 +699,11 @@ struct ConnectionPool { } impl ConnectionPool { - async fn new(agent_id: &str, entrypoint_tag: &str, pool_size: usize) -> Result> { + async fn new( + agent_id: &str, + entrypoint_tag: &str, + pool_size: usize + ) -> Result> { let mut pool = Vec::new(); for _ in 0..pool_size { @@ -685,10 +724,13 @@ impl ConnectionPool { client } - async fn run(&self, params: &[(&str, serde_json::Value)]) -> Result> { + async fn run( + &self, + params: &[(&str, serde_json::Value)] + ) -> Result> { let client = self.get_client().await; let client = client.lock().await; - client.run(params).await.map_err(|e| e.into()) + client.run(params).await.map(|v| v.to_string()).map_err(|e| e.into()) } } @@ -701,7 +743,9 @@ async fn main() -> Result<(), Box> { } ``` -### 2. **Caching** +### Response Caching + +Cache responses for frequently requested data: ```rust use runagent::client::RunAgentClient; @@ -716,7 +760,10 @@ struct CachedAgent { } impl CachedAgent { - async fn new(agent_id: &str, entrypoint_tag: &str) -> Result> { + async fn new( + agent_id: &str, + entrypoint_tag: &str + ) -> Result> { let client = RunAgentClient::new(agent_id, entrypoint_tag, true).await?; Ok(Self { client, @@ -725,11 +772,13 @@ impl CachedAgent { } fn cache_key(&self, params: &[(&str, serde_json::Value)]) -> String { - // Simple cache key generation (in production, use a proper hash) format!("{:?}", params) } - async fn run(&self, params: &[(&str, serde_json::Value)]) -> Result> { + async fn run( + &self, + params: &[(&str, serde_json::Value)] + ) -> Result> { let cache_key = self.cache_key(params); // Check cache @@ -740,8 +789,8 @@ impl CachedAgent { } } - // Call agent - let result = self.client.run(params).await?; + // Execute agent + let result = self.client.run(params).await?.to_string(); // Store in cache { @@ -766,13 +815,17 @@ async fn main() -> Result<(), Box> { ### Unit Testing +Test your agent interactions: + ```rust use runagent::client::RunAgentClient; use serde_json::json; #[tokio::test] async fn test_agent_client() { - let client = RunAgentClient::new("test_agent", "main", true).await.unwrap(); + let client = RunAgentClient::new("test_agent", "main", true) + .await + .unwrap(); let result = client.run(&[("message", json!("Hello"))]).await; assert!(result.is_ok()); @@ -780,7 +833,9 @@ async fn test_agent_client() { #[tokio::test] async fn test_agent_client_error() { - let client = RunAgentClient::new("invalid_agent", "main", true).await.unwrap(); + let client = RunAgentClient::new("invalid_agent", "main", true) + .await + .unwrap(); let result = client.run(&[("message", json!("Hello"))]).await; assert!(result.is_err()); @@ -789,6 +844,8 @@ async fn test_agent_client_error() { ### Integration Testing +Test complete workflows: + ```rust use runagent::client::RunAgentClient; use serde_json::json; @@ -796,7 +853,9 @@ use futures::StreamExt; #[tokio::test] async fn test_agent_integration() { - let client = RunAgentClient::new("test_agent", "main", true).await.unwrap(); + let client = RunAgentClient::new("test_agent", "main", true) + .await + .unwrap(); let result = client.run(&[("message", json!("Hello"))]).await; assert!(result.is_ok()); @@ -804,13 +863,17 @@ async fn test_agent_integration() { #[tokio::test] async fn test_agent_streaming() { - let client = RunAgentClient::new("test_agent", "streaming", true).await.unwrap(); + let client = RunAgentClient::new("test_agent", "streaming", true) + .await + .unwrap(); - let mut stream = client.run_stream(&[("message", json!("Hello"))]).await.unwrap(); - let mut response = String::new(); + let mut stream = client.run_stream(&[("message", json!("Hello"))]) + .await + .unwrap(); + let mut response = String::new(); while let Some(chunk) = stream.next().await { - response.push_str(&chunk.unwrap()); + response.push_str(&chunk.unwrap().to_string()); } assert!(!response.is_empty()); @@ -819,21 +882,13 @@ async fn test_agent_streaming() { ## Next Steps - - - Learn advanced async patterns for Rust - - - Deploy your Rust applications to production - - - Integrate with other programming languages - - - Optimize your Rust applications for production - - - - -**πŸŽ‰ Great work!** You've learned how to use RunAgent agents from Rust applications. The Rust SDK provides high-performance access with zero-cost abstractions and compile-time type safety! - +Continue your journey with RunAgent: + +- **[Async Patterns](/how-to/advanced-tasks)** - Learn advanced async patterns for Rust +- **[Production Deployment](/how-to/deployment/cloud-deployment)** - Deploy your Rust applications to production +- **[Multi-Language Integration](/tutorials/multi-language-wrapper)** - Integrate with other programming languages +- **[Performance Tuning](/explanation/production-considerations)** - Optimize your Rust applications + +## Summary + +You have learned how to use RunAgent agents from Rust applications. The Rust SDK provides high-performance access with zero-cost abstractions and compile-time type safety, making it ideal for production deployments requiring maximum performance and reliability. \ No newline at end of file diff --git a/docs/how-to/frameworks/ag2.mdx b/docs/how-to/frameworks/ag2.mdx new file mode 100644 index 0000000..b33b2da --- /dev/null +++ b/docs/how-to/frameworks/ag2.mdx @@ -0,0 +1,746 @@ +# AG2 Integration + +Deploy AG2 (AutoGen 2.0) multi-agent systems with RunAgent + +--- + +## Prerequisites + +- Basic understanding of AG2/AutoGen +- Completed [Deploy Your First Agent](/tutorials/deploy-your-first-agent) tutorial +- Python 3.8 or higher + +--- + +## Overview + +AG2 (AutoGen 2.0) is a framework for building multi-agent conversational systems with automated agent collaboration. RunAgent makes it easy to deploy AG2 agents and access them from any programming language while maintaining conversation flow. + +--- + +## Installation & Setup + +### 1. Install AG2 + +```bash +pip install ag2>=0.9.6 +``` + +### 2. Set Environment Variables + +AG2 requires API keys for LLM providers: + +```bash +export OPENAI_API_KEY=your_openai_api_key_here +``` + +### 3. Quick Start with RunAgent + +```bash +runagent init my-ag2-agent --framework ag2 +cd my-ag2-agent +``` + +--- + +## Quick Start + +### 1. Project Structure + +After initialization, your project will have: + +``` +my-ag2-agent/ +β”œβ”€β”€ conversation.py # Main agent code +β”œβ”€β”€ .env # Environment variables +β”œβ”€β”€ requirements.txt # Python dependencies +└── runagent.config.json # RunAgent configuration +``` + +### 2. Configuration + +The generated `runagent.config.json`: + +```json +{ + "agent_name": "my-ag2-agent", + "description": "AG2 multi-agent conversation system", + "framework": "ag2", + "version": "1.0.0", + "agent_architecture": { + "entrypoints": [ + { + "file": "conversation.py", + "module": "invoke", + "tag": "ag2_invoke" + }, + { + "file": "conversation.py", + "module": "stream", + "tag": "ag2_stream" + } + ] + } +} +``` + +### 3. Create `.env` File + +```bash +OPENAI_API_KEY=your_openai_api_key_here +``` + +--- + +## Basic AG2 Agent + +Here's a simple AG2 agent with two conversational agents: + +```python +# conversation.py +from autogen import ConversableAgent, LLMConfig + +# Configure LLM using environment variables +llm_config = LLMConfig( + api_type="openai", + model="gpt-4o-mini" +) + +# Create agents within the LLM config context +with llm_config: + assistant = ConversableAgent( + name="assistant", + system_message="You are a helpful assistant that responds concisely and accurately.", + ) + + fact_checker = ConversableAgent( + name="fact_checker", + system_message="You are a fact-checking assistant. Verify claims and provide accurate corrections when needed.", + ) + + +def invoke(message: str, max_turns: int = 5): + """ + Non-streaming conversation between assistant and fact checker. + + Args: + message: The user's message to start the conversation + max_turns: Maximum number of conversation turns (default: 5) + + Returns: + Conversation result with chat history + """ + try: + result = assistant.initiate_chat( + recipient=fact_checker, + message=message, + max_turns=max_turns + ) + + return { + "status": "success", + "result": result, + "message": "Conversation completed successfully" + } + + except Exception as e: + return { + "status": "error", + "error": str(e), + "message": f"Error in AG2 conversation: {str(e)}" + } + + +def stream(message: str, max_turns: int = 5): + """ + Streaming conversation between assistant and fact checker. + + Args: + message: The user's message to start the conversation + max_turns: Maximum number of conversation turns (default: 5) + + Yields: + Conversation events as they occur + """ + try: + # Start the conversation + response = assistant.run( + recipient=fact_checker, + message=message, + max_turns=max_turns + ) + + # Stream events + for event in response.events: + if event.type == "text": + yield { + "type": "text", + "content": event.content, + "sender": getattr(event, 'sender', 'unknown') + } + else: + yield { + "type": event.type, + "data": event.model_dump() if hasattr(event, 'model_dump') else str(event) + } + + except Exception as e: + yield { + "type": "error", + "error": str(e), + "message": f"Error in AG2 streaming: {str(e)}" + } +``` + +--- + +## Advanced AG2 Patterns + +### 1. Multi-Agent Collaboration + +```python +# advanced_conversation.py +from autogen import ConversableAgent, LLMConfig + +llm_config = LLMConfig(api_type="openai", model="gpt-4o-mini") + +with llm_config: + # Research agent + researcher = ConversableAgent( + name="researcher", + system_message="You are a research specialist. Gather and present factual information.", + ) + + # Analyst agent + analyst = ConversableAgent( + name="analyst", + system_message="You are a data analyst. Analyze information and provide insights.", + ) + + # Writer agent + writer = ConversableAgent( + name="writer", + system_message="You are a technical writer. Synthesize information into clear, concise reports.", + ) + + +def research_workflow(topic: str, max_turns: int = 10): + """ + Multi-agent research workflow. + + Flow: User β†’ Researcher β†’ Analyst β†’ Writer + """ + try: + # Stage 1: Research + research_result = researcher.initiate_chat( + recipient=analyst, + message=f"Research the following topic: {topic}", + max_turns=max_turns // 2 + ) + + # Stage 2: Analysis and Writing + final_result = analyst.initiate_chat( + recipient=writer, + message="Based on the research, create a comprehensive report.", + max_turns=max_turns // 2 + ) + + return { + "status": "success", + "research": research_result, + "final_report": final_result, + "topic": topic + } + + except Exception as e: + return { + "status": "error", + "error": str(e), + "topic": topic + } +``` + +### 2. Agent with Custom Tools + +```python +# tool_agent.py +from autogen import ConversableAgent, LLMConfig +from typing import Annotated + +llm_config = LLMConfig(api_type="openai", model="gpt-4o-mini") + + +def calculate( + expression: Annotated[str, "Mathematical expression to evaluate"] +) -> str: + """Calculate a mathematical expression.""" + try: + # Safe evaluation with limited scope + import math + allowed = { + "abs": abs, "round": round, "min": min, "max": max, + "sum": sum, "pow": pow, **{k: v for k, v in math.__dict__.items() if not k.startswith("__")} + } + result = eval(expression, {"__builtins__": {}}, allowed) + return f"Result: {result}" + except Exception as e: + return f"Error: {str(e)}" + + +def search_info( + query: Annotated[str, "Search query"] +) -> str: + """Search for information (mock implementation).""" + # In production, integrate with actual search API + return f"Search results for '{query}': [Mock data - integrate with real search API]" + + +with llm_config: + # Tool-enabled assistant + assistant = ConversableAgent( + name="assistant", + system_message="You are a helpful assistant with access to tools. Use them when appropriate.", + ) + + # Register tools + assistant.register_function(calculate) + assistant.register_function(search_info) + + # User proxy + user_proxy = ConversableAgent( + name="user", + system_message="You represent the user.", + human_input_mode="NEVER", # Automated for RunAgent + ) + + +def invoke_with_tools(message: str, max_turns: int = 5): + """Conversation with tool access.""" + try: + result = user_proxy.initiate_chat( + recipient=assistant, + message=message, + max_turns=max_turns + ) + + return { + "status": "success", + "result": result, + "tools_used": ["calculate", "search_info"] + } + + except Exception as e: + return { + "status": "error", + "error": str(e) + } +``` + +### 3. Conditional Conversation Flow + +```python +# conditional_flow.py +from autogen import ConversableAgent, LLMConfig + +llm_config = LLMConfig(api_type="openai", model="gpt-4o-mini") + +with llm_config: + classifier = ConversableAgent( + name="classifier", + system_message="Classify user queries into categories: technical, general, urgent.", + ) + + technical_expert = ConversableAgent( + name="technical_expert", + system_message="You are a technical expert. Provide detailed technical answers.", + ) + + general_assistant = ConversableAgent( + name="general_assistant", + system_message="You are a general assistant. Provide helpful, friendly responses.", + ) + + +def smart_route(message: str, max_turns: int = 5): + """Route queries to appropriate agents based on classification.""" + try: + # First, classify the query + classification = classifier.initiate_chat( + recipient=general_assistant, + message=f"Classify this query: {message}", + max_turns=2 + ) + + # Route based on classification (simplified) + if "technical" in str(classification).lower(): + agent = technical_expert + else: + agent = general_assistant + + # Handle the query with appropriate agent + result = agent.initiate_chat( + recipient=classifier, + message=message, + max_turns=max_turns + ) + + return { + "status": "success", + "classification": str(classification), + "result": result, + "handler": agent.name + } + + except Exception as e: + return { + "status": "error", + "error": str(e) + } +``` + +--- + +## Testing Your AG2 Agent + +### Python Client + +```python +# test_ag2.py +from runagent import RunAgentClient + +# Test non-streaming +client = RunAgentClient( + agent_id="your_agent_id_here", + entrypoint_tag="ag2_invoke", + local=True +) + +result = client.run( + message="The solar system has 8 planets.", + max_turns=3 +) + +print(f"Conversation result: {result}") + +# Test streaming +stream_client = RunAgentClient( + agent_id="your_agent_id_here", + entrypoint_tag="ag2_stream", + local=True +) + +print("\nStreaming conversation:") +for chunk in stream_client.run( + message="Explain quantum computing", + max_turns=4 +): + if chunk.get("type") == "text": + print(f"{chunk.get('sender', 'Agent')}: {chunk.get('content')}") +``` + +### JavaScript Client + +```javascript +// test_ag2.js +import { RunAgentClient } from 'runagent'; + +const client = new RunAgentClient({ + agentId: 'your_agent_id_here', + entrypointTag: 'ag2_invoke', + local: true +}); + +await client.initialize(); + +// Test conversation +const result = await client.run({ + message: 'What is artificial intelligence?', + max_turns: 3 +}); + +console.log('Result:', result); + +// Test streaming +const streamClient = new RunAgentClient({ + agentId: 'your_agent_id_here', + entrypointTag: 'ag2_stream', + local: true +}); + +await streamClient.initialize(); + +for await (const chunk of streamClient.run({ + message: 'Explain machine learning', + max_turns: 3 +})) { + if (chunk.type === 'text') { + console.log(`${chunk.sender}: ${chunk.content}`); + } +} +``` + +### Go Client + +```go +package main + +import ( + "context" + "fmt" + "github.com/runagent-dev/runagent-go/pkg/client" + "encoding/json" +) + +func main() { + client, _ := client.New( + "your_agent_id_here", + "ag2_invoke", + true, + ) + defer client.Close() + + ctx := context.Background() + + result, _ := client.Run(ctx, map[string]interface{}{ + "message": "Explain neural networks", + "max_turns": 3, + }) + + fmt.Printf("Result: %v\n", result) +} +``` + +--- + +## Configuration Examples + +### Single Conversation Agent + +```json +{ + "agent_name": "ag2-assistant", + "framework": "ag2", + "agent_architecture": { + "entrypoints": [ + { + "file": "conversation.py", + "module": "invoke", + "tag": "ag2_invoke" + }, + { + "file": "conversation.py", + "module": "stream", + "tag": "ag2_stream" + } + ] + } +} +``` + +### Multi-Agent Workflow + +```json +{ + "agent_name": "ag2-research-team", + "framework": "ag2", + "agent_architecture": { + "entrypoints": [ + { + "file": "conversation.py", + "module": "invoke", + "tag": "simple_chat" + }, + { + "file": "advanced_conversation.py", + "module": "research_workflow", + "tag": "research" + }, + { + "file": "tool_agent.py", + "module": "invoke_with_tools", + "tag": "tools" + } + ] + } +} +``` + +--- + +## Best Practices + +### 1. **Agent Design** +- Keep system messages clear and specific +- Define roles explicitly for each agent +- Use appropriate max_turns to prevent infinite loops + +### 2. **Conversation Management** +- Set reasonable max_turns (typically 3-10) +- Handle conversation state appropriately +- Implement timeout mechanisms for long conversations + +### 3. **Error Handling** +- Always wrap AG2 operations in try-catch blocks +- Return structured error responses +- Log conversation failures for debugging + +### 4. **Tool Integration** +- Register tools explicitly with agents +- Use type annotations for tool parameters +- Implement safe tool execution with proper validation + +### 5. **Performance** +- Reuse agent instances when possible +- Monitor conversation length and token usage +- Implement caching for repeated queries + +--- + +## Common Patterns + +### Fact-Checking Pattern +Use multiple agents to verify information: +```python +assistant β†’ fact_checker β†’ validator +``` + +### Research Pattern +Multi-stage information gathering: +```python +researcher β†’ analyst β†’ writer +``` + +### Routing Pattern +Direct queries to specialized agents: +```python +classifier β†’ [technical_expert | general_assistant] +``` + +### Tool-Augmented Pattern +Agents with external capabilities: +```python +assistant + [calculator, search, database] +``` + +--- + +## Troubleshooting + +### Common Issues + +**1. API Key Not Found** +- **Solution**: Ensure `OPENAI_API_KEY` is set in environment +- Check `.env` file exists and is loaded +- Verify key is valid and has credits + +**2. Conversation Hangs** +- **Solution**: Set appropriate `max_turns` parameter +- Reduce conversation complexity +- Implement timeout mechanisms + +**3. Agent Not Responding** +- **Solution**: Check system messages are clear +- Verify LLM config is correct +- Review agent initialization code + +**4. Tool Execution Fails** +- **Solution**: Verify tool registration +- Check tool function signatures +- Ensure type annotations are correct + +**5. Streaming Not Working** +- **Solution**: Use `assistant.run()` instead of `initiate_chat()` +- Check event handling in streaming loop +- Verify client supports streaming + +### Debug Tips + +Enable verbose logging: + +```python +import logging +logging.basicConfig(level=logging.DEBUG) + +def invoke(message, max_turns): + print(f"Debug: Starting conversation with message: {message}") + print(f"Debug: Max turns: {max_turns}") + # ... rest of code +``` + +Test conversation locally: + +```python +# test_local.py +from conversation import invoke + +result = invoke("Test message", max_turns=2) +print(f"Result: {result}") +``` + +--- + +## Performance Optimization + +### 1. **Agent Reuse** +Create agents once and reuse: + +```python +# Global agent instances +_assistant = None +_fact_checker = None + +def get_agents(): + global _assistant, _fact_checker + + if _assistant is None: + llm_config = LLMConfig(api_type="openai", model="gpt-4o-mini") + with llm_config: + _assistant = ConversableAgent(...) + _fact_checker = ConversableAgent(...) + + return _assistant, _fact_checker +``` + +### 2. **Conversation Limits** +Set appropriate limits: + +```python +def invoke(message, max_turns=5): + # Reasonable max_turns prevents runaway conversations + if max_turns > 20: + max_turns = 20 + # ... rest of code +``` + +### 3. **Caching** +Implement response caching for repeated queries: + +```python +from functools import lru_cache + +@lru_cache(maxsize=100) +def invoke_cached(message, max_turns): + return invoke(message, max_turns) +``` + +--- + +## Next Steps + +- **[Advanced Patterns](/how-to/advanced-tasks)** - Learn advanced AG2 patterns +- **[Production Deployment](/how-to/deployment/cloud-deployment)** - Deploy to production +- **[Multi-Language Access](/tutorials/multi-language-wrapper)** - Access from different languages +- **[Performance Tuning](/explanation/production-considerations)** - Optimize for production + +--- + +## Additional Resources + +- [AG2 Documentation](https://ag2.ai/docs) +- [AG2 GitHub](https://github.com/ag2ai/ag2) +- [RunAgent Discord Community](https://discord.gg/Q9P9AdHVHz) +- [RunAgent Documentation](https://docs.run-agent.ai) + +--- + +**πŸŽ‰ Great work!** You've learned how to deploy AG2 multi-agent systems with RunAgent. AG2's collaborative agent architecture combined with RunAgent's multi-language access creates powerful, flexible conversational AI systems! \ No newline at end of file diff --git a/docs/how-to/frameworks/agno.mdx b/docs/how-to/frameworks/agno.mdx index c577a1a..94f9e40 100644 --- a/docs/how-to/frameworks/agno.mdx +++ b/docs/how-to/frameworks/agno.mdx @@ -1,7 +1,6 @@ --- -title: 'Agno Integration' +title: 'Agno' description: 'Deploy Agno AI agents with RunAgent' -icon: 'terminal' --- diff --git a/docs/how-to/frameworks/crewai.mdx b/docs/how-to/frameworks/crewai.mdx index 46aa985..fe18fcf 100644 --- a/docs/how-to/frameworks/crewai.mdx +++ b/docs/how-to/frameworks/crewai.mdx @@ -1,7 +1,6 @@ --- -title: 'CrewAI Integration' +title: 'CrewAI' description: 'Deploy CrewAI multi-agent systems with RunAgent' -icon: 'users' --- diff --git a/docs/how-to/frameworks/custom.mdx b/docs/how-to/frameworks/custom.mdx index c2c365c..e51e30e 100644 --- a/docs/how-to/frameworks/custom.mdx +++ b/docs/how-to/frameworks/custom.mdx @@ -1,7 +1,6 @@ --- -title: 'Custom Framework Integration' +title: 'Custom Framework' description: 'Deploy agents built with any Python-based framework' -icon: 'person-running-fast' --- diff --git a/docs/how-to/frameworks/langgraph.mdx b/docs/how-to/frameworks/langgraph.mdx index 9e0e7e8..7531bff 100644 --- a/docs/how-to/frameworks/langgraph.mdx +++ b/docs/how-to/frameworks/langgraph.mdx @@ -1,7 +1,6 @@ --- -title: 'LangGraph Integration' +title: 'LangGraph' description: 'Deploy LangGraph agents with RunAgent' -icon: 'diagram-project' --- import NeedHelp from '/snippets/need-help.mdx'; diff --git a/docs/how-to/frameworks/letta.mdx b/docs/how-to/frameworks/letta.mdx index 000ad5d..d1f4e37 100644 --- a/docs/how-to/frameworks/letta.mdx +++ b/docs/how-to/frameworks/letta.mdx @@ -1,20 +1,67 @@ ---- -title: 'Letta Integration' -description: 'Deploy Letta memory-enabled agents with RunAgent' -icon: 'message' +# Letta Integration + +Deploy Letta memory-enabled agents with RunAgent + --- - - **Prerequisites**: Basic understanding of Letta and completed [Deploy Your First Agent](/tutorials/deploy-your-first-agent) tutorial - +## Prerequisites + +- Basic understanding of Letta +- Completed [Deploy Your First Agent](/tutorials/deploy-your-first-agent) tutorial +- Python 3.8 or higher + +--- ## Overview Letta is a framework for building conversational AI agents with persistent memory and context awareness. RunAgent makes it easy to deploy Letta agents and access them from any programming language while maintaining conversation state. -## Quick Start +--- + +## Installation & Setup + +### 1. Install Letta Server + +```bash +pip install -U letta +``` + +### 2. Set Environment Variables + +Letta requires API keys for LLM providers. Set them before starting the server: + +```bash +export OPENAI_API_KEY=your_openai_api_key_here +``` + +### 3. Start Letta Server -### 1. Create a Letta Agent +The Letta server must be running before deploying RunAgent agents: + +```bash +letta server +``` + +The server will start on `http://localhost:8283`. Keep this terminal window open. + +**Note**: If you need to use a different port, you can specify it with: +```bash +letta server --port 8284 +``` + +### 4. Install Letta Client (Optional) + +For Python SDK usage, install the client package: + +```bash +pip install letta-client +``` + +--- + +## Quick Start with RunAgent + +### 1. Create a Letta Agent Project ```bash runagent init my-letta-agent --framework letta @@ -27,7 +74,22 @@ cd my-letta-agent pip install -r requirements.txt ``` -### 3. Configure Your Agent +The generated `requirements.txt` will include: +``` +letta-client +python-dotenv +``` + +### 3. Configure Environment + +Create a `.env` file in your project directory: + +```bash +OPENAI_API_KEY=your_openai_api_key_here +LETTA_SERVER_URL=http://localhost:8283 +``` + +### 4. Review Configuration The generated `runagent.config.json` will be pre-configured for Letta: @@ -36,367 +98,355 @@ The generated `runagent.config.json` will be pre-configured for Letta: "agent_name": "my-letta-agent", "description": "Letta conversational agent with memory", "framework": "letta", + "version": "1.0.0", "agent_architecture": { "entrypoints": [ { "file": "agent.py", - "module": "letta_agent", + "module": "letta_run", "tag": "chat" + }, + { + "file": "agent.py", + "module": "letta_run_stream", + "tag": "chat_stream" } ] } } ``` +--- + ## Basic Letta Agent -Here's a simple Letta agent that demonstrates memory and conversation capabilities: - -```python agent.py -from letta import Agent, Memory, Tool -from typing import Dict, Any, List, Optional -import json -from datetime import datetime - -# Initialize memory -memory = Memory() - -# Define tools -def get_weather(location: str) -> str: - """Get weather information for a location""" - # Simulate weather API call - return f"Weather in {location}: 72Β°F, partly cloudy" - -def get_time() -> str: - """Get current time""" - return datetime.now().strftime("%Y-%m-%d %H:%M:%S") - -def remember_fact(fact: str) -> str: - """Remember a fact about the user""" - memory.store(fact) - return f"I'll remember that: {fact}" - -def recall_memory(query: str) -> str: - """Recall information from memory""" - results = memory.search(query) - if results: - return f"From my memory: {', '.join(results)}" - else: - return "I don't have that information in my memory." - -# Create tools -tools = [ - Tool(name="get_weather", function=get_weather, description="Get weather for a location"), - Tool(name="get_time", function=get_time, description="Get current time"), - Tool(name="remember_fact", function=remember_fact, description="Remember a fact about the user"), - Tool(name="recall_memory", function=recall_memory, description="Recall information from memory") -] +Here's a simple Letta agent demonstrating conversational capabilities: + +```python +# agent.py +import os +from dotenv import load_dotenv +from letta_client import CreateBlock, Letta + +# Load environment variables +load_dotenv() -# Create Letta agent -letta_agent = Agent( - name="Assistant", - memory=memory, - tools=tools, - system_message="You are a helpful assistant with memory capabilities. You can remember facts about users and recall them later." -) -def letta_agent(message: str, user_id: str = "default", session_id: str = None) -> Dict[str, Any]: +def _extract_message_from_input(*input_args, **input_kwargs) -> str: + """Extract message text from various input formats""" + # Try direct message parameter + if input_kwargs.get("message"): + return str(input_kwargs["message"]) + + # Try messages list format + if input_kwargs.get("messages"): + messages = input_kwargs["messages"] + if isinstance(messages, list) and messages: + last_message = messages[-1] + if isinstance(last_message, dict) and "content" in last_message: + return last_message["content"] + + # Try first positional argument as string + if input_args and isinstance(input_args[0], str): + return input_args[0] + + # Try first positional argument as dict with messages + if input_args and isinstance(input_args[0], dict): + data = input_args[0] + if "messages" in data: + messages = data["messages"] + if isinstance(messages, list) and messages: + last_message = messages[-1] + if isinstance(last_message, dict) and "content" in last_message: + return last_message["content"] + if "message" in data: + return str(data["message"]) + + return "Hello" + + +def letta_run(*input_args, **input_kwargs): """Main entrypoint for the Letta agent""" try: - # Create or get session - if not session_id: - session_id = f"session_{user_id}_{datetime.now().strftime('%Y%m%d_%H%M%S')}" + # Initialize Letta client + letta_url = os.getenv("LETTA_SERVER_URL", "http://localhost:8283") + client = Letta(base_url=letta_url) - # Process message - response = letta_agent.process_message( - message=message, - user_id=user_id, - session_id=session_id + # Create memory blocks + memory_blocks = [ + CreateBlock( + label="human", + value="You are talking to a user through RunAgent framework", + ), + CreateBlock( + label="persona", + value="You are a helpful AI assistant. Be friendly and concise.", + ), + ] + + # Create agent + agent = client.agents.create( + name=f"runagent-letta-{os.getpid()}", + memory_blocks=memory_blocks, + system="You are a helpful AI assistant integrated with RunAgent. Respond naturally and helpfully.", + model="openai/gpt-4o-mini", + embedding="openai/text-embedding-ada-002", + include_base_tools=True ) + + print(f"βœ… Letta agent created with ID: {agent.id}") - return { - "response": response, - "user_id": user_id, - "session_id": session_id, - "timestamp": datetime.now().isoformat(), - "status": "success" - } + # Extract message from input + message = _extract_message_from_input(*input_args, **input_kwargs) + + # Send message to Letta agent + response = client.agents.messages.create( + agent_id=agent.id, + messages=[{ + "role": "user", + "content": message + }] + ) + + # Clean up agent after use + try: + client.agents.delete(agent.id) + print(f"πŸ—‘οΈ Cleaned up agent: {agent.id}") + except: + pass + + return response except Exception as e: - return { - "response": f"Error: {str(e)}", - "user_id": user_id, - "session_id": session_id, - "timestamp": datetime.now().isoformat(), - "status": "error" - } + return {"error": f"Letta execution error: {str(e)}"} + + +def letta_run_stream(*input_args, **input_kwargs): + """Streaming entrypoint for the Letta agent""" + try: + # Initialize Letta client + letta_url = os.getenv("LETTA_SERVER_URL", "http://localhost:8283") + client = Letta(base_url=letta_url) + + # Create memory blocks + memory_blocks = [ + CreateBlock( + label="human", + value="You are talking to a user through RunAgent framework", + ), + CreateBlock( + label="persona", + value="You are a helpful AI assistant. Be friendly and concise.", + ), + ] + + # Create agent + agent = client.agents.create( + name=f"runagent-letta-stream-{os.getpid()}", + memory_blocks=memory_blocks, + system="You are a helpful AI assistant integrated with RunAgent. Respond naturally and helpfully.", + model="openai/gpt-4o-mini", + embedding="openai/text-embedding-ada-002", + include_base_tools=True + ) + + print(f"βœ… Letta streaming agent created with ID: {agent.id}") + + # Extract message from input + message = _extract_message_from_input(*input_args, **input_kwargs) + + # Create streaming response + stream = client.agents.messages.create_stream( + agent_id=agent.id, + messages=[{ + "role": "user", + "content": message + }], + stream_tokens=True, + ) + + # Yield chunks + for chunk in stream: + yield chunk + + # Clean up agent after streaming completes + try: + client.agents.delete(agent.id) + print(f"πŸ—‘οΈ Cleaned up streaming agent: {agent.id}") + except: + pass + + except Exception as e: + yield {"error": f"Letta streaming error: {str(e)}"} ``` -## Advanced Letta Patterns +--- + +## Advanced: Letta Agent with Custom Tools + +Here's an example with custom tools (keyword extraction, RAG): -### 1. Multi-Session Memory Management +```python +# agent.py (Advanced with Tools) +import os +from typing import Dict +from dotenv import load_dotenv +from letta_client import CreateBlock, Letta + +# Load environment variables +load_dotenv() -```python multi_session_agent.py -from letta import Agent, Memory, Tool -from typing import Dict, Any, List, Optional -import json -from datetime import datetime, timedelta -class SessionManager: - def __init__(self): - self.sessions = {} - self.session_timeout = timedelta(hours=24) +def _register_tools(client: Letta) -> Dict[str, str]: + """Register custom tools with Letta client""" + from keyword_tool import extract_keywords + from rag_tool import rag_tool - def get_session(self, session_id: str) -> Memory: - """Get or create a session""" - if session_id not in self.sessions: - self.sessions[session_id] = Memory() - return self.sessions[session_id] + tool_fns = [extract_keywords, rag_tool] + tool_map: Dict[str, str] = {} - def cleanup_expired_sessions(self): - """Clean up expired sessions""" - current_time = datetime.now() - expired_sessions = [] - - for session_id, memory in self.sessions.items(): - if hasattr(memory, 'last_accessed'): - if current_time - memory.last_accessed > self.session_timeout: - expired_sessions.append(session_id) - - for session_id in expired_sessions: - del self.sessions[session_id] + for fn in tool_fns: + try: + print(f" πŸ“ Registering {fn.__name__}...") + tool = client.tools.upsert_from_function(func=fn) + tool_map[tool.name] = tool.id + print(f" βœ… Successfully registered: {tool.name}") + except Exception as e: + print(f" ❌ Failed to register tool {fn.__name__}: {e}") + + return tool_map -# Initialize session manager -session_manager = SessionManager() -def multi_session_agent(message: str, user_id: str, session_id: str) -> Dict[str, Any]: - """Multi-session Letta agent with memory management""" +def letta_run(message: str): + """Advanced Letta agent with custom tools""" try: - # Get session memory - memory = session_manager.get_session(session_id) - memory.last_accessed = datetime.now() + # Initialize Letta client + letta_url = os.getenv("LETTA_SERVER_URL", "http://localhost:8283") + client = Letta(base_url=letta_url) + + # Register tools + print("πŸ”§ Registering tools...") + tool_map = _register_tools(client) - # Create agent with session memory - agent = Agent( - name="MultiSessionAssistant", - memory=memory, - tools=tools, - system_message="You are a helpful assistant with persistent memory across sessions." + # Create memory blocks + memory_blocks = [ + CreateBlock( + label="human", + value="You are talking to a user through RunAgent framework", + ), + CreateBlock( + label="persona", + value="You are an expert assistant with access to research tools.", + ), + ] + + # Create agent with tools + agent = client.agents.create( + name=f"runagent-letta-tools-{os.getpid()}", + memory_blocks=memory_blocks, + system="You are an expert assistant with access to keyword extraction and RAG tools. When users ask for research or information, use the rag_tool. When they need keywords extracted, use extract_keywords.", + model="openai/gpt-4o-mini", + embedding="openai/text-embedding-ada-002", + tool_ids=list(tool_map.values()), + include_base_tools=True ) + + print(f"βœ… Letta agent created with {len(tool_map)} tools") - # Process message - response = agent.process_message( - message=message, - user_id=user_id, - session_id=session_id + # Send message to Letta agent + response = client.agents.messages.create( + agent_id=agent.id, + messages=[{ + "role": "user", + "content": message + }] ) - # Cleanup expired sessions - session_manager.cleanup_expired_sessions() + # Clean up + try: + client.agents.delete(agent.id) + except: + pass - return { - "response": response, - "user_id": user_id, - "session_id": session_id, - "memory_size": len(memory.get_all()), - "timestamp": datetime.now().isoformat(), - "status": "success" - } + return response except Exception as e: - return { - "response": f"Error: {str(e)}", - "user_id": user_id, - "session_id": session_id, - "timestamp": datetime.now().isoformat(), - "status": "error" - } + return {"error": f"Letta execution error: {str(e)}"} ``` -### 2. Streaming Letta Agent +### Custom Tool Example: Keyword Extraction -```python streaming_letta.py -from typing import Iterator, Dict, Any -from letta import Agent, Memory, Tool +```python +# keyword_tool.py +def extract_keywords(text: str, num_keywords: int = 5) -> dict: + """ + Extract keywords from input text. -def streaming_letta_agent(message: str, user_id: str = "default") -> Iterator[str]: - """Streaming Letta agent with memory""" - yield f"🧠 Letta Agent starting conversation with {user_id}...\n\n" - - # Initialize memory and agent - memory = Memory() - agent = Agent( - name="StreamingAssistant", - memory=memory, - tools=tools, - system_message="You are a helpful assistant with memory capabilities." - ) - - yield f"πŸ’­ Processing message: {message}\n\n" - - # Simulate thinking process - yield "πŸ” Analyzing message...\n" - yield "🧠 Checking memory for context...\n" - yield "πŸ› οΈ Selecting appropriate tools...\n" - yield "πŸ’¬ Generating response...\n\n" - - # Process message - response = agent.process_message( - message=message, - user_id=user_id, - session_id=f"stream_{user_id}" - ) - - yield f"πŸ€– Response: {response}\n\n" - yield f"πŸ’Ύ Memory updated with {len(memory.get_all())} items\n" - yield "βœ… Conversation complete!" -``` + Args: + text (str): The input text to extract keywords from + num_keywords (int): Number of keywords to extract (default: 5) -### 3. Specialized Letta Agents + Returns: + dict: Dictionary containing the extracted keywords + """ + from langchain_openai import ChatOpenAI + from langchain.prompts import PromptTemplate -```python specialized_agents.py -from letta import Agent, Memory, Tool -from typing import Dict, Any, List + try: + # Initialize the language model + llm = ChatOpenAI(model_name="gpt-4o-mini", temperature=0.2) -# Customer Support Agent -def customer_support_agent(message: str, user_id: str) -> Dict[str, Any]: - """Customer support agent with memory""" - memory = Memory() - - # Customer support tools - support_tools = [ - Tool(name="check_order", function=lambda x: f"Order {x} status: Shipped", description="Check order status"), - Tool(name="create_ticket", function=lambda x: f"Ticket created: {x}", description="Create support ticket"), - Tool(name="escalate_issue", function=lambda x: f"Issue escalated: {x}", description="Escalate to human agent") - ] - - agent = Agent( - name="CustomerSupport", - memory=memory, - tools=support_tools, - system_message="You are a customer support agent. Help users with their issues and remember their preferences." - ) - - response = agent.process_message(message, user_id, f"support_{user_id}") - - return { - "response": response, - "agent_type": "customer_support", - "user_id": user_id, - "status": "success" - } - -# Personal Assistant Agent -def personal_assistant_agent(message: str, user_id: str) -> Dict[str, Any]: - """Personal assistant agent with memory""" - memory = Memory() - - # Personal assistant tools - pa_tools = [ - Tool(name="schedule_meeting", function=lambda x: f"Meeting scheduled: {x}", description="Schedule a meeting"), - Tool(name="set_reminder", function=lambda x: f"Reminder set: {x}", description="Set a reminder"), - Tool(name="get_calendar", function=lambda x: "Calendar events for today", description="Get calendar events") - ] - - agent = Agent( - name="PersonalAssistant", - memory=memory, - tools=pa_tools, - system_message="You are a personal assistant. Help users manage their tasks and remember their preferences." - ) - - response = agent.process_message(message, user_id, f"pa_{user_id}") - - return { - "response": response, - "agent_type": "personal_assistant", - "user_id": user_id, - "status": "success" - } - -# Educational Agent -def educational_agent(message: str, user_id: str) -> Dict[str, Any]: - """Educational agent with memory""" - memory = Memory() - - # Educational tools - edu_tools = [ - Tool(name="explain_concept", function=lambda x: f"Explanation of {x}", description="Explain a concept"), - Tool(name="quiz_question", function=lambda x: f"Quiz question about {x}", description="Generate quiz question"), - Tool(name="track_progress", function=lambda x: f"Progress tracked: {x}", description="Track learning progress") - ] - - agent = Agent( - name="EducationalAssistant", - memory=memory, - tools=edu_tools, - system_message="You are an educational assistant. Help users learn and remember their progress." - ) - - response = agent.process_message(message, user_id, f"edu_{user_id}") - - return { - "response": response, - "agent_type": "educational", - "user_id": user_id, - "status": "success" - } -``` + # Create prompt template + prompt_template = """ + Extract exactly {num_keywords} relevant keywords from the following text. + Return them as a comma-separated list without numbering or bullet points. + Only include the keywords, no additional information or explanation. -## Configuration for Multiple Agents + Text: {input_text} -Update your `runagent.config.json` to include multiple Letta agents: + Keywords: + """ -```json -{ - "agent_name": "advanced-letta-agent", - "description": "Advanced Letta multi-agent system", - "framework": "letta", - "agent_architecture": { - "entrypoints": [ - { - "file": "agent.py", - "module": "letta_agent", - "tag": "chat" - }, - { - "file": "multi_session_agent.py", - "module": "multi_session_agent", - "tag": "multi_session" - }, - { - "file": "streaming_letta.py", - "module": "streaming_letta_agent", - "tag": "streaming" - }, - { - "file": "specialized_agents.py", - "module": "customer_support_agent", - "tag": "support" - }, - { - "file": "specialized_agents.py", - "module": "personal_assistant_agent", - "tag": "assistant" - }, - { - "file": "specialized_agents.py", - "module": "educational_agent", - "tag": "education" - } - ] - } -} + prompt = PromptTemplate( + input_variables=["input_text", "num_keywords"], + template=prompt_template + ) + + # Create chain + chain = prompt | llm + + # Run the chain + result = chain.invoke({"input_text": text, "num_keywords": num_keywords}) + + # Extract content + content = result.content if hasattr(result, 'content') else str(result) + keywords = [kw.strip() for kw in content.split(',')] + + return { + "status": "success", + "keywords": keywords, + "message": f"Successfully extracted {len(keywords)} keywords." + } + + except Exception as e: + return { + "status": "error", + "keywords": [], + "message": f"Error extracting keywords: {str(e)}" + } ``` +--- + ## Testing Your Letta Agent ### Python Client -```python test_letta.py +```python +# test_letta.py from runagent import RunAgentClient -# Connect to your Letta agent +# Ensure Letta server is running on port 8283 client = RunAgentClient( agent_id="your_agent_id_here", entrypoint_tag="chat", @@ -404,173 +454,300 @@ client = RunAgentClient( ) # Test basic conversation -result1 = client.run(message="Hello, my name is John", user_id="john_doe") -print(f"Response 1: {result1['response']}") +result = client.run(message="Hello, I'm interested in AI agents") +print(f"Response: {result}") -# Test memory recall -result2 = client.run(message="What's my name?", user_id="john_doe") -print(f"Response 2: {result2['response']}") +# Test with message dict format +result2 = client.run({ + "messages": [ + {"role": "user", "content": "What can you help me with?"} + ] +}) +print(f"Response 2: {result2}") +``` -# Test multi-session -multi_client = RunAgentClient( - agent_id="your_agent_id_here", - entrypoint_tag="multi_session", - local=True -) +### JavaScript Client -session_result = multi_client.run( - message="Remember I like pizza", - user_id="john_doe", - session_id="session_123" -) -print(f"Session Response: {session_result['response']}") +```javascript +// test_letta.js +import { RunAgentClient } from 'runagent'; + +const client = new RunAgentClient({ + agentId: 'your_agent_id_here', + entrypointTag: 'chat', + local: true +}); + +await client.initialize(); + +// Test conversation +const result = await client.run({ + message: 'Tell me about Letta agents' +}); + +console.log('Response:', result); +``` + +### Streaming Example + +```python +# test_letta_stream.py +from runagent import RunAgentClient -# Test streaming stream_client = RunAgentClient( agent_id="your_agent_id_here", - entrypoint_tag="streaming", + entrypoint_tag="chat_stream", local=True ) print("Streaming conversation:") -for chunk in stream_client.run(message="Tell me about the weather", user_id="john_doe"): +for chunk in stream_client.run(message="Explain the benefits of AI memory"): print(chunk, end="", flush=True) ``` -### JavaScript Client +--- -```javascript test_letta.js -import { RunAgentClient } from 'runagent'; +## Project Structure -const client = new RunAgentClient({ - agentId: 'your_agent_id_here', - entrypointTag: 'chat', - local: true -}); +``` +my-letta-agent/ +β”œβ”€β”€ agent.py # Main agent code +β”œβ”€β”€ keyword_tool.py # Optional: Custom keyword tool +β”œβ”€β”€ rag_tool.py # Optional: Custom RAG tool +β”œβ”€β”€ .env # Environment variables +β”œβ”€β”€ requirements.txt # Python dependencies +└── runagent.config.json # RunAgent configuration +``` -await client.initialize(); +--- -// Test conversation with memory -const result1 = await client.run({ - message: 'Hello, my name is John', - user_id: 'john_doe' -}); +## Configuration Options -console.log('Response 1:', result1.response); +### Multiple Entrypoints -const result2 = await client.run({ - message: "What's my name?", - user_id: 'john_doe' -}); +You can expose multiple entrypoints for different use cases: -console.log('Response 2:', result2.response); +```json +{ + "agent_name": "advanced-letta-agent", + "description": "Advanced Letta multi-entrypoint system", + "framework": "letta", + "agent_architecture": { + "entrypoints": [ + { + "file": "agent.py", + "module": "letta_run", + "tag": "chat" + }, + { + "file": "agent.py", + "module": "letta_run_stream", + "tag": "chat_stream" + }, + { + "file": "agent.py", + "module": "letta_run_with_tools", + "tag": "research" + } + ] + } +} ``` +--- + ## Best Practices ### 1. **Memory Management** -- Use appropriate memory storage strategies -- Implement memory cleanup for long-running agents -- Consider memory size limits +- Use appropriate memory blocks for context +- Clean up agents after use to prevent memory leaks +- Consider session-based memory for multi-user scenarios -### 2. **Session Handling** -- Implement proper session management -- Use unique session IDs -- Handle session timeouts - -### 3. **Tool Design** +### 2. **Tool Design** - Create focused, single-purpose tools -- Provide clear tool descriptions -- Handle tool errors gracefully +- Provide clear docstrings for tool functions +- Handle tool errors gracefully with try-catch blocks +- Return structured dictionaries from tools + +### 3. **Error Handling** +- Always wrap Letta operations in try-catch blocks +- Return meaningful error messages to users +- Log errors for debugging purposes + +### 4. **Server Management** +- Ensure Letta server is running before deploying agents +- Use environment variables for server URL configuration +- Monitor server logs for debugging + +### 5. **Agent Cleanup** +- Delete temporary agents after use +- Implement cleanup in both success and error paths +- Use context managers for resource management -### 4. **Conversation Flow** -- Design natural conversation patterns -- Handle context switching -- Implement conversation state management +--- ## Common Patterns - - - Use Letta's memory capabilities to maintain context across conversations. - - - Implement session management for multiple concurrent users. - - - Create domain-specific agents with specialized tools and memory. - - - Integrate external tools and APIs with Letta agents. - - +### Conversational Memory +Use Letta's memory blocks to maintain context across interactions: + +```python +memory_blocks = [ + CreateBlock( + label="human", + value="User preferences: prefers technical explanations, interested in AI" + ), + CreateBlock( + label="persona", + value="You are a technical AI expert. Provide detailed, accurate information." + ), +] +``` + +### Tool Integration +Register custom tools for specialized functionality: + +```python +def _register_tools(client: Letta) -> Dict[str, str]: + tool_fns = [search_tool, calculate_tool, analyze_tool] + tool_map = {} + + for fn in tool_fns: + tool = client.tools.upsert_from_function(func=fn) + tool_map[tool.name] = tool.id + + return tool_map +``` + +### Session Management +For multi-user scenarios, create agents per session: + +```python +def create_session_agent(user_id: str, session_id: str): + agent = client.agents.create( + name=f"user-{user_id}-session-{session_id}", + memory_blocks=load_user_memory(user_id), + # ... other configuration + ) + return agent +``` + +--- ## Troubleshooting ### Common Issues -1. **Memory Persistence** - - Check memory storage configuration - - Verify session management - - Monitor memory usage +**1. Connection Error: "Cannot connect to Letta server"** +- **Solution**: Ensure Letta server is running (`letta server`) +- Check the server URL in your `.env` file +- Verify the server is accessible at `http://localhost:8283` + +**2. API Key Error** +- **Solution**: Set `OPENAI_API_KEY` in your environment +- Verify the key is valid and has sufficient credits +- Check that the key is loaded in the Letta server environment + +**3. Tool Registration Fails** +- **Solution**: Ensure tool functions have proper docstrings +- Check that tool function signatures are compatible with Letta +- Verify all tool dependencies are installed -2. **Tool Execution** - - Verify tool definitions - - Check tool permissions - - Handle tool errors +**4. Memory Persistence Issues** +- **Solution**: Agents are ephemeral by default in this setup +- Implement custom session storage if needed +- Use Letta's built-in persistence features for production -3. **Session Management** - - Ensure unique session IDs - - Check session timeout settings - - Monitor session cleanup +**5. Streaming Not Working** +- **Solution**: Ensure you're using `create_stream` method +- Check that `stream_tokens=True` is set +- Verify the entrypoint is correctly configured for streaming ### Debug Tips +Enable verbose logging: + ```python -# Add debugging to your Letta agents -def debug_letta_agent(message: str, user_id: str) -> Dict[str, Any]: - print(f"Debug: Processing message from {user_id}: {message}") - - # Your agent logic here - - print(f"Debug: Response generated: {response}") - return result +import logging +logging.basicConfig(level=logging.DEBUG) + +def letta_run(message: str): + print(f"Debug: Processing message: {message}") + # ... rest of code ``` +Test Letta server connection: + +```python +from letta_client import Letta + +try: + client = Letta(base_url="http://localhost:8283") + print("βœ… Successfully connected to Letta server") +except Exception as e: + print(f"❌ Connection failed: {e}") +``` + +--- + ## Performance Optimization -### 1. **Memory Optimization** -- Use efficient memory storage -- Implement memory compression -- Monitor memory usage +### 1. **Connection Pooling** +Reuse Letta client connections when possible: + +```python +# Initialize once +_letta_client = None + +def get_letta_client(): + global _letta_client + if _letta_client is None: + _letta_client = Letta(base_url=os.getenv("LETTA_SERVER_URL")) + return _letta_client +``` + +### 2. **Tool Caching** +Register tools once and reuse: -### 2. **Session Management** -- Optimize session cleanup -- Use connection pooling -- Implement session caching +```python +_tool_registry = None -### 3. **Tool Performance** -- Cache tool results -- Optimize tool execution -- Use async tools when possible +def get_tools(client: Letta): + global _tool_registry + if _tool_registry is None: + _tool_registry = _register_tools(client) + return _tool_registry +``` + +### 3. **Memory Management** +Clean up agents promptly: + +```python +finally: + try: + client.agents.delete(agent.id) + except: + pass +``` + +--- ## Next Steps - - - Learn advanced Letta patterns and techniques - - - Deploy your Letta agents to production - - - Access your Letta agents from different languages - - - Optimize your Letta agents for production - - - - -**πŸŽ‰ Great work!** You've learned how to deploy Letta memory-enabled agents with RunAgent. Letta's conversational memory capabilities combined with RunAgent's multi-language access create powerful, context-aware AI systems! - +- **[Advanced Patterns](/how-to/advanced-tasks)** - Learn advanced Letta patterns +- **[Production Deployment](/how-to/deployment/cloud-deployment)** - Deploy to production +- **[Multi-Language Access](/tutorials/multi-language-wrapper)** - Access from different languages +- **[Performance Tuning](/explanation/production-considerations)** - Optimize for production + +--- + +## Additional Resources + +- [Letta Documentation](https://docs.letta.ai) +- [Letta GitHub](https://github.com/letta-ai/letta) +- [RunAgent Discord Community](https://discord.gg/Q9P9AdHVHz) +- [RunAgent Documentation](https://docs.run-agent.ai) + +--- + +**πŸŽ‰ Great work!** You've learned how to deploy Letta memory-enabled agents with RunAgent. Letta's conversational memory capabilities combined with RunAgent's multi-language access create powerful, context-aware AI systems! \ No newline at end of file diff --git a/docs/how-to/frameworks/llamaindex.mdx b/docs/how-to/frameworks/llamaindex.mdx new file mode 100644 index 0000000..0cf739a --- /dev/null +++ b/docs/how-to/frameworks/llamaindex.mdx @@ -0,0 +1,1093 @@ +# LlamaIndex Integration + +Deploy LlamaIndex agent workflows with RunAgent + +--- + +## Prerequisites + +- Basic understanding of LlamaIndex +- Completed [Deploy Your First Agent](/tutorials/deploy-your-first-agent) tutorial +- Python 3.8 or higher + +--- + +## Overview + +LlamaIndex is a data framework for building LLM applications with advanced indexing, retrieval, and agent workflows. RunAgent makes it easy to deploy LlamaIndex agents and access them from any programming language. + +--- + +## Installation & Setup + +### 1. Install LlamaIndex + +```bash +pip install llama-index>=0.12.48 +``` + +### 2. Set Environment Variables + +LlamaIndex requires API keys for LLM providers: + +```bash +export OPENAI_API_KEY=your_openai_api_key_here +``` + +### 3. Quick Start with RunAgent + +```bash +runagent init my-llamaindex-agent --framework llamaindex +cd my-llamaindex-agent +``` + +--- + +## Quick Start + +### 1. Project Structure + +After initialization: + +``` +my-llamaindex-agent/ +β”œβ”€β”€ math_genius.py # Main agent code +β”œβ”€β”€ .env # Environment variables +β”œβ”€β”€ requirements.txt # Python dependencies +└── runagent.config.json # RunAgent configuration +``` + +### 2. Configuration + +The generated `runagent.config.json`: + +```json +{ + "agent_name": "llamaindex-agent", + "description": "LlamaIndex agent with tool capabilities", + "framework": "llamaindex", + "version": "1.0.0", + "agent_architecture": { + "entrypoints": [ + { + "file": "math_genius.py", + "module": "do_multiply", + "tag": "math_run" + }, + { + "file": "math_genius.py", + "module": "stream_multiply", + "tag": "math_stream" + } + ] + }, + "env_vars": { + "OPENAI_API_KEY": "" + } +} +``` + +### 3. Create `.env` File + +```bash +OPENAI_API_KEY=your_openai_api_key_here +``` + +--- + +## Basic LlamaIndex Agent + +Here's a simple LlamaIndex agent with a calculator tool: + +```python +# math_genius.py +from llama_index.llms.openai import OpenAI +from llama_index.core.agent.workflow import AgentStream, FunctionAgent + + +# Define calculator tools +def multiply(a: float, b: float) -> float: + """Multiply two numbers together.""" + return a * b + + +def add(a: float, b: float) -> float: + """Add two numbers together.""" + return a + b + + +def subtract(a: float, b: float) -> float: + """Subtract b from a.""" + return a - b + + +def divide(a: float, b: float) -> float: + """Divide a by b. Returns error if b is zero.""" + if b == 0: + raise ValueError("Cannot divide by zero") + return a / b + + +# Create an agent workflow with calculator tools +agent = FunctionAgent( + tools=[multiply, add, subtract, divide], + llm=OpenAI(model="gpt-4o-mini"), + system_prompt="You are a helpful mathematical assistant. Use the provided tools to perform calculations.", +) + + +async def do_multiply(math_query: str): + """ + Non-streaming math agent. + + Args: + math_query: The mathematical query or expression + + Returns: + The calculation result + """ + try: + result = await agent.run(math_query) + return { + "status": "success", + "result": str(result), + "query": math_query + } + except Exception as e: + return { + "status": "error", + "error": str(e), + "query": math_query + } + + +async def stream_multiply(math_query: str): + """ + Streaming math agent. + + Args: + math_query: The mathematical query or expression + + Yields: + Streaming events from the agent + """ + try: + handler = agent.run(user_msg=math_query) + + async for event in handler.stream_events(): + if isinstance(event, AgentStream): + yield { + "type": "agent_stream", + "content": str(event), + "query": math_query + } + else: + yield { + "type": "event", + "data": str(event) + } + + except Exception as e: + yield { + "type": "error", + "error": str(e), + "query": math_query + } +``` + +--- + +## Advanced LlamaIndex Patterns + +### 1. RAG Agent with Document Indexing + +```python +# rag_agent.py +from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings +from llama_index.llms.openai import OpenAI +from llama_index.embeddings.openai import OpenAIEmbedding +from llama_index.core.agent import ReActAgent +from llama_index.core.tools import QueryEngineTool, ToolMetadata + +# Configure global settings +Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0.1) +Settings.embed_model = OpenAIEmbedding(model="text-embedding-ada-002") + + +# Load and index documents +def create_index(data_dir: str = "./data"): + """Create vector index from documents.""" + try: + documents = SimpleDirectoryReader(data_dir).load_data() + index = VectorStoreIndex.from_documents(documents) + return index + except Exception as e: + print(f"Error creating index: {e}") + return None + + +# Create query engine tool +def create_rag_tools(index): + """Create RAG tools from index.""" + query_engine = index.as_query_engine(similarity_top_k=3) + + query_tool = QueryEngineTool( + query_engine=query_engine, + metadata=ToolMetadata( + name="document_search", + description="Search through indexed documents to find relevant information. Use this for questions about the document content." + ) + ) + + return [query_tool] + + +# Initialize index and agent +index = create_index() +if index: + tools = create_rag_tools(index) + rag_agent = ReActAgent.from_tools( + tools=tools, + llm=Settings.llm, + verbose=True + ) +else: + rag_agent = None + + +async def rag_query(query: str): + """ + Query documents using RAG. + + Args: + query: User question about documents + + Returns: + Answer based on document content + """ + if rag_agent is None: + return { + "status": "error", + "error": "RAG agent not initialized. Check if documents exist in ./data directory." + } + + try: + response = await rag_agent.achat(query) + return { + "status": "success", + "response": str(response), + "query": query + } + except Exception as e: + return { + "status": "error", + "error": str(e), + "query": query + } + + +async def rag_query_stream(query: str): + """ + Streaming RAG query. + + Args: + query: User question about documents + + Yields: + Streaming response chunks + """ + if rag_agent is None: + yield { + "status": "error", + "error": "RAG agent not initialized" + } + return + + try: + response = await rag_agent.astream_chat(query) + + async for chunk in response.async_response_gen(): + yield { + "type": "text", + "content": chunk + } + + except Exception as e: + yield { + "type": "error", + "error": str(e) + } +``` + +### 2. Multi-Tool Agent + +```python +# multi_tool_agent.py +from llama_index.llms.openai import OpenAI +from llama_index.core.agent import ReActAgent +from llama_index.core.tools import FunctionTool +from typing import Dict, Any +import json + + +# Define multiple tools +def get_weather(city: str) -> str: + """Get current weather for a city.""" + # Mock weather data + weather_data = { + "new york": {"temp": 22, "condition": "Sunny"}, + "london": {"temp": 15, "condition": "Rainy"}, + "tokyo": {"temp": 18, "condition": "Cloudy"}, + "paris": {"temp": 20, "condition": "Partly Cloudy"} + } + + city_lower = city.lower() + if city_lower in weather_data: + data = weather_data[city_lower] + return f"Weather in {city}: {data['temp']}Β°C, {data['condition']}" + return f"Weather data not available for {city}" + + +def calculate_tip(bill_amount: float, tip_percentage: float = 15.0) -> str: + """Calculate tip amount and total bill.""" + tip = bill_amount * (tip_percentage / 100) + total = bill_amount + tip + return f"Tip: ${tip:.2f}, Total: ${total:.2f}" + + +def convert_currency(amount: float, from_curr: str, to_curr: str) -> str: + """Convert between currencies.""" + # Mock conversion rates + rates = {"USD": 1.0, "EUR": 0.85, "GBP": 0.73, "JPY": 110.0} + + from_rate = rates.get(from_curr.upper(), 1.0) + to_rate = rates.get(to_curr.upper(), 1.0) + + result = amount * (to_rate / from_rate) + return f"{amount} {from_curr.upper()} = {result:.2f} {to_curr.upper()}" + + +def search_definition(term: str) -> str: + """Search for term definition.""" + # Mock definitions + definitions = { + "ai": "Artificial Intelligence: The simulation of human intelligence by machines", + "ml": "Machine Learning: A subset of AI that enables systems to learn from data", + "llm": "Large Language Model: AI models trained on vast amounts of text data" + } + + term_lower = term.lower() + return definitions.get(term_lower, f"Definition for '{term}' not found. This is a mock search.") + + +# Create function tools +weather_tool = FunctionTool.from_defaults(fn=get_weather) +tip_tool = FunctionTool.from_defaults(fn=calculate_tip) +currency_tool = FunctionTool.from_defaults(fn=convert_currency) +definition_tool = FunctionTool.from_defaults(fn=search_definition) + +# Create multi-tool agent +multi_agent = ReActAgent.from_tools( + tools=[weather_tool, tip_tool, currency_tool, definition_tool], + llm=OpenAI(model="gpt-4o-mini"), + verbose=True +) + + +async def multi_tool_query(query: str): + """ + Query using multiple tools. + + Args: + query: User query that may require one or more tools + + Returns: + Response using appropriate tools + """ + try: + response = await multi_agent.achat(query) + return { + "status": "success", + "response": str(response), + "query": query, + "tools_available": ["weather", "tip_calculator", "currency_converter", "definition_search"] + } + except Exception as e: + return { + "status": "error", + "error": str(e), + "query": query + } + + +async def multi_tool_stream(query: str): + """ + Streaming query with multiple tools. + + Args: + query: User query + + Yields: + Streaming response chunks + """ + try: + response = await multi_agent.astream_chat(query) + + async for chunk in response.async_response_gen(): + yield { + "type": "text", + "content": chunk + } + + except Exception as e: + yield { + "type": "error", + "error": str(e) + } +``` + +### 3. Workflow-Based Agent + +```python +# workflow_agent.py +from llama_index.core.workflow import ( + Workflow, + StartEvent, + StopEvent, + step, + Event, + Context +) +from llama_index.llms.openai import OpenAI +from llama_index.core.agent import FunctionAgent +from typing import Any + + +# Define workflow events +class QueryEvent(Event): + query: str + + +class AnalysisEvent(Event): + analysis: str + + +class ResponseEvent(Event): + response: str + + +# Create workflow +class AgentWorkflow(Workflow): + """Custom workflow for agent processing.""" + + def __init__(self): + super().__init__() + self.llm = OpenAI(model="gpt-4o-mini") + + @step + async def process_query(self, ctx: Context, ev: StartEvent) -> QueryEvent: + """Initial query processing step.""" + query = ev.get("query", "") + print(f"Step 1: Processing query: {query}") + + # Store in context + await ctx.set("original_query", query) + + return QueryEvent(query=query) + + @step + async def analyze_query(self, ctx: Context, ev: QueryEvent) -> AnalysisEvent: + """Analyze query intent and requirements.""" + print(f"Step 2: Analyzing query intent") + + analysis_prompt = f"Analyze this query and determine what tools or information are needed: {ev.query}" + response = await self.llm.acomplete(analysis_prompt) + + analysis = str(response) + await ctx.set("analysis", analysis) + + return AnalysisEvent(analysis=analysis) + + @step + async def generate_response(self, ctx: Context, ev: AnalysisEvent) -> StopEvent: + """Generate final response based on analysis.""" + print(f"Step 3: Generating response") + + original_query = await ctx.get("original_query") + + response_prompt = f"Based on this analysis: {ev.analysis}\n\nAnswer this query: {original_query}" + response = await self.llm.acomplete(response_prompt) + + return StopEvent(result={ + "query": original_query, + "analysis": ev.analysis, + "response": str(response), + "workflow": "completed" + }) + + +# Initialize workflow +workflow = AgentWorkflow() + + +async def workflow_query(query: str): + """ + Process query through custom workflow. + + Args: + query: User query + + Returns: + Workflow result + """ + try: + result = await workflow.run(query=query) + return { + "status": "success", + "result": result + } + except Exception as e: + return { + "status": "error", + "error": str(e), + "query": query + } +``` + +### 4. Agent with Memory + +```python +# memory_agent.py +from llama_index.llms.openai import OpenAI +from llama_index.core.agent import ReActAgent +from llama_index.core.memory import ChatMemoryBuffer +from llama_index.core.tools import FunctionTool +from typing import Dict + + +# Memory storage (in production, use proper database) +user_memories = {} + + +def remember_fact(user_id: str, key: str, value: str) -> str: + """Remember a fact about the user.""" + if user_id not in user_memories: + user_memories[user_id] = {} + + user_memories[user_id][key] = value + return f"I'll remember that {key}: {value}" + + +def recall_fact(user_id: str, key: str) -> str: + """Recall a fact about the user.""" + if user_id not in user_memories: + return f"I don't have any information about {key}" + + value = user_memories[user_id].get(key) + if value: + return f"I remember that {key}: {value}" + return f"I don't have information about {key}" + + +def list_memories(user_id: str) -> str: + """List all remembered facts for a user.""" + if user_id not in user_memories or not user_memories[user_id]: + return "I don't have any memories stored yet." + + memories = user_memories[user_id] + return "Here's what I remember:\n" + "\n".join( + f"- {k}: {v}" for k, v in memories.items() + ) + + +# Create memory tools +remember_tool = FunctionTool.from_defaults(fn=remember_fact) +recall_tool = FunctionTool.from_defaults(fn=recall_fact) +list_tool = FunctionTool.from_defaults(fn=list_memories) + + +def create_memory_agent(user_id: str): + """Create an agent with memory for a specific user.""" + # Create chat memory buffer + memory = ChatMemoryBuffer.from_defaults(token_limit=3000) + + # Create agent with memory + agent = ReActAgent.from_tools( + tools=[remember_tool, recall_tool, list_tool], + llm=OpenAI(model="gpt-4o-mini"), + memory=memory, + verbose=True, + system_prompt=f"You are a helpful assistant with memory capabilities for user {user_id}. " + f"You can remember and recall information about the user." + ) + + return agent + + +# Agent cache +agent_cache = {} + + +async def memory_chat(user_id: str, message: str): + """ + Chat with memory-enabled agent. + + Args: + user_id: Unique user identifier + message: User message + + Returns: + Agent response with memory context + """ + try: + # Get or create agent for user + if user_id not in agent_cache: + agent_cache[user_id] = create_memory_agent(user_id) + + agent = agent_cache[user_id] + + # Chat with agent + response = await agent.achat(message) + + return { + "status": "success", + "response": str(response), + "user_id": user_id, + "has_memory": True + } + except Exception as e: + return { + "status": "error", + "error": str(e), + "user_id": user_id + } + + +async def memory_chat_stream(user_id: str, message: str): + """ + Streaming chat with memory. + + Args: + user_id: Unique user identifier + message: User message + + Yields: + Streaming response chunks + """ + try: + if user_id not in agent_cache: + agent_cache[user_id] = create_memory_agent(user_id) + + agent = agent_cache[user_id] + response = await agent.astream_chat(message) + + async for chunk in response.async_response_gen(): + yield { + "type": "text", + "content": chunk, + "user_id": user_id + } + + except Exception as e: + yield { + "type": "error", + "error": str(e), + "user_id": user_id + } +``` + +--- + +## Testing Your LlamaIndex Agent + +### Python Client + +```python +# test_llamaindex.py +from runagent import RunAgentClient +import asyncio + +# Test basic math agent +client = RunAgentClient( + agent_id="your_agent_id_here", + entrypoint_tag="math_run", + local=True +) + +result = client.run(math_query="What is 25 * 4?") +print(f"Math result: {result}") + +# Test streaming +stream_client = RunAgentClient( + agent_id="your_agent_id_here", + entrypoint_tag="math_stream", + local=True +) + +print("\nStreaming calculation:") +for chunk in stream_client.run(math_query="Calculate 100 + 250 - 50"): + if chunk.get("content"): + print(chunk["content"]) + +# Test RAG agent (if configured) +rag_client = RunAgentClient( + agent_id="your_agent_id_here", + entrypoint_tag="rag_query", + local=True +) + +rag_result = rag_client.run(query="What does the document say about AI?") +print(f"\nRAG result: {rag_result}") +``` + +### JavaScript Client + +```javascript +// test_llamaindex.js +import { RunAgentClient } from 'runagent'; + +const client = new RunAgentClient({ + agentId: 'your_agent_id_here', + entrypointTag: 'math_run', + local: true +}); + +await client.initialize(); + +// Test calculation +const result = await client.run({ + math_query: 'What is 15 * 8?' +}); + +console.log('Result:', result); + +// Test streaming +const streamClient = new RunAgentClient({ + agentId: 'your_agent_id_here', + entrypointTag: 'math_stream', + local: true +}); + +await streamClient.initialize(); + +console.log('\nStreaming:'); +for await (const chunk of streamClient.run({ + math_query: 'Calculate the sum of 10, 20, and 30' +})) { + if (chunk.content) { + process.stdout.write(chunk.content); + } +} +``` + +### Go Client + +```go +package main + +import ( + "context" + "fmt" + "github.com/runagent-dev/runagent-go/pkg/client" +) + +func main() { + client, _ := client.New( + "your_agent_id_here", + "math_run", + true, + ) + defer client.Close() + + ctx := context.Background() + + result, _ := client.Run(ctx, map[string]interface{}{ + "math_query": "What is 2 * 2?", + }) + + fmt.Printf("Result: %v\n", result) +} +``` + +### Rust Client + +```rust +use runagent::client::RunAgentClient; +use serde_json::json; + +#[tokio::main] +async fn main() -> Result<(), Box> { + let client = RunAgentClient::new( + "your_agent_id_here", + "math_run", + true + ).await?; + + let result = client.run(&[ + ("math_query", json!("What is 5 * 9?")) + ]).await?; + + println!("Result: {}", result); + + Ok(()) +} +``` + +--- + +## Configuration Examples + +### Basic Math Agent + +```json +{ + "agent_name": "llamaindex-math", + "framework": "llamaindex", + "agent_architecture": { + "entrypoints": [ + { + "file": "math_genius.py", + "module": "do_multiply", + "tag": "math_run" + }, + { + "file": "math_genius.py", + "module": "stream_multiply", + "tag": "math_stream" + } + ] + } +} +``` + +### Multi-Feature Agent + +```json +{ + "agent_name": "llamaindex-advanced", + "framework": "llamaindex", + "agent_architecture": { + "entrypoints": [ + { + "file": "math_genius.py", + "module": "do_multiply", + "tag": "math" + }, + { + "file": "rag_agent.py", + "module": "rag_query", + "tag": "rag" + }, + { + "file": "multi_tool_agent.py", + "module": "multi_tool_query", + "tag": "multi_tool" + }, + { + "file": "memory_agent.py", + "module": "memory_chat", + "tag": "memory" + } + ] + } +} +``` + +--- + +## Best Practices + +### 1. **Tool Design** +- Keep tools simple and focused +- Provide clear docstrings for LLM understanding +- Handle errors gracefully within tools +- Use type hints for parameters + +### 2. **Agent Configuration** +- Choose appropriate LLM models for your use case +- Set reasonable temperature values +- Configure memory limits appropriately +- Use verbose mode during development + +### 3. **RAG Implementation** +- Index documents efficiently +- Choose appropriate chunk sizes +- Use optimal similarity thresholds +- Implement caching for repeated queries + +### 4. **Memory Management** +- Set appropriate token limits for memory +- Clean up old agent instances +- Implement user-based memory isolation +- Persist important memories to database + +### 5. **Error Handling** +- Always wrap async operations in try-catch +- Return structured error responses +- Log errors for debugging +- Provide helpful error messages + +--- + +## Common Patterns + +### Tool-Based Pattern +Simple agents with specific capabilities: +```python +agent + [calculator, weather, search] β†’ responses +``` + +### RAG Pattern +Knowledge-augmented responses: +```python +query β†’ document_search β†’ llm_synthesis β†’ answer +``` + +### Workflow Pattern +Multi-step processing: +```python +query β†’ analyze β†’ process β†’ generate β†’ response +``` + +### Memory Pattern +Context-aware conversations: +```python +user_memory + current_query β†’ contextual_response +``` + +--- + +## Troubleshooting + +### Common Issues + +**1. API Key Not Found** +- **Solution**: Set `OPENAI_API_KEY` in environment +- Verify key is valid and has credits +- Check `.env` file is loaded properly + +**2. Import Errors** +- **Solution**: Install correct LlamaIndex version +- Check all required packages are installed +- Verify virtual environment is activated + +**3. Agent Not Responding** +- **Solution**: Check LLM configuration +- Verify tools are properly registered +- Review system prompts for clarity + +**4. RAG Returning Poor Results** +- **Solution**: Adjust similarity thresholds +- Review document chunking strategy +- Check embedding model quality +- Verify document indexing completed + +**5. Streaming Not Working** +- **Solution**: Use `astream_chat` instead of `achat` +- Check async implementation +- Verify streaming is supported by the model + +### Debug Tips + +Enable verbose logging: + +```python +import logging +logging.basicConfig(level=logging.DEBUG) + +# Enable LlamaIndex debug logging +from llama_index.core import set_global_handler +set_global_handler("simple") +``` + +Test agent locally: + +```python +# test_local.py +import asyncio +from math_genius import do_multiply + +async def test(): + result = await do_multiply("What is 5 * 3?") + print(f"Result: {result}") + +asyncio.run(test()) +``` + +--- + +## Performance Optimization + +### 1. **Agent Caching** +Cache agent instances: + +```python +_agent_cache = {} + +def get_agent(agent_type: str): + if agent_type not in _agent_cache: + _agent_cache[agent_type] = create_agent(agent_type) + return _agent_cache[agent_type] +``` + +### 2. **Index Optimization** +Optimize RAG indexing: + +```python +# Use persistent storage +from llama_index.core import StorageContext, load_index_from_storage + +def get_or_create_index(): + try: + storage_context = StorageContext.from_defaults(persist_dir="./storage") + index = load_index_from_storage(storage_context) + except: + documents = SimpleDirectoryReader("./data").load_data() + index = VectorStoreIndex.from_documents(documents) + index.storage_context.persist() + return index +``` + +### 3. **Memory Management** +Implement memory limits: + +```python +from llama_index.core.memory import ChatMemoryBuffer + +memory = ChatMemoryBuffer.from_defaults( + token_limit=2000, # Reasonable limit + ttl=3600 # 1 hour TTL +) +``` + +### 4. **Async Operations** +Use async throughout: + +```python +# Always use async methods +response = await agent.achat(query) # Good +# response = agent.chat(query) # Avoid blocking calls +``` + +--- + +## Next Steps + +- **[Advanced Patterns](/how-to/advanced-tasks)** - Learn advanced LlamaIndex patterns +- **[Production Deployment](/how-to/deployment/cloud-deployment)** - Deploy to production +- **[Multi-Language Access](/tutorials/multi-language-wrapper)** - Access from different languages +- **[Performance Tuning](/explanation/production-considerations)** - Optimize for production + +--- + +## Additional Resources + +- [LlamaIndex Documentation](https://docs.llamaindex.ai) +- [LlamaIndex GitHub](https://github.com/run-llama/llama_index) +- [LlamaIndex Discord](https://discord.gg/llamaindex) +- [RunAgent Discord Community](https://discord.gg/Q9P9AdHVHz) +- [RunAgent Documentation](https://docs.run-agent.ai) + +--- + +**πŸŽ‰ Great work!** You've learned how to deploy LlamaIndex agents with RunAgent. LlamaIndex's powerful data framework combined with RunAgent's multi-language access creates sophisticated, knowledge-augmented AI systems! \ No newline at end of file diff --git a/docs/how-to/frameworks/parlant.mdx b/docs/how-to/frameworks/parlant.mdx new file mode 100644 index 0000000..1e41622 --- /dev/null +++ b/docs/how-to/frameworks/parlant.mdx @@ -0,0 +1,1066 @@ +# Parlant Integration + +Deploy Parlant guideline-driven conversational agents with RunAgent + +--- + +## Prerequisites + +- Basic understanding of Parlant +- Completed [Deploy Your First Agent](/tutorials/deploy-your-first-agent) tutorial +- Python 3.10 or higher + +--- + +## Overview + +Parlant is a framework for building guideline-driven conversational AI agents with structured behavior management. RunAgent makes it easy to deploy Parlant agents and access them from any programming language. + +--- + +## Installation & Setup + +### 1. Install Parlant + +```bash +pip install parlant +``` + +### 2. Set Environment Variables + +Parlant requires API keys for LLM providers: + +```bash +export OPENAI_API_KEY=your_openai_api_key_here +export PARLANT_LOG_LEVEL=INFO # Optional +``` + +### 3. Start Parlant Server + +The Parlant server must be running before deploying RunAgent agents: + +```bash +parlant-server run +``` + +The server will start on `http://localhost:8800`. Keep this terminal window open. + +**Note**: If you need to use a different port: +```bash +parlant-server run --port 8801 +``` + +### 4. Quick Start with RunAgent + +```bash +runagent init my-parlant-agent --framework parlant +cd my-parlant-agent +``` + +--- + +## Quick Start + +### 1. Project Structure + +After initialization: + +``` +my-parlant-agent/ +β”œβ”€β”€ agent.py # Main agent code +β”œβ”€β”€ .env # Environment variables +β”œβ”€β”€ requirements.txt # Python dependencies +└── runagent.config.json # RunAgent configuration +``` + +### 2. Configuration + +The generated `runagent.config.json`: + +```json +{ + "agent_name": "parlant-assistant", + "description": "Parlant-based conversational AI assistant", + "framework": "parlant", + "version": "1.0.0", + "agent_architecture": { + "entrypoints": [ + { + "file": "agent.py", + "module": "simple_chat", + "tag": "parlant_simple" + }, + { + "file": "agent.py", + "module": "chat_stream", + "tag": "parlant_stream" + } + ] + }, + "env_vars": { + "OPENAI_API_KEY": "", + "PARLANT_LOG_LEVEL": "INFO" + } +} +``` + +### 3. Create `.env` File + +```bash +OPENAI_API_KEY=your_openai_api_key_here +PARLANT_LOG_LEVEL=INFO +``` + +--- + +## Basic Parlant Agent + +Here's a complete Parlant agent with tools and guidelines: + +```python +# agent.py +import asyncio +from parlant.client import AsyncParlantClient +from typing import Dict, Any, AsyncGenerator +import time + +# Global client and agent_id for reuse +_parlant_client = None +_agent_id = None + + +async def get_parlant_client(): + """Get or create the Parlant client instance.""" + global _parlant_client + + if _parlant_client is None: + _parlant_client = AsyncParlantClient(base_url="http://localhost:8800") + + # Test connection + try: + await _parlant_client.agents.list() + except Exception as e: + raise Exception( + f"Failed to connect to Parlant server at http://localhost:8800. " + f"Make sure it's running with 'parlant-server run'. Error: {e}" + ) + + return _parlant_client + + +async def create_agent_with_tools(client: AsyncParlantClient): + """Create an agent with calculator, time, and weather tools.""" + + # Check if agent already exists + agents = await client.agents.list() + for agent in agents: + if agent.name == "RunAgent Assistant": + print(f"βœ… Found existing agent: {agent.name}") + return agent.id + + # Create new agent + agent = await client.agents.create( + name="RunAgent Assistant", + description="A helpful assistant that can provide information, perform calculations, tell time, and check weather." + ) + + print(f"βœ… Created new agent: {agent.name}") + + # Add calculator tool + await client.agents.tools.create( + agent_id=agent.id, + name="calculator", + description="Calculate mathematical expressions. Supports +, -, *, /, **, sqrt(), sin(), cos(), etc.", + parameters={ + "type": "object", + "properties": { + "expression": { + "type": "string", + "description": "Mathematical expression to calculate" + } + }, + "required": ["expression"] + }, + implementation=""" +try: + import math + allowed_names = {k: v for k, v in math.__dict__.items() if not k.startswith("__")} + allowed_names.update({"abs": abs, "round": round, "min": min, "max": max, "sum": sum, "pow": pow}) + + expression = expression.replace("^", "**") + result = eval(expression, {"__builtins__": {}}, allowed_names) + return f"Result: {result}" +except Exception as e: + return f"Error calculating '{expression}': {str(e)}" +""" + ) + + # Add time tool + await client.agents.tools.create( + agent_id=agent.id, + name="get_current_time", + description="Get the current date and time", + parameters={"type": "object", "properties": {}, "required": []}, + implementation=""" +from datetime import datetime +current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S") +return f"Current time: {current_time}" +""" + ) + + # Add weather tool (mock implementation) + await client.agents.tools.create( + agent_id=agent.id, + name="weather_info", + description="Get weather information for a city", + parameters={ + "type": "object", + "properties": { + "city": { + "type": "string", + "description": "Name of the city to get weather for" + } + }, + "required": ["city"] + }, + implementation=""" +# Mock weather data +weather_data = { + "paris": "Partly cloudy, 18Β°C", + "london": "Rainy, 12Β°C", + "new york": "Sunny, 22Β°C", + "tokyo": "Cloudy, 16Β°C", + "berlin": "Overcast, 15Β°C", + "sydney": "Clear, 25Β°C" +} + +city_lower = city.lower() +weather = weather_data.get(city_lower, f"Weather data not available for {city}") +return f"Weather in {city}: {weather}" +""" + ) + + # Add guidelines + guidelines = [ + { + "condition": "User asks for the current time or date", + "action": "Get the current time and provide it in a friendly manner", + "tools": ["get_current_time"] + }, + { + "condition": "User asks to calculate something or provides a math expression", + "action": "Use the calculator tool to solve the mathematical expression and explain the result", + "tools": ["calculator"] + }, + { + "condition": "User asks about weather", + "action": "Get weather information using the weather_info tool and provide a helpful response", + "tools": ["weather_info"] + }, + { + "condition": "User greets or says hello", + "action": "Respond with a warm greeting and explain what you can help with" + }, + { + "condition": "User asks what you can do or for help", + "action": "Explain that you can perform calculations, provide time, check weather, and answer questions" + }, + { + "condition": "User says goodbye or thanks", + "action": "Respond politely and invite them to ask if they need anything else" + } + ] + + for guideline in guidelines: + await client.agents.guidelines.create( + agent_id=agent.id, + **guideline + ) + + print(f"βœ… Added {len(guidelines)} guidelines and 3 tools") + return agent.id + + +async def get_or_create_agent(): + """Get or create the agent with full tools and guidelines setup.""" + global _agent_id + + if _agent_id is not None: + return _agent_id + + client = await get_parlant_client() + _agent_id = await create_agent_with_tools(client) + return _agent_id + + +async def chat_with_agent(message: str, agent_id: str) -> str: + """Send a message to the agent and get response.""" + client = await get_parlant_client() + + try: + # Create a chat session + session = await client.sessions.create( + agent_id=agent_id, + allow_greeting=False + ) + + # Send message + event = await client.sessions.create_event( + session_id=session.id, + kind="message", + source="customer", + message=message, + ) + + # Get response + agent_messages = await client.sessions.list_events( + session_id=session.id, + min_offset=event.offset, + source="ai_agent", + kinds="message", + wait_for_data=30, + ) + + if agent_messages: + response_data = agent_messages[0].model_dump() + return response_data.get("data", {}).get("message", "No response received") + else: + return "No response from agent" + + except Exception as e: + return f"Error processing message: {str(e)}" + + +async def simple_chat(message: str) -> Dict[str, Any]: + """ + Simple chat function that processes messages through Parlant. + + Args: + message: The user's message + + Returns: + Dict containing the response and metadata + """ + try: + # Ensure agent is available with full setup + agent_id = await get_or_create_agent() + + # Get response from agent + response = await chat_with_agent(message, agent_id) + + return { + "content": response, + "type": "parlant_response", + "agent_id": agent_id, + "message_received": message, + "timestamp": time.time(), + "framework": "parlant" + } + + except Exception as e: + return { + "content": f"Error: {str(e)}", + "type": "error", + "error": str(e), + "message_received": message, + "framework": "parlant" + } + + +async def chat_stream(message: str) -> AsyncGenerator[Dict[str, Any], None]: + """ + Streaming chat function. + + Args: + message: The user's message + + Yields: + Dict containing streaming response chunks + """ + try: + # Get the response first + response = await simple_chat(message) + + if response["type"] == "error": + yield response + return + + # Simulate streaming by breaking up the response + content = response["content"] + words = content.split() + + for word in words: + yield { + "content": word + " " + } + await asyncio.sleep(0.05) + + # Final chunk + yield { + "content": "", + "type": "completion", + "agent_id": response.get("agent_id"), + "is_final": True, + "framework": "parlant" + } + + except Exception as e: + yield { + "content": f"Error: {str(e)}", + "type": "error", + "error": str(e), + "framework": "parlant" + } +``` + +--- + +## Advanced Patterns + +### 1. Custom Guidelines System + +```python +# advanced_guidelines.py +from parlant.client import AsyncParlantClient +from typing import List, Dict + +async def create_customer_support_agent(client: AsyncParlantClient): + """Create a customer support agent with specialized guidelines.""" + + agent = await client.agents.create( + name="Customer Support Assistant", + description="Specialized customer support agent with escalation protocols" + ) + + # Support tools + await client.agents.tools.create( + agent_id=agent.id, + name="check_order_status", + description="Check the status of an order", + parameters={ + "type": "object", + "properties": { + "order_id": {"type": "string", "description": "Order ID to check"} + }, + "required": ["order_id"] + }, + implementation=""" +# Mock order status check +order_statuses = { + "ORD123": "Shipped - Arriving tomorrow", + "ORD456": "Processing - Expected to ship today", + "ORD789": "Delivered on 2025-01-15" +} +return order_statuses.get(order_id, f"Order {order_id} not found in system") +""" + ) + + await client.agents.tools.create( + agent_id=agent.id, + name="create_support_ticket", + description="Create a support ticket for complex issues", + parameters={ + "type": "object", + "properties": { + "issue": {"type": "string", "description": "Description of the issue"}, + "priority": {"type": "string", "enum": ["low", "medium", "high"]} + }, + "required": ["issue", "priority"] + }, + implementation=""" +ticket_id = f"TKT{hash(issue) % 10000}" +return f"Support ticket {ticket_id} created with {priority} priority. A team member will contact you within 24 hours." +""" + ) + + # Customer support guidelines + support_guidelines = [ + { + "condition": "User asks about order status", + "action": "Use check_order_status tool to get current status and provide helpful tracking information", + "tools": ["check_order_status"] + }, + { + "condition": "User reports a problem or complaint", + "action": "Acknowledge the issue empathetically, gather details, and create a support ticket", + "tools": ["create_support_ticket"] + }, + { + "condition": "User asks about refund or return policy", + "action": "Explain that refunds are available within 30 days and guide them through the process" + }, + { + "condition": "User is frustrated or angry", + "action": "Respond with empathy, apologize for inconvenience, and escalate by creating high-priority ticket" + } + ] + + for guideline in support_guidelines: + await client.agents.guidelines.create(agent_id=agent.id, **guideline) + + return agent.id + + +async def support_chat(message: str) -> Dict: + """Customer support conversation handler.""" + client = AsyncParlantClient(base_url="http://localhost:8800") + agent_id = await create_customer_support_agent(client) + + # Create session and get response + session = await client.sessions.create(agent_id=agent_id, allow_greeting=False) + + await client.sessions.create_event( + session_id=session.id, + kind="message", + source="customer", + message=message + ) + + response = await client.sessions.list_events( + session_id=session.id, + source="ai_agent", + kinds="message", + wait_for_data=30 + ) + + return { + "response": response[0].data.get("message") if response else "No response", + "agent_type": "customer_support" + } +``` + +### 2. Multi-Domain Agent + +```python +# multi_domain_agent.py +from parlant.client import AsyncParlantClient + +async def create_multi_domain_agent(client: AsyncParlantClient): + """Create agent that handles multiple domains.""" + + agent = await client.agents.create( + name="Multi-Domain Assistant", + description="Assistant capable of handling finance, health, and travel queries" + ) + + # Finance tools + await client.agents.tools.create( + agent_id=agent.id, + name="currency_converter", + description="Convert between currencies", + parameters={ + "type": "object", + "properties": { + "amount": {"type": "number"}, + "from_currency": {"type": "string"}, + "to_currency": {"type": "string"} + }, + "required": ["amount", "from_currency", "to_currency"] + }, + implementation=""" +# Mock conversion rates +rates = {"USD": 1.0, "EUR": 0.85, "GBP": 0.73, "JPY": 110.0} +result = amount * (rates.get(to_currency, 1) / rates.get(from_currency, 1)) +return f"{amount} {from_currency} = {result:.2f} {to_currency}" +""" + ) + + # Health tools + await client.agents.tools.create( + agent_id=agent.id, + name="bmi_calculator", + description="Calculate Body Mass Index", + parameters={ + "type": "object", + "properties": { + "weight_kg": {"type": "number"}, + "height_m": {"type": "number"} + }, + "required": ["weight_kg", "height_m"] + }, + implementation=""" +bmi = weight_kg / (height_m ** 2) +category = "Underweight" if bmi < 18.5 else "Normal" if bmi < 25 else "Overweight" if bmi < 30 else "Obese" +return f"BMI: {bmi:.1f} - Category: {category}" +""" + ) + + # Travel tools + await client.agents.tools.create( + agent_id=agent.id, + name="flight_time", + description="Get approximate flight duration between cities", + parameters={ + "type": "object", + "properties": { + "from_city": {"type": "string"}, + "to_city": {"type": "string"} + }, + "required": ["from_city", "to_city"] + }, + implementation=""" +# Mock flight times +flights = { + ("new york", "london"): "7 hours", + ("paris", "tokyo"): "12 hours", + ("sydney", "singapore"): "8 hours" +} +key = (from_city.lower(), to_city.lower()) +return flights.get(key, "Flight time not available") +""" + ) + + # Domain-specific guidelines + guidelines = [ + { + "condition": "User asks about currency conversion or exchange rates", + "action": "Use currency_converter tool to provide accurate conversion", + "tools": ["currency_converter"] + }, + { + "condition": "User asks about BMI or body mass index", + "action": "Use bmi_calculator to calculate and explain the result", + "tools": ["bmi_calculator"] + }, + { + "condition": "User asks about flight duration or travel time", + "action": "Use flight_time tool to provide estimated duration", + "tools": ["flight_time"] + } + ] + + for guideline in guidelines: + await client.agents.guidelines.create(agent_id=agent.id, **guideline) + + return agent.id +``` + +### 3. Contextual Conversation Agent + +```python +# contextual_agent.py +from parlant.client import AsyncParlantClient +from typing import Dict, Optional + +class ContextualAgent: + def __init__(self): + self.client = None + self.agent_id = None + self.active_sessions = {} + + async def initialize(self): + """Initialize the agent with context-aware capabilities.""" + self.client = AsyncParlantClient(base_url="http://localhost:8800") + + agent = await self.client.agents.create( + name="Contextual Assistant", + description="Agent that maintains conversation context across interactions" + ) + self.agent_id = agent.id + + # Add memory tool + await self.client.agents.tools.create( + agent_id=self.agent_id, + name="remember_preference", + description="Remember user preferences", + parameters={ + "type": "object", + "properties": { + "key": {"type": "string"}, + "value": {"type": "string"} + }, + "required": ["key", "value"] + }, + implementation=""" +# In production, use proper database +return f"Remembered: {key} = {value}" +""" + ) + + # Context-aware guidelines + await self.client.agents.guidelines.create( + agent_id=self.agent_id, + condition="User mentions a preference or personal information", + action="Use remember_preference to store it for future reference", + tools=["remember_preference"] + ) + + await self.client.agents.guidelines.create( + agent_id=self.agent_id, + condition="User refers to previous conversation ('as I said', 'earlier')", + action="Acknowledge the reference and build upon the previous context" + ) + + async def chat(self, user_id: str, message: str) -> str: + """Chat with context awareness per user.""" + # Get or create session for user + if user_id not in self.active_sessions: + session = await self.client.sessions.create( + agent_id=self.agent_id, + allow_greeting=True + ) + self.active_sessions[user_id] = session.id + + session_id = self.active_sessions[user_id] + + # Send message + await self.client.sessions.create_event( + session_id=session_id, + kind="message", + source="customer", + message=message + ) + + # Get response + events = await self.client.sessions.list_events( + session_id=session_id, + source="ai_agent", + kinds="message", + wait_for_data=30 + ) + + if events: + return events[0].data.get("message", "No response") + return "No response from agent" + + +# Usage +async def contextual_chat(user_id: str, message: str) -> Dict: + """Handle contextual conversation.""" + agent = ContextualAgent() + await agent.initialize() + + response = await agent.chat(user_id, message) + + return { + "response": response, + "user_id": user_id, + "has_context": True + } +``` + +--- + +## Testing Your Parlant Agent + +### Python Client + +```python +# test_parlant.py +from runagent import RunAgentClient +import asyncio + +# Ensure Parlant server is running on port 8800 + +# Test simple chat +client = RunAgentClient( + agent_id="your_agent_id_here", + entrypoint_tag="parlant_simple", + local=True +) + +result = client.run(message="What time is it?") +print(f"Response: {result['content']}") + +# Test with calculation +result2 = client.run(message="Calculate 25 * 4 + 10") +print(f"Calculation result: {result2['content']}") + +# Test streaming +stream_client = RunAgentClient( + agent_id="your_agent_id_here", + entrypoint_tag="parlant_stream", + local=True +) + +print("\nStreaming conversation:") +for chunk in stream_client.run(message="What's the weather in Tokyo?"): + if chunk.get("content"): + print(chunk["content"], end="", flush=True) +``` + +### JavaScript Client + +```javascript +// test_parlant.js +import { RunAgentClient } from 'runagent'; + +const client = new RunAgentClient({ + agentId: 'your_agent_id_here', + entrypointTag: 'parlant_simple', + local: true +}); + +await client.initialize(); + +// Test conversation +const result = await client.run({ + message: 'Can you help me with a calculation?' +}); + +console.log('Response:', result.content); + +// Test streaming +const streamClient = new RunAgentClient({ + agentId: 'your_agent_id_here', + entrypointTag: 'parlant_stream', + local: true +}); + +await streamClient.initialize(); + +for await (const chunk of streamClient.run({ + message: 'Tell me about the weather' +})) { + if (chunk.content) { + process.stdout.write(chunk.content); + } +} +``` + +### Rust Client + +```rust +use runagent::client::RunAgentClient; +use serde_json::json; +use futures::StreamExt; + +#[tokio::main] +async fn main() -> Result<(), Box> { + // Non-streaming test + let client = RunAgentClient::new( + "your_agent_id_here", + "parlant_simple", + true + ).await?; + + let result = client.run(&[ + ("message", json!("What can you help me with?")) + ]).await?; + + println!("Response: {}", result); + + // Streaming test + let stream_client = RunAgentClient::new( + "your_agent_id_here", + "parlant_stream", + true + ).await?; + + let mut stream = stream_client.run_stream(&[ + ("message", json!("Calculate 100 + 250")) + ]).await?; + + while let Some(chunk) = stream.next().await { + match chunk { + Ok(data) => print!("{}", data), + Err(e) => eprintln!("Error: {}", e) + } + } + + Ok(()) +} +``` + +--- + +## Best Practices + +### 1. **Guideline Design** +- Write clear, specific conditions +- Define concrete actions for agents to take +- Associate appropriate tools with guidelines +- Test guidelines with various inputs + +### 2. **Tool Implementation** +- Keep tools focused and single-purpose +- Handle errors gracefully within tool code +- Use type-safe parameters +- Provide clear descriptions for LLM understanding + +### 3. **Session Management** +- Create sessions per user for context +- Clean up old sessions periodically +- Handle session timeouts appropriately +- Store session metadata when needed + +### 4. **Error Handling** +- Always wrap async operations in try-catch +- Return structured error responses +- Log errors for debugging +- Provide helpful error messages to users + +### 5. **Performance** +- Reuse client connections +- Cache agent IDs +- Implement connection pooling for high traffic +- Monitor API usage and latency + +--- + +## Common Patterns + +### Guideline-Driven Routing +Route user requests based on guidelines: +```python +condition β†’ action β†’ tool_usage +``` + +### Tool Composition +Combine multiple tools for complex tasks: +```python +calculator + weather + time β†’ comprehensive_response +``` + +### Context Preservation +Maintain conversation history: +```python +session_per_user β†’ contextual_responses +``` + +### Escalation Pattern +Handle complex requests: +```python +simple_query β†’ agent_response +complex_query β†’ escalation β†’ support_ticket +``` + +--- + +## Troubleshooting + +### Common Issues + +**1. Server Connection Failed** +- **Solution**: Ensure Parlant server is running with `parlant-server run` +- Check server URL (default: `http://localhost:8800`) +- Verify no firewall blocking port 8800 + +**2. Agent Not Found** +- **Solution**: Agent is created on first run +- Check agent creation logs +- Verify client connection is successful + +**3. Tool Execution Fails** +- **Solution**: Check tool implementation code +- Verify parameter types match specification +- Test tools independently before integration + +**4. Guidelines Not Triggering** +- **Solution**: Make conditions more specific +- Test with various phrasings +- Review guideline condition matching + +**5. Session Timeout** +- **Solution**: Increase `wait_for_data` timeout +- Check server responsiveness +- Monitor server logs for errors + +### Debug Tips + +Enable debug logging: + +```python +import logging +logging.basicConfig(level=logging.DEBUG) + +async def simple_chat(message: str): + print(f"Debug: Processing message: {message}") + # ... rest of code +``` + +Test Parlant server connection: + +```python +from parlant.client import AsyncParlantClient +import asyncio + +async def test_connection(): + try: + client = AsyncParlantClient(base_url="http://localhost:8800") + agents = await client.agents.list() + print(f"βœ… Connected! Found {len(agents)} agents") + except Exception as e: + print(f"❌ Connection failed: {e}") + +asyncio.run(test_connection()) +``` + +--- + +## Performance Optimization + +### 1. **Client Reuse** +Reuse client connections: + +```python +_global_client = None + +async def get_client(): + global _global_client + if _global_client is None: + _global_client = AsyncParlantClient(base_url="http://localhost:8800") + return _global_client +``` + +### 2. **Agent Caching** +Cache agent IDs: + +```python +_agent_cache = {} + +async def get_or_create_agent(agent_type: str): + if agent_type not in _agent_cache: + client = await get_client() + agent_id = await create_agent(client, agent_type) + _agent_cache[agent_type] = agent_id + return _agent_cache[agent_type] +``` + +### 3. **Session Pooling** +Manage session lifecycle: + +```python +from collections import defaultdict +from datetime import datetime, timedelta + +class SessionPool: + def __init__(self, ttl_minutes=30): + self.sessions = defaultdict(dict) + self.ttl = timedelta(minutes=ttl_minutes) + + def get_session(self, user_id: str): + session_data = self.sessions.get(user_id) + if session_data and datetime.now() - session_data['created'] < self.ttl: + return session_data['session_id'] + return None + + def add_session(self, user_id: str, session_id: str): + self.sessions[user_id] = { + 'session_id': session_id, + 'created': datetime.now() + } +``` + +--- + +## Next Steps + +- **[Advanced Patterns](/how-to/advanced-tasks)** - Learn advanced Parlant patterns +- **[Production Deployment](/how-to/deployment/cloud-deployment)** - Deploy to production +- **[Multi-Language Access](/tutorials/multi-language-wrapper)** - Access from different languages +- **[Performance Tuning](/explanation/production-considerations)** - Optimize for production + +--- + +## Additional Resources + +- [Parlant Documentation](https://docs.parlant.io) +- [Parlant GitHub](https://github.com/emcie-co/parlant) +- [Parlant Discord](https://discord.gg/parlant) +- [RunAgent Discord Community](https://discord.gg/Q9P9AdHVHz) +- [RunAgent Documentation](https://docs.run-agent.ai) + +--- + +**πŸŽ‰ Great work!** You've learned how to deploy Parlant guideline-driven agents with RunAgent. Parlant's structured behavior system combined with RunAgent's multi-language access creates powerful, controllable conversational AI systems! \ No newline at end of file diff --git a/docs/how-to/overview.mdx b/docs/how-to/overview.mdx index 5b352a1..151ee1f 100644 --- a/docs/how-to/overview.mdx +++ b/docs/how-to/overview.mdx @@ -15,6 +15,9 @@ Pick a task and follow the steps. Each guide is 5–10 minutes. + + + diff --git a/docs/sdk/go/getting-started.mdx b/docs/sdk/go/getting-started.mdx index 0d52586..8e93a43 100644 --- a/docs/sdk/go/getting-started.mdx +++ b/docs/sdk/go/getting-started.mdx @@ -1,6 +1,7 @@ --- title: 'Getting Started with Go SDK' description: 'Start using the RunAgent Go SDK' +icon: 'golang' --- The RunAgent Go SDK provides a simple and powerful interface for interacting with your deployed agents. It supports both synchronous and asynchronous operations, with built-in streaming capabilities. diff --git a/docs/sdk/python/getting-started.mdx b/docs/sdk/python/getting-started.mdx index fc5b06e..ecaf624 100644 --- a/docs/sdk/python/getting-started.mdx +++ b/docs/sdk/python/getting-started.mdx @@ -1,6 +1,7 @@ --- title: 'Getting Started with Python SDK' description: 'Start using the RunAgent Python SDK' +icon: python --- The RunAgent Python SDK provides a simple and powerful interface for interacting with your deployed agents. It supports both synchronous and asynchronous operations, with built-in streaming capabilities. diff --git a/docs/sdk/rust/getting-started.mdx b/docs/sdk/rust/getting-started.mdx index c3e604d..ac9b2ae 100644 --- a/docs/sdk/rust/getting-started.mdx +++ b/docs/sdk/rust/getting-started.mdx @@ -1,3 +1,10 @@ +--- +title: 'Rust SDK Getting Started' +description: 'Get started with the RunAgent Rust SDK' +icon: 'rust' +--- + + The RunAgent Rust SDK provides a high-performance interface for interacting with your deployed agents. It supports both synchronous and asynchronous operations with built-in streaming capabilities. ## Installation diff --git a/docs/tutorials/deploy-your-first-agent.mdx b/docs/tutorials/deploy-your-first-agent.mdx index 191bae3..bbb26df 100644 --- a/docs/tutorials/deploy-your-first-agent.mdx +++ b/docs/tutorials/deploy-your-first-agent.mdx @@ -179,7 +179,7 @@ Now comes the mind-blowing part. Your Python agent is now accessible from any su **What just happened?** You called your Python function from the command line with zero setup! - + ```python from runagent import RunAgentClient @@ -201,7 +201,7 @@ Now comes the mind-blowing part. Your Python agent is now accessible from any su **The magic**: This feels like calling a local Python function, but it's actually making network calls to your deployed agent! - + First, install in your JS project: ```bash npm install runagent @@ -288,7 +288,7 @@ Watch your Python agent stream responses in real-time to any language: **What's happening**: Your Python generator function is streaming across the network in real-time! - + ```python from runagent import RunAgentClient @@ -309,7 +309,7 @@ Watch your Python agent stream responses in real-time to any language: **The beauty**: This looks like a normal Python iterator, but chunks are streaming from your deployed agent in real-time! - + ```javascript import { RunAgentClient } from 'runagent'; @@ -334,7 +334,7 @@ Watch your Python agent stream responses in real-time to any language: **Mind-blown moment**: JavaScript developers are consuming real-time streams from your Python agent using native async iterators! - + ```rust use runagent::client::RunAgentClient; use serde_json::json; @@ -359,7 +359,7 @@ Watch your Python agent stream responses in real-time to any language: **Performance revelation**: Rust developers get zero-copy streaming from your Python agent with full type safety! - + ```go package main @@ -423,7 +423,7 @@ Ready to move beyond mock responses? RunAgent works with any Python AI framework **What you get**: Multi-step reasoning agents with state management, accessible from any language! - + ```bash runagent init my_crew_agent --crewai cd my_crew_agent @@ -472,12 +472,6 @@ graph TD ROUTER --> PY ROUTER --> JS ROUTER --> RS - - style FUNC fill:#e8f5e8 - style ROUTER fill:#f0f8ff - style PY fill:#ffe8cc - style JS fill:#fff9cc - style RS fill:#ffe8e8 ``` **The Revolutionary Insight**: Your Python function signature automatically defines the API contract for all programming languages. Change the Python function, and all language SDKs automatically adapt! @@ -530,16 +524,16 @@ In these 5 minutes, you've experienced something revolutionary: ## The Path Forward: Your RunAgent Journey - + Understand the architecture that makes this magic possible - + Master the native patterns for each supported language - + See production-ready agents built with popular frameworks - + Scale your agents to serve millions of requests