Phase 1: Robot Framework E2E Test Foundation for University Agent

src/orchestrators/workflow-orchestrator/orchestrator/.python-version

@@ -1 +1 @@
-3.11
+3.12

src/orchestrators/workflow-orchestrator/orchestrator/pyproject.toml

@@ -3,7 +3,7 @@ name = "workflow-orchestrator"
 version = "0.1.0"
 description = "Teal Agents - Workflow Orchestrator"
 readme = "README.md"
-requires-python = ">=3.11"
+requires-python = ">=3.12"
 dependencies = [
     "fastapi [standard]",
     "python-dotenv",

src/sk-agents/pyproject.toml

@@ -67,6 +67,9 @@ dev = [
     "mkdocs-material>=9.6.0",
     "mkdocstrings[python]>=0.28.0",
     "mkdocs-static-i18n>=1.3.0",
+    "robotframework>=7.0.0",
+    "robotframework-requests>=0.9.7",
+    "robotframework-seleniumlibrary>=6.6.0",
 ]
 
 [tool.ruff]

src/sk-agents/tests/e2e/README.md

@@ -0,0 +1,146 @@
+# University Agent System E2E Tests
+
+Robot Framework end-to-end tests for the University Agent System.
+
+## Discovered System Components
+
+### 1. UniversityPlugin
+**Location**: `src/orchestrators/assistant-orchestrator/example/university/custom_plugins.py`
+
+The core plugin providing university search functionality:
+- `search_universities(query: str)` - Search universities by name
+- `get_universities_by_country(country: str)` - Find universities in a specific country
+
+**External API**: universities.hipolabs.com
+- No authentication required
+- Returns JSON array of university objects
+- Free tier, no rate limiting mentioned
+
+### 2. Streamlit UI
+**Location**: `src/orchestrators/assistant-orchestrator/example/university/streamlit_ui.py`
+
+Interactive chat interface:
+- Default URL: http://localhost:8001 (configurable)
+- Connects to `/UniversityAgent/0.1` endpoint
+- Chat-based interaction with conversation history
+- Formats university data for user-friendly display
+
+**Key Functions**:
+- `call_university_agent()` - POST requests to agent API
+- `format_university_data()` - Display formatting
+- `check_agent_status()` - Health check
+
+### 3. Agent Configuration
+**Location**: `src/orchestrators/assistant-orchestrator/example/university/config.yaml`
+
+Sequential agent configuration:
+- **API Version**: skagents/v1
+- **Service Name**: UniversityAgent
+- **Version**: 0.1
+- **Model**: gemini-2.0-flash-lite
+- **Plugin**: UniversityPlugin
+
+### 4. FastAPI Service
+**Location**: `src/sk-agents/src/sk_agents/app.py`, `appv1.py`
+
+Dynamic endpoint creation based on configuration:
+- **Endpoint Pattern**: `/{name}/{version}` (e.g., `/UniversityAgent/0.1`)
+- **Docs**: `/{name}/{version}/docs`
+- Supports both REST and WebSocket routes
+- Uses custom chat completion factory
+
+### 5. Gemini Integration
+**Location**: `src/sk-agents/src/sk_agents/chat_completion/custom/gemini_chat_completion_factory.py`
+
+Custom completion factory for Google Gemini:
+- **Supported Models**: gemini-1.5-flash, gemini-1.5-pro, gemini-1.0-pro, gemini-2.0-flash-lite
+- **Environment Variable**: GEMINI_API_KEY
+- Integrates with Semantic Kernel via GoogleAIChatCompletion
+
+## Architecture
+
+```
+User → Streamlit UI (Port 8502)
+         ↓ POST /UniversityAgent/0.1
+      FastAPI Agent Service (Port 8001)
+         ↓ LLM calls
+      Gemini API (gemini-2.0-flash-lite)
+         ↓ Plugin invocation
+      UniversityPlugin
+         ↓ HTTP GET
+      universities.hipolabs.com API
+```
+
+## Test Modes
+
+### Local Mode
+Tests start and manage the FastAPI and Streamlit processes:
+- Starts agent service on port 8001
+- Starts Streamlit UI on port 8502
+- Can mock external APIs
+- Full process lifecycle control
+
+### UAT Mode
+Tests run against already-running instances:
+- Assumes services are already deployed
+- No process management
+- Uses real external APIs (typically)
+- For testing deployed environments
+
+## Configuration
+
+Test configuration is in `resources/environment_configs.yaml`:
+- Service URLs and ports
+- Startup commands (local mode)
+- Environment variables
+- Timeouts and settings
+- Mock server configuration
+
+## Test Data
+
+Test scenarios and expected responses are in:
+- `resources/test_data.yaml` - Test cases and expected results
+- `resources/mock_responses.json` - Canned API responses for mocking
+
+## Phase 1 Status
+
+✅ Directory structure created
+✅ Placeholder test files with basic Robot Framework structure
+✅ Keyword resource files for different testing aspects
+✅ Configuration files for local and UAT modes
+✅ Python library files with class definitions and docstrings
+✅ Dependencies added to pyproject.toml
+✅ System components documented
+
+## Next Steps (Future Phases)
+
+- **Phase 2**: Implement agent lifecycle keywords and basic API tests
+- **Phase 3**: Implement Streamlit UI automation
+- **Phase 4**: Implement external API mocking
+- **Phase 5**: Implement full system integration tests
+- **Phase 6**: CI/CD integration and reporting
+
+## Running Tests (Future)
+
+Once implemented, tests will be run with:
+```bash
+# Local mode
+robot --variable MODE:local tests/e2e/university_agent_system.robot
+
+# UAT mode
+robot --variable MODE:uat tests/e2e/university_agent_system.robot
+```
+
+## Dependencies
+
+Robot Framework libraries:
+- robotframework - Core framework
+- robotframework-requests - HTTP API testing
+- robotframework-process - Process management
+- robotframework-seleniumlibrary - Browser automation
+
+Install with:
+```bash
+cd ~/repos/teal-agents/src/sk-agents
+uv sync
+```

src/sk-agents/tests/e2e/keywords/agent_lifecycle.robot

@@ -0,0 +1,13 @@
+*** Settings ***
+Documentation     Keywords for managing agent lifecycle (start/stop/health checks)
+Library           ../libraries/ProcessManager.py
+
+*** Keywords ***
+Start University Agent
+    [Documentation]    Start the FastAPI agent service
+
+Stop University Agent
+    [Documentation]    Stop the FastAPI agent service
+
+Check Agent Health
+    [Documentation]    Verify agent is responding

src/sk-agents/tests/e2e/keywords/api_testing.robot

@@ -0,0 +1,12 @@
+*** Settings ***
+Documentation     Keywords for HTTP request/response testing
+Library           RequestsLibrary
+
+*** Keywords ***
+Send Agent Request
+    [Documentation]    Send a POST request to the agent API
+    [Arguments]    ${endpoint}    ${payload}
+
+Verify Agent Response
+    [Documentation]    Verify the agent response structure and content
+    [Arguments]    ${response}

src/sk-agents/tests/e2e/keywords/external_mocking.robot

@@ -0,0 +1,14 @@
+*** Settings ***
+Documentation     Keywords for mock setup/teardown of external APIs
+Library           ../libraries/MockServer.py
+
+*** Keywords ***
+Start Mock Universities API
+    [Documentation]    Start mock server for universities.hipolabs.com
+
+Stop Mock Universities API
+    [Documentation]    Stop the mock universities API server
+
+Configure Mock Response
+    [Documentation]    Set up a specific mock response
+    [Arguments]    ${endpoint}    ${response_data}

src/sk-agents/tests/e2e/keywords/streamlit_automation.robot

@@ -0,0 +1,15 @@
+*** Settings ***
+Documentation     Keywords for Streamlit UI automation
+Library           SeleniumLibrary
+Library           ../libraries/StreamlitHelper.py
+
+*** Keywords ***
+Open Streamlit UI
+    [Documentation]    Open the Streamlit application in browser
+
+Send Chat Message
+    [Documentation]    Send a message in the Streamlit chat interface
+    [Arguments]    ${message}
+
+Verify University Results Displayed
+    [Documentation]    Check that university search results are shown

src/sk-agents/tests/e2e/libraries/MockServer.py

@@ -0,0 +1,53 @@
+"""External API mocking utilities for E2E tests.
+
+This module provides mock server functionality for external APIs
+like universities.hipolabs.com and Gemini API during testing.
+
+TODO: Implement in Phase 2
+"""
+
+from typing import Any
+
+
+class MockServer:
+    """Mock server for external API responses."""
+
+    def __init__(self, port: int = 8888):
+        """Initialize the mock server.
+
+        Args:
+            port: Port number for the mock server
+        """
+        self.port = port
+        self.server = None
+        self.routes: dict[str, Any] = {}
+
+    def start(self) -> bool:
+        """Start the mock server.
+
+        Returns:
+            True if server started successfully, False otherwise
+        """
+        raise NotImplementedError("To be implemented in Phase 2")
+
+    def stop(self) -> bool:
+        """Stop the mock server.
+
+        Returns:
+            True if server stopped successfully, False otherwise
+        """
+        raise NotImplementedError("To be implemented in Phase 2")
+
+    def add_route(self, path: str, method: str, response: Any) -> None:
+        """Add a mock route with specified response.
+
+        Args:
+            path: URL path for the route
+            method: HTTP method (GET, POST, etc.)
+            response: Response data to return
+        """
+        raise NotImplementedError("To be implemented in Phase 2")
+
+    def clear_routes(self) -> None:
+        """Clear all configured routes."""
+        raise NotImplementedError("To be implemented in Phase 2")

src/sk-agents/tests/e2e/libraries/ProcessManager.py

@@ -0,0 +1,57 @@
+"""Process lifecycle management for E2E tests.
+
+This module provides utilities for starting, stopping, and monitoring
+the University Agent and Streamlit UI processes during testing.
+
+TODO: Implement in Phase 2
+"""
+
+import subprocess
+
+
+class ProcessManager:
+    """Manages process lifecycle for E2E tests."""
+
+    def __init__(self):
+        """Initialize the ProcessManager."""
+        self.processes: dict[str, subprocess.Popen] = {}
+
+    def start_process(self, name: str, command: str, timeout: int = 30) -> bool:
+        """Start a process and wait for it to be ready.
+
+        Args:
+            name: Identifier for the process
+            command: Shell command to execute
+            timeout: Seconds to wait for process startup
+
+        Returns:
+            True if process started successfully, False otherwise
+        """
+        raise NotImplementedError("To be implemented in Phase 2")
+
+    def stop_process(self, name: str, timeout: int = 10) -> bool:
+        """Stop a running process gracefully.
+
+        Args:
+            name: Identifier for the process
+            timeout: Seconds to wait for graceful shutdown
+
+        Returns:
+            True if process stopped successfully, False otherwise
+        """
+        raise NotImplementedError("To be implemented in Phase 2")
+
+    def is_process_running(self, name: str) -> bool:
+        """Check if a process is currently running.
+
+        Args:
+            name: Identifier for the process
+
+        Returns:
+            True if process is running, False otherwise
+        """
+        raise NotImplementedError("To be implemented in Phase 2")
+
+    def cleanup_all(self):
+        """Stop all managed processes."""
+        raise NotImplementedError("To be implemented in Phase 2")