UdaPlayAgent

AI game research agent that answers video game questions with a local ChromaDB knowledge base and Tavily web-search fallback.

Overview

UdaPlayAgent is a notebook-driven AI research assistant for video game questions. It combines a local retrieval pipeline built on ChromaDB with a custom agent that decides when internal knowledge is enough and when a live web search is needed.

The project is split into two parts. Part 1 ingests a small structured game dataset into a persistent vector database for semantic search. Part 2 builds an agent around that database using three tools: local retrieval, retrieval evaluation, and Tavily web search fallback. The result is a simple but clear example of a RAG-plus-agent workflow for domain-specific question answering.

Problem Statement

Gaming analysts, product teams, and players often need fast answers to questions about release dates, publishers, platforms, and related game information. A single static dataset is useful for known titles, but it becomes insufficient when a question depends on newer or missing information.

This project addresses that gap with a two-stage strategy:

• answer from a local vector database whenever possible
• evaluate whether the retrieved evidence is sufficient
• fall back to web search when the local data is not enough

Key Features

Core Capabilities

• Persistent game index: Loads JSON game records into a local ChromaDB collection for semantic retrieval.
• Custom retrieval tool: Searches the vector database and returns structured game metadata.
• LLM-based retrieval evaluation: Judges whether retrieved context is strong enough to answer the user’s question.
• Web-search fallback: Uses Tavily when the local database cannot answer confidently.
• State-machine agent flow: Orchestrates tool use, message history, and final answer generation through custom reusable abstractions in lib/.

Methodology

Part 1: Offline RAG Setup

1. Load game JSON files from games/.
2. Create a persistent ChromaDB client.
3. Build or load the udaplay collection with embeddings.
4. Index each game as a searchable document with metadata.
5. Validate retrieval with semantic search queries.

Part 2: Agent Workflow

1. Load the same persistent collection created in Part 1.
2. Define retrieve_game to search the local vector database.
3. Define evaluate_retrieval to judge document usefulness.
4. Define game_web_search to query Tavily when confidence is low.
5. Run the custom Agent to combine tool outputs into a structured answer.

Architecture / Pipeline

Game JSON Files -> Embeddings -> ChromaDB Collection

User Question
  -> retrieve_game
  -> evaluate_retrieval
  -> if insufficient: game_web_search
  -> agent synthesis
  -> final answer with sources when web search is used

Project Structure

UdaPlayAgent/
├── README.md
├── .env.example
├── requirements.txt
├── Udaplay_01_starter_project.ipynb
├── Udaplay_02_starter_project.ipynb
├── games/
│   ├── 001.json
│   ├── ...
│   └── 015.json
└── lib/
    ├── agents.py
    ├── documents.py
    ├── evaluation.py
    ├── llm.py
    ├── loaders.py
    ├── memory.py
    ├── messages.py
    ├── parsers.py
    ├── rag.py
    ├── state_machine.py
    ├── tooling.py
    └── vector_db.py

File Guide

File / Folder	Purpose
Udaplay_01_starter_project.ipynb	Builds the persistent ChromaDB game collection and validates semantic retrieval.
Udaplay_02_starter_project.ipynb	Defines the agent tools and demonstrates end-to-end question answering.
games/	Source dataset of game records used for indexing.
lib/vector_db.py	Vector store wrapper and ChromaDB manager utilities.
lib/agents.py	Custom agent abstraction with tool execution and short-term session memory.
lib/state_machine.py	Generic state-machine implementation used by the agent and RAG flow.
lib/tooling.py	Tool schema generation and tool decorator helpers.
lib/rag.py	Standalone retrieval-augment-generate workflow implementation.

Setup

Prerequisites

• Python 3.11+
• Access to the OpenAI-compatible endpoint used by the starter code
• A Tavily API key for web-search fallback
• JupyterLab, VS Code notebooks, or another notebook runner

1. Clone the Repository

git clone https://github.com/gour6380/UdaPlayAgent.git
cd UdaPlayAgent

2. Install Dependencies

python3 -m pip install -r requirements.txt

3. Create Your .env File

Start from the included template:

cp .env.example .env

Then open .env and fill in your keys:

OPENAI_API_KEY="your-openai-compatible-key"
CHROMA_OPENAI_API_KEY="your-openai-compatible-key"
TAVILY_API_KEY="your-tavily-key"
OPENAI_MODEL="gpt-4o-mini"

Notes:

• CHROMA_OPENAI_API_KEY can reuse the same value as OPENAI_API_KEY.
• Part 2 requires both OPENAI_API_KEY and TAVILY_API_KEY.
• Part 1 can fall back to local sentence-transformer embeddings if no Chroma/OpenAI embedding key is available, but the agent notebook still needs the API keys above.
• The current implementation is configured for the course's Vocareum endpoint (https://openai.vocareum.com/v1). If you want to use the public OpenAI API directly, update the endpoint handling in lib/llm.py and the embedding setup cells in the notebooks.

How to Run

1. Build the Local Game Index

Open Udaplay_01_starter_project.ipynb in JupyterLab or VS Code and run the notebook from top to bottom.

Optional launch command:

jupyter lab

This notebook:

• loads environment variables
• creates a persistent ChromaDB collection named udaplay
• ingests the game dataset from games/
• validates retrieval with a sample semantic query

2. Run the Agent Notebook

Open Udaplay_02_starter_project.ipynb and run it after Part 1 has completed.

This notebook:

• reconnects to the persistent ChromaDB collection
• defines the three project tools
• configures the custom agent instructions
• runs sample game-industry questions through the tool-enabled workflow

Example Queries and Observed Outputs

The executed notebook outputs already demonstrate the following results:

Query	Observed Result
racing games for PlayStation	The Part 1 retrieval check returns Gran Turismo as the top result, followed by Gran Turismo 5.
When Pokémon Gold and Silver was released?	The agent answers that Pokémon Gold and Silver was released in 1999 for Game Boy Color.
Which one was the first 3D platformer Mario game?	The agent identifies Super Mario 64 as the first 3D Mario platformer and notes its 1996 release on Nintendo 64.
Was Mortal Kombat X released for Playstation 5?	The agent rejects the local retrieval as insufficient, uses web search, and concludes that Mortal Kombat X was not released specifically for PS5, though it is playable through backward compatibility.

Limitations

• The local dataset is intentionally small, so many real-world questions require web fallback.
• The primary workflow is notebook-based rather than packaged as a CLI or web app.
• Long-term memory utilities exist in lib/memory.py, but they are not fully integrated into the demonstrated notebook agent flow.
• The project demonstrates qualitative behavior and sample outputs, not a formal benchmark or evaluation suite for retrieval quality.

Future Improvements

• Expand the local game dataset with more titles, studios, and platform coverage.
• Persist web-derived facts back into long-term memory for future sessions.
• Add automated tests for tools, state transitions, and retrieval behavior.
• Expose the agent through a lightweight API or chat interface.
• Improve retrieval ranking and fallback logic with stricter confidence thresholds.