Guarded RAG Assistant

The guarded RAG assistant is an easily customizable recipe for building a RAG-powered chatbot.

In addition to creating a hosted, shareable user interface, the guarded RAG assistant provides:

• Business logic and LLM-based guardrails.
• A predictive secondary model that evaluates response quality.
• GenAI-focused custom metrics.
• DataRobot MLOps hosting, monitoring, and governing the individual back-end deployments.

Warning

Application templates are intended to be starting points that provide guidance on how to develop, serve, and maintain AI applications. They require a developer or data scientist to adapt and modify them for their business requirements before being put into production.

Table of contents

1. Setup
2. Architecture overview
3. Why build AI Apps with DataRobot app templates?
4. Make changes
   • Change the RAG documents
   • Change the LLM
   • Change the RAG prompt
   • Custom front-end
   • Custom RAG logic
5. Share results
6. Delete all provisioned resources
7. Setup for advanced users
8. Data Privacy

Setup

Prerequisites: If you are using DataRobot Codespaces, this is already complete for you. If not, install:

• Python 3.9+
• uv (Python package manager)
• Pulumi (infrastructure as code)

DataRobot Codespaces users: If you opened this template from the Application Templates gallery, you can skip steps 1 and 2. If you created a fresh codespace, you can skip step 1 but still need to clone the repository (step 2).

For local development, follow all of the following steps:

1. If pulumi is not already installed, install the CLI following instructions here. After installing for the first time, restart your terminal and run:

   pulumi login --local  # omit --local to use Pulumi Cloud (requires separate account)

2. Clone the template repository.

   git clone https://github.com/datarobot-community/guarded-rag-assistant.git
   cd guarded-rag-assistant

3. Rename the file .env.template to .env in the root directory of the repo and populate your credentials.

   DATAROBOT_API_TOKEN=...
   DATAROBOT_ENDPOINT=...  # e.g. https://app.datarobot.com/api/v2
   OPENAI_API_KEY=...
   OPENAI_API_VERSION=...  # e.g. 2024-02-01
   OPENAI_API_BASE=...  # e.g. https://your_org.openai.azure.com/
   OPENAI_API_DEPLOYMENT_ID=...  # e.g. gpt-4o
   PULUMI_CONFIG_PASSPHRASE=...  # Required. Choose your own alphanumeric passphrase to be used for encrypting pulumi config

   Use the following resources to locate the required credentials:

   • DataRobot API Token: Refer to the Create a DataRobot API Key section of the DataRobot API Quickstart docs.
   • DataRobot Endpoint: Refer to the Retrieve the API Endpoint section of the same DataRobot API Quickstart docs.
   • LLM Endpoint and API Key: Refer to the Azure OpenAI documentation.

4. In a terminal, run:

   python quickstart.py YOUR_PROJECT_NAME  # Windows users may have to use `py` instead of `python`

   What does quickstart.py do?

   The quickstart script automates the entire setup process for you:

   • Creates and activates a Python virtual environment
   • Installs all required dependencies (using uv for faster installation, falling back to pip)
   • Loads your .env configuration
   • Sets up the Pulumi stack with your project name
   • Runs pulumi up to deploy your application
   • Displays your application URL when complete

   This single command replaces all the manual steps described in the advanced setup section.

   Python 3.9+ is required.

Advanced users desiring control over virtual environment creation, dependency installation, environment variable setup and pulumi invocation see here.

Architecture overview

App templates contain three families of complementary logic. For Guarded RAG you can opt-in to fully custom RAG logic and a fully custom frontend or utilize DR's off the shelf offerings:

• AI logic: Necessary to service AI requests and produce predictions and completions.

  deployment_*/  # Predictive model scoring logic, RAG completion logic (DIY RAG)
  notebooks/  # Document chunking, VDB creation logic (DIY RAG)
  

• App Logic: Necessary for user consumption; whether via a hosted front-end or integrating into an external consumption layer.

  frontend/  # Streamlit frontend (DIY frontend)
  docsassist/  # App business logic & runtime helpers (DIY front-end)
  

• Operational Logic: Necessary to activate DataRobot assets.

  infra/  # Settings for resources and assets to be created in DataRobot
  infra/__main__.py  # Pulumi program for configuring DataRobot to serve and monitor AI and App logic
  

Why build AI Apps with DataRobot app templates?

App Templates transform your AI projects from notebooks to production-ready applications. Too often, getting models into production means rewriting code, juggling credentials, and coordinating with multiple tools & teams just to make simple changes. DataRobot's composable AI apps framework eliminates these bottlenecks, letting you spend more time experimenting with your ML and app logic and less time wrestling with plumbing and deployment.

• Start building in minutes: Deploy complete AI applications instantly, then customize the AI logic or the front-end independently (no architectural rewrites needed).
• Keep working your way: Data scientists keep working in notebooks, developers in IDEs, and configs stay isolated. Update any piece without breaking others.
• Iterate with confidence: Make changes locally and deploy with confidence. Spend less time writing and troubleshooting plumbing and more time improving your app.

Each template provides an end-to-end AI architecture, from raw inputs to deployed application, while remaining highly customizable for specific business requirements.

Make changes

Change the RAG documents

1. Replace assets/datarobot_english_documentation_docsassist.zip with a new zip file containing .pdf, .docx, .md, or .txt documents (example alternative docs here).
2. Update the rag_documents setting in infra/settings_main.py to specify the local path to the new zip file.
3. Run pulumi up to update your stack.

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

Change the LLM

1. Modify the LLM setting in infra/settings_generative.py by changing LLM=LLMs.AZURE_OPENAI_GPT_4_O to any other LLM from the LLMs object.
   • Trial users: Please set LLM=LLMs.AZURE_OPENAI_GPT_4_O_MINI since GPT-4o is not supported in the trial. Use the OPENAI_API_DEPLOYMENT_ID in .env to override which model is used in your Azure organization. You'll still see GPT 4o-mini in the playground, but the deployed app will use the provided Azure deployment.
2. To use an existing TextGen model or deployment:
   • In infra/settings_generative.py: Set LLM=LLMs.DEPLOYED_LLM.
   • In .env: Set either the TEXTGEN_REGISTERED_MODEL_ID or the TEXTGEN_DEPLOYMENT_ID
   • In .env: Set CHAT_MODEL_NAME to the model name expected by the deployment (e.g. "claude-3-7-sonnet-20250219" for an anthropic deployment,"datarobot-deployed-llm" for NIM models )
3. In .env: If not using an existing TextGen model or deployment, provide the required credentials dependent on your choice.
4. Run pulumi up to update your stack (Or rerun your quickstart).

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

⚠️ Availability information:
Using a NIM model requires custom model GPU inference, a premium feature. You will experience errors by using this type of model without the feature enabled. Contact your DataRobot representative or administrator for information on enabling this feature.

Change the RAG prompt

1. Modify the system_prompt variable in infra/settings_generative.py with your desired prompt.
2. If using fully custom RAG logic, instead please change the stuff_prompt variable in notebooks/build_rag.ipynb.

Fully custom front-end

1. Edit infra/settings_main.py and update application_type to ApplicationType.DIY
   • Optionally, update APP_LOCALE in docsassist/i18n.py to toggle the language. Supported locales are Japanese and English, with English set as the default.
2. Run pulumi up to update your stack with the example custom Streamlit frontend:

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

3. After provisioning the stack at least once, you can also edit and test the Streamlit front-end locally using streamlit run app.py from the frontend/ directory (don't forget to initialize your environment using set_env).

   source set_env.sh  # On windows use `set_env.bat`
   cd frontend
   streamlit run app.py

Fully custom RAG chunking, vectorization, and retrieval

1. Install additional requirements (e.g. FAISS, HuggingFace).

   source set_env.sh  # On windows use `set_env.bat`
   pip install -r requirements-extra.txt

2. Edit infra/settings_main.py and update rag_type to RAGType.DIY.
3. Run pulumi up to update your stack with the example custom RAG logic.

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

4. Edit notebooks/build_rag.ipynb to customize the doc chunking, vectorization logic.
5. Edit deployment_diy_rag/custom.py to customize the retrieval logic & LLM call.
6. Run pulumi up to update your stack.

   source set_env.sh  # On windows use `set_env.bat`
   pulumi up

Share results

1. Log into the DataRobot application.
2. Navigate to Registry > Applications.
3. Navigate to the application you want to share, open the actions menu, and select Share from the dropdown.

Delete all provisioned resources

pulumi down

Setup for advanced users

For manual control over the setup process adapt the following steps for MacOS/Linux to your environent:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
source set_env.sh
pulumi stack init YOUR_PROJECT_NAME
pulumi up

e.g. for Windows/conda/cmd.exe this would be:

conda create --prefix .venv pip
conda activate .\.venv
pip install -r requirements.txt
set_env.bat
pulumi stack init YOUR_PROJECT_NAME
pulumi up

For projects that will be maintained, DataRobot recommends forking the repo so upstream fixes and improvements can be merged in the future.

Data Privacy

Your data privacy is important to us. Data handling is governed by the DataRobot Privacy Policy, please review before using your own data with DataRobot.