Skip to content

IMPLEMENTATION.md

Latest commit

 

History

History
294 lines (240 loc) · 8.96 KB

IMPLEMENTATION.md

File metadata and controls

294 lines (240 loc) · 8.96 KB

RAG Tool Implementation Summary

Project Overview

A complete Retrieval-Augmented Generation (RAG) system built for educational purposes, featuring:

  • FastAPI backend with document processing, vector search, and LLM integration
  • React + TypeScript frontend with Vite
  • Qdrant vector database for semantic search
  • Ollama for local LLM inference (Qwen models)
  • Multiple deployment options (native, Docker)

Implementation Status

✅ Completed Components

Backend (FastAPI)

  • API Routes:

    • /api/v1/documents/upload - Upload and process documents
    • /api/v1/documents/list - List all documents
    • /api/v1/documents/{id} - Delete document
    • /api/v1/documents/sync - Sync from data folder
    • /api/v1/query/query - Non-streaming RAG query
    • /api/v1/query/query/stream - Streaming RAG query (SSE)
    • /api/v1/chat/new - Create chat session
    • /api/v1/chat/list - List chat sessions
    • /api/v1/chat/{id} - Get chat history
    • /api/v1/chat/{id} - Delete chat session

  • Services:

    • DocumentProcessor - Parse and chunk documents (PDF, DOCX, TXT, MD, HTML, XML)
    • EmbeddingService - Generate embeddings using SentenceTransformers
    • QdrantService - Vector database operations
    • LLMService - Ollama integration with streaming support
    • ChatHistoryManager - Persistent chat sessions

  • Features:

    • SHA-256 document hashing for deduplication
    • 512 token chunks with 128 token overlap
    • Cosine similarity search
    • Server-Sent Events for streaming responses
    • JSON-based chat history persistence

Frontend (React + Vite)

  • Components:

    • DocumentManagement - Upload and manage documents
    • QueryInterface - Query with streaming responses
    • Settings panel (basic)

  • Features:

    • File upload with drag-and-drop support
    • Real-time streaming responses via SSE
    • Parameter controls (temperature, top-k, etc.)
    • Retrieved chunks display with scores

Infrastructure

  • Docker Setup:

    • docker-compose.yml - Multi-service orchestration
    • Dockerfile.backend - Backend container
    • Dockerfile.frontend - Frontend with Nginx

  • Scripts:

    • backend/setup.sh - Backend installation
    • scripts/start_qdrant.sh - Start Qdrant
    • scripts/start_ollama.sh - Start Ollama
    • scripts/setup_all.sh - Complete setup
    • scripts/start_all.sh - Start all services (tmux)
    • scripts/stop_all.sh - Stop all services

Documentation

  • Main README with architecture diagram
  • Backend-specific README
  • Frontend-specific README
  • Environment configuration examples
  • MIT License

Project Structure

workshop-rag/
├── backend/                  # FastAPI backend
│   ├── app/
│   │   ├── api/             # API routes
│   │   │   ├── documents.py # Document management
│   │   │   ├── query.py     # RAG queries
│   │   │   └── chat.py      # Chat history
│   │   ├── core/            # Configuration
│   │   ├── schemas/         # Pydantic models
│   │   └── services/        # Business logic
│   ├── pyproject.toml       # Dependencies
│   ├── .env.example         # Config template
│   └── setup.sh             # Setup script
├── frontend/                # React frontend
│   ├── src/
│   │   ├── components/      # React components
│   │   └── services/        # API client
│   ├── package.json         # Dependencies
│   └── vite.config.ts       # Vite config
├── scripts/                 # Setup & startup scripts
├── data/                    # Document storage
├── chat_history/            # Chat sessions
├── qdrant_storage/          # Vector DB storage
├── models/                  # Downloaded models
├── docker-compose.yml       # Docker orchestration
├── Dockerfile.backend       # Backend container
├── Dockerfile.frontend      # Frontend container
└── README.md                # Main documentation

Technical Specifications

Backend

  • Framework: FastAPI
  • Python: 3.10+
  • Package Manager: uv (recommended)
  • Dependencies:
    • fastapi, uvicorn (web server)
    • qdrant-client (vector DB)
    • sentence-transformers (embeddings)
    • httpx (Ollama API client)
    • PyPDF2, python-docx, beautifulsoup4 (document parsing)
    • pydantic, pydantic-settings (configuration)

Frontend

  • Framework: React 18
  • Language: TypeScript
  • Build Tool: Vite
  • Dependencies:
    • react, react-dom
    • axios (HTTP client)
    • react-markdown (optional)

Infrastructure

  • Qdrant: Vector database (port 6333)
  • Ollama: LLM inference server (port 11434)
  • Backend: FastAPI server (port 8000)
  • Frontend: Development server (port 3000)

Models

  • Embedding: sentence-transformers/all-MiniLM-L6-v2 (384 dimensions)
  • LLM: Qwen 2.5 7B Instruct (via Ollama)

Document Processing

  • Supported Formats: PDF, DOCX, TXT, MD, HTML, XML
  • Chunking: 512 tokens per chunk, 128 token overlap
  • Document ID: SHA-256 hash of file content
  • Vector Distance: Cosine similarity

LLM Configuration

  • Context Window: 8192 tokens
  • Default Temperature: 0.7
  • Default Max Tokens: 512
  • Top-p: 0.9
  • Top-k: 40

Installation Options

Option 1: Automated Setup

./scripts/setup_all.sh
./scripts/start_all.sh

Option 2: Manual Setup

# Backend
cd backend && ./setup.sh && source .venv/bin/activate

# Install Ollama and pull model
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5:7b-instruct

# Start services (3 terminals)
./scripts/start_qdrant.sh
ollama serve
uvicorn app.main:app --reload

# Frontend (if Node.js available)
cd frontend && npm install && npm run dev

Option 3: Docker

# Start all services
docker-compose up -d

Usage

  1. Access Frontend: http://localhost:3000
  2. API Documentation: http://localhost:8000/docs
  3. Upload Documents: Use "Upload Documents" tab
  4. Query Documents: Use "Query Documents" tab with streaming responses
  5. View Retrieved Chunks: See sources with similarity scores

Git Repository

Commits

  1. Initial commit: Backend implementation with FastAPI, Qdrant, and Ollama
  2. Complete implementation: Frontend, Docker setup, and documentation

Files Created: 50+

  • 18 Python backend files
  • 11 TypeScript/React frontend files
  • 5 Shell scripts
  • 3 Docker files
  • 3 README files
  • Configuration and environment files

Next Steps for Enhancement

Priority Additions

  1. Chat History UI: Complete chat interface in frontend
  2. Testing: Unit tests for backend, integration tests
  3. Error Handling: Enhanced error messages and recovery
  4. Authentication: Basic auth for document access
  5. Metadata Search: Filter by document type, date, etc.

Future Features

  1. Multi-modal Support: Images, audio transcripts
  2. Advanced Chunking: Semantic chunking strategies
  3. Query Rewriting: Automatic query enhancement
  4. Response Citations: Direct links to document sections
  5. Export/Import: Backup and restore functionality
  6. Monitoring: Prometheus metrics, logging dashboard

Known Limitations

  1. Frontend: Requires Node.js/npm for installation
  2. Memory: ~16GB RAM recommended for running all services
  3. Model Size: Qwen models require varying disk space (1-20GB)
  4. Chat History: Basic implementation, no search functionality

Performance Considerations

  • Document Upload: ~1-5 seconds per document depending on size
  • Query Response: ~2-10 seconds depending on LLM and retrieved chunks
  • Streaming: Real-time token generation reduces perceived latency
  • Vector Search: <100ms for most queries with proper indexing

Security Notes

  • No authentication implemented (educational purpose)
  • CORS configured for localhost only
  • File uploads not sanitized beyond type checking
  • Consider adding authentication for production use

Development Environment

  • Developed on Linux (bash shell)
  • Git repository initialized
  • All scripts have execute permissions
  • Environment variables configured via .env files

Maintenance

Starting Services

./scripts/start_all.sh  # All services in tmux

Stopping Services

./scripts/stop_all.sh

Checking Logs

# Attach to tmux session
tmux attach -t rag-tool

# Switch between windows: Ctrl+b then 0, 1, 2
# Detach: Ctrl+b then d

Updating Models

# Pull new Ollama model
ollama pull qwen2.5:14b-instruct

# Update OLLAMA_MODEL in backend/.env
# Restart backend service

Conclusion

The RAG tool implementation is complete and functional with:

  • ✅ Full backend API with document processing, vector search, and LLM integration
  • ✅ React frontend with document management and query interfaces
  • ✅ Docker deployment option
  • ✅ Comprehensive setup and startup scripts
  • ✅ Documentation and examples
  • ✅ Ollama-based local LLM inference with Qwen models

The system is ready for testing and educational use.