diff --git a/.github/workflows/docker-publish.yml b/.github/workflows/docker-publish.yml index 8a7be988..7b498b06 100644 --- a/.github/workflows/docker-publish.yml +++ b/.github/workflows/docker-publish.yml @@ -3,7 +3,7 @@ name: Build and Publish Docker Image on: push: branches: - - main + - '**' tags: - 'v*' pull_request: diff --git a/CLAUDE.md b/CLAUDE.md index 642a4fbe..b4322b46 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,33 +1,62 @@ -# CLAUDE.md - -This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. - ## Project Overview +ExtraSuite is an open source project (https://github.com/think41/extrasuite) that enables an AI agent such as claude code or codex to get temporary service account tokens on behalf of the user so that it can read or edit google sheets, docs or slides. Each end user gets their own private service account. `extrasuite-server` generates this service account on first access and returns a short lived token. End users must share the google drive file with this service account, and then instruct their AI agent to access the file. This ensures the AI agent has temporary access to the files. It also ensures that any edits made by the AI show up in the version history clearly attributed to the service account email, thereby maintaining an audit trail. -ExtraSuite is a headless authentication service for CLI tools accessing Google Workspace APIs. It enables users to obtain short-lived service account tokens for interacting with Google Sheets, Docs, and Drive. +This project publishes a public docker image on google cloud artifact registry. Each organization or group that wishes to use this project must deploy the container via google cloud run in a google cloud project. In addition, to authenticate end users belonging to that organization or group, they must setup OAuth credentials in google cloud. +## Packages The project consists of three packages: -1. **extrasuite-server** - Containerized fastapi based application to provide employee specific service account and short lived acccess tokens that can be used to call google drive/sheets/docs/slides APIs. It also has minimal UI to allow employees to install skills via a command line installation command. -2. **extrasuite-client** - Package that has a CLI based application to call extrasuite-server on behalf of an LLM based agent and provide it shortlived access tokens. -3. **website** - mkdocs based documentation website, hosted on github pages, automatically deployed to https://extrasuite.think41.com on every commit to main branch. +1. **extrasuite-server** - Containerized FastAPI application to provide employee-specific service accounts and short-lived access tokens that can be used to call google drive/sheets/docs/slides APIs. It also has minimal UI to allow employees to install skills via a command line installation command. +2. **extrasuite-client** - CLI based application to call extrasuite-server from an AI Agent and provide it shortlived access tokens. This will eventually be published to a pypi package, but currently extrasuite-client/src/extrasuite_client/credentials.py is manually copied by the projects that wish to use it. +3. **website** - mkdocs based documentation website, hosted on github pages, automatically deployed to https://extrasuite.think41.com on every commit to main branch. The website also has instructions to deploy, end user documentation and other product usage. + +## Skills for Slides, Docs and Sheets Specific Packages +The AI agent needs instructions on how to read or edit google drive files. These are provided as "Agent Skills" which are an open standard. See https://agentskills.io/home. At its core, a skill is a markdown file /SKILL.md that is saved by end users at a well known agent specific location. The skills are distributed by extrasuite-server, see extrasuite-server/skills. + +In addition to instructions, the AI agent needs python libraries to manipulate files. We have 3 related open source python projects. On a developer machine, each of these projects is cloned in folders parallel to the root of this project. + +The SKILL.md file explains how to use `extrasuite-client` to get the temporary service account token, and then use one of the following projects to read or edit the specific file type. + +1. **gsheetx** - See https://github.com/think41/gsheetx, forked from gspread, provides methods to manipulate google sheets. +1. **gslidex** - See https://github.com/think41/gslidex. "Pulls" google slides into an XML file called Slide Markup Language or SML. AI agents make edits to this XML file. A "diff" process identifies the exact changes that need to be carried out, and then "push" invokes appropriate google slides API to ensure the diffs are applied. This gives AI agents a simpler model to edit google slides. This library is alpha quality. +1. **gdocx** - See https://github.com/think41/gdocx. Similar workflow to gslidex, but the intermediate format is an HTML file representing the google doc. This is under development and not meant for end users yet. + +Currently, these three packages haven't been published to PyPI. Only the gsheetx skill is working, and it uses the underlying gspread library directly. The wrapper code is in `extrasuite-server/skills/gsheetx/gsheet_utils.py`, which will be replaced once gsheetx is published to PyPI. + +## Organization Setup (prerequisite) + +Before end users can use ExtraSuite, an administrator must deploy extrasuite-server for their organization: + +1. **Create a Google Cloud project** with billing enabled +2. **Enable required APIs**: IAM, Service Account Credentials, Firestore, Drive, Sheets, Docs, Slides +3. **Configure OAuth consent screen** and create OAuth 2.0 credentials (Web application type) +4. **Create a Firestore database** in the project +5. **Deploy extrasuite-server to Cloud Run** using the public image: `asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server` +6. **Set environment variables** on Cloud Run: `GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`, `GOOGLE_CLOUD_PROJECT`, `SECRET_KEY` +7. **Share the Cloud Run URL** with end users in the organization + +See `website/docs/deployment/` for detailed deployment instructions. ## User Flow -1. User logs in to the **extrasuite-server** using OAuth via their google workspace or gmail account -2. This creates a 1:1 service acccount for the employee, and grants this service account read permissions for google drive API and read/write permissions to slides/docs/sheets API. We don't create credentials for this service account. -3. User copies the ` | sh` script to install the skill. -4. User instructs agent to access the sheet -5. Agent calls `CredentialsManager` in `extrasuite-client` to get a short lived access token -6. `CredentialsManager` returns the cached token if available, otherwise -7. `CredentialsManager` starts a http server on random port, then opens browser to `/api/token/auth?port=`. -8. Alternatively, it prints the URL and asks user to authenticate. -9. User is redirected to google to authenticate, and then redirected back to `/api/auth/callback` after authentication -10. `extrasuite-server` `/api/auth/callback` is invoked. It redirects the browser back to http://localhost:/on-authentication?code= and/or displays the to the user. -11. `CredentialsManager` then calls `/api/token/exchange` with the auth code to get the token. -12. At this point, `extrasuite-server` impersonates the user specific service account using server credentials. Then it returns a short lived access token back to the `CredentialsManager` in `extrasuite-client`. -13. `CredentialsManager` saves the token + service account email + expiry on disk with appropriate linux permissions. -14. `CredentialsManager` provides the token + service account email to the LLM Agent -15. LLM Agent then writes python code to make API calls to google sides/sheets/docs/drive directly from the user's device -16. Once the token expires, the same flow repeats. If the user has a valid session with `extrasuite-server` - the browser will open but authentication against google server will be skipped. This will result in browser opening and closing in a few seconnds. + +### One-Time Setup (per user) +1. User logs in to **extrasuite-server** via OAuth using their Google Workspace or Gmail account +2. Server creates a dedicated service account for this user, granting it read access to Google Drive API and read/write access to Sheets/Docs/Slides APIs (no credentials are stored for this service account) +3. User copies the skill installation command from the server UI and runs it to install the skill into their AI agent (e.g., Claude Code's `~/.claude/commands/` directory) + +### Runtime Flow (each time agent needs access) +4. User shares a Google Drive file with their service account email, then instructs the agent to access it +5. Agent invokes the skill, which calls `CredentialsManager` (in `extrasuite-client`) to get a short-lived access token +6. If a valid cached token exists in `~/.config/extrasuite/token.json`, it's returned immediately +7. Otherwise, `CredentialsManager` starts a local HTTP server on a random port and opens the browser to `/api/token/auth?port=` (or prints the URL if browser launch fails) +8. User authenticates with Google, then is redirected to `/api/auth/callback` +9. Server redirects browser to `http://localhost:/on-authentication?code=` (also displays the code for manual entry if needed) +10. `CredentialsManager` exchanges the auth code via `/api/token/exchange` +11. Server uses domain-wide delegation to impersonate the user's service account and returns a short-lived access token +12. `CredentialsManager` caches the token locally (with 600 permissions) and provides it to the agent +13. Agent uses the token to make Google API calls directly from the user's device + +### Token Refresh +When the token expires, the runtime flow repeats from step 7. If the user still has a valid session with extrasuite-server, the browser opens briefly and closes automatically (no re-authentication required). ## Development Commands @@ -83,28 +112,39 @@ gcloud firestore databases create --location=asia-south1 --project= - **Server-side:** Tokens are not stored on the server. They are generated on demand and returned immediately to the client. - **Client-side:** Short-lived SA tokens in `~/.config/extrasuite/token.json` -## Testing (Auth Flows) +## Testing -Use `extrasuite-client/examples/basic_usage.py` to validate the three main flows. Replace `` with your deployed server URL (e.g., `http://localhost:8001` for local development): +Due to tight integration with Google Cloud APIs, local testing is impractical. Developers should set up their own Google Cloud project following the same steps as Organization Setup (see `website/docs/deployment/`). -1. **First run (no cache):** token file missing, browser opens, user authenticates. - ```bash - rm -f ~/.config/extrasuite/token.json - PYTHONPATH=extrasuite-client/src python3 extrasuite-client/examples/basic_usage.py \ - --server https:// - ``` -2. **Cached token:** token file present and valid, no browser. - ```bash - PYTHONPATH=extrasuite-client/src python3 extrasuite-client/examples/basic_usage.py \ - --server https:// - ``` -3. **Session reuse (no cache, no re-auth):** delete token cache, browser opens, SSO/session skips login. +### Deploy from a branch + +To test changes before merging to main: + +1. Push your changes to a feature branch on GitHub +2. GitHub Actions automatically builds and pushes a container tagged with the branch name +3. Deploy the branch image to your Cloud Run instance: ```bash - rm -f ~/.config/extrasuite/token.json - PYTHONPATH=extrasuite-client/src python3 extrasuite-client/examples/basic_usage.py \ - --server https:// + gcloud run deploy extrasuite-server \ + --image asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server: \ + --region asia-southeast1 \ + --project ``` +### Validate auth flows + +Use `extrasuite-client/examples/basic_usage.py` to test the three main authentication scenarios: + +1. **First run (no cache):** Token file missing, browser opens, user authenticates +2. **Cached token:** Token file present and valid, no browser opens +3. **Session reuse:** Delete token cache, browser opens but SSO skips login + +```bash +# Clear cache and test fresh authentication +rm -f ~/.config/extrasuite/token.json +PYTHONPATH=extrasuite-client/src python3 extrasuite-client/examples/basic_usage.py \ + --server https:// +``` + ## Exception Handling The server uses centralized exception handling via FastAPI's `add_exception_handler`. Follow these principles: @@ -129,7 +169,7 @@ Docker images are automatically built and published to Google Artifact Registry **Automatic tagging:** | Trigger | Tags Created | |---------|--------------| -| Push to `main` | `main`, `sha-` | +| Push to any branch | ``, `sha-` | | Git tag `v*` | ``, `latest` | | Pull request | Build only (no push) | diff --git a/website/docs/deployment/cloud-run.md b/website/docs/deployment/cloud-run.md index 8533c9ec..404e84a6 100644 --- a/website/docs/deployment/cloud-run.md +++ b/website/docs/deployment/cloud-run.md @@ -1,80 +1,235 @@ -# Deploying to Cloud Run +# Step-by-Step Deployment Guide -This guide covers deploying the ExtraSuite server to Google Cloud Run. +This guide walks you through deploying ExtraSuite to Google Cloud Run. Each step includes both **gcloud CLI** commands and **Google Cloud Console** instructions. -## Prerequisites +--- -1. **Google Cloud Project** with billing enabled -2. **gcloud CLI** installed and configured +## Before You Begin -## Step 1: Enable Required APIs +**If using gcloud CLI:** Open a terminal and set your project ID. You'll use this variable throughout the guide. ```bash export PROJECT_ID=your-project-id +``` + +**If using Google Cloud Console:** Open [console.cloud.google.com](https://console.cloud.google.com) and select your project from the project dropdown at the top of the page. + +--- + +## Step 1: Enable Required APIs + +ExtraSuite needs several Google Cloud APIs to function. This step enables them. + +### Using gcloud CLI +```bash gcloud services enable \ run.googleapis.com \ firestore.googleapis.com \ iam.googleapis.com \ iamcredentials.googleapis.com \ secretmanager.googleapis.com \ + drive.googleapis.com \ + sheets.googleapis.com \ + docs.googleapis.com \ + slides.googleapis.com \ --project=$PROJECT_ID ``` +### Using Google Cloud Console + +1. Go to **APIs & Services > Library** ([direct link](https://console.cloud.google.com/apis/library)) +2. Search for and enable each of the following APIs: + - Cloud Run Admin API + - Cloud Firestore API + - Identity and Access Management (IAM) API + - IAM Service Account Credentials API + - Secret Manager API + - Google Drive API + - Google Sheets API + - Google Docs API + - Google Slides API + +**Verify:** After enabling, you should see all APIs listed in **APIs & Services > Enabled APIs**. + +--- + ## Step 2: Create Firestore Database +ExtraSuite uses Firestore to store user records and session data. Collections are created automatically when the server first runs. + +### Using gcloud CLI + ```bash -gcloud firestore databases create --location=asia-southeast1 --project=$PROJECT_ID +gcloud firestore databases create \ + --location=asia-southeast1 \ + --project=$PROJECT_ID ``` -Collections are created automatically on first use. +!!! note "Choose a location close to your users" + Replace `asia-southeast1` with your preferred [Firestore location](https://cloud.google.com/firestore/docs/locations). Common choices: `us-central1`, `europe-west1`, `asia-southeast1`. + +### Using Google Cloud Console + +1. Go to **Firestore** ([direct link](https://console.cloud.google.com/firestore)) +2. Click **Create Database** +3. Select **Native mode** (not Datastore mode) +4. Choose a location close to your users +5. Click **Create Database** + +**Verify:** The Firestore page should show "No collections yet" - this is expected. + +--- + +## Step 3: Configure OAuth Consent Screen + +Before creating OAuth credentials, you must configure the consent screen that users see when logging in. + +### Using Google Cloud Console + +1. Go to **APIs & Services > OAuth consent screen** ([direct link](https://console.cloud.google.com/apis/credentials/consent)) + +2. **Select User Type:** + - Choose **Internal** if all users are in your Google Workspace organization + - Choose **External** if users have personal Gmail accounts or are from multiple organizations + +3. Click **Create** + +4. **Fill in App Information:** + - **App name:** `ExtraSuite` (or your preferred name) + - **User support email:** Your email address + - **Developer contact email:** Your email address + +5. Click **Save and Continue** + +6. **Scopes:** Click **Save and Continue** (no additional scopes needed) + +7. **Test users (External only):** Add email addresses of users who can test before verification. Click **Save and Continue**. + +8. **Summary:** Review and click **Back to Dashboard** + +**Verify:** The OAuth consent screen page should show your app name with "Publishing status" displayed. + +--- + +## Step 4: Create OAuth Credentials + +Create the OAuth client ID and secret that ExtraSuite uses to authenticate users. + +### Using Google Cloud Console + +1. Go to **APIs & Services > Credentials** ([direct link](https://console.cloud.google.com/apis/credentials)) + +2. Click **Create Credentials > OAuth client ID** + +3. **Application type:** Select **Web application** + +4. **Name:** Enter `ExtraSuite Server` + +5. **Authorized redirect URIs:** Click **Add URI** and enter: + ``` + https://placeholder.example.com/api/auth/callback + ``` + (You'll update this with your actual URL after deployment in Step 7) -## Step 3: Create OAuth 2.0 Credentials +6. Click **Create** -1. Go to [Google Cloud Console > APIs & Services > Credentials](https://console.cloud.google.com/apis/credentials) -2. Click **Create Credentials** > **OAuth client ID** -3. Select **Web application** -4. Add authorized redirect URIs: - - `https://your-domain.com/api/auth/callback` - - `http://localhost:8001/api/auth/callback` (for development) -5. Save the **Client ID** and **Client Secret** +7. **Save your credentials:** A dialog shows your Client ID and Client Secret. Copy both values - you'll need them in Step 6. -## Step 4: Create Service Account for Cloud Run +!!! warning "Keep your Client Secret secure" + The Client Secret is like a password. Don't share it or commit it to version control. + +**Verify:** The Credentials page should list your new OAuth client under "OAuth 2.0 Client IDs". + +--- + +## Step 5: Create Service Account for ExtraSuite + +ExtraSuite needs a service account with permissions to create user service accounts and generate access tokens. + +### Using gcloud CLI + +**Create the service account:** ```bash -# Create service account gcloud iam service-accounts create extrasuite-server \ --display-name="ExtraSuite Server" \ --project=$PROJECT_ID +``` + +**Grant required permissions:** -# Grant Firestore access +```bash +# Permission to read/write Firestore gcloud projects add-iam-policy-binding $PROJECT_ID \ --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/datastore.user" -# Grant service account admin (for creating user SAs) +# Permission to create service accounts for users gcloud projects add-iam-policy-binding $PROJECT_ID \ --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/iam.serviceAccountAdmin" -# Grant token creator (for impersonation) +# Permission to generate access tokens gcloud projects add-iam-policy-binding $PROJECT_ID \ --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ --role="roles/iam.serviceAccountTokenCreator" ``` -## Step 5: Store Secrets in Secret Manager +### Using Google Cloud Console + +**Create the service account:** + +1. Go to **IAM & Admin > Service Accounts** ([direct link](https://console.cloud.google.com/iam-admin/serviceaccounts)) +2. Click **Create Service Account** +3. **Service account name:** `extrasuite-server` +4. **Service account ID:** Leave as auto-generated (`extrasuite-server`) +5. Click **Create and Continue** +6. Click **Done** (we'll add roles next) + +**Grant permissions:** + +1. Go to **IAM & Admin > IAM** ([direct link](https://console.cloud.google.com/iam-admin/iam)) +2. Click **Grant Access** +3. **New principals:** Enter `extrasuite-server@YOUR_PROJECT_ID.iam.gserviceaccount.com` +4. **Assign roles:** Add these three roles (click **Add Another Role** between each): + - `Cloud Datastore User` + - `Service Account Admin` + - `Service Account Token Creator` +5. Click **Save** + +**Verify:** In **IAM & Admin > IAM**, you should see `extrasuite-server@...` with three roles listed. + +--- + +## Step 6: Store Secrets in Secret Manager + +Store your OAuth credentials securely using Secret Manager. + +### Using gcloud CLI ```bash -# Create secrets (replace with your actual values) -echo -n "your-oauth-client-id" | gcloud secrets create extrasuite-client-id \ - --data-file=- --project=$PROJECT_ID -echo -n "your-oauth-client-secret" | gcloud secrets create extrasuite-client-secret \ - --data-file=- --project=$PROJECT_ID +# Store OAuth Client ID +echo -n "YOUR_CLIENT_ID" | gcloud secrets create extrasuite-client-id \ + --data-file=- \ + --project=$PROJECT_ID + +# Store OAuth Client Secret +echo -n "YOUR_CLIENT_SECRET" | gcloud secrets create extrasuite-client-secret \ + --data-file=- \ + --project=$PROJECT_ID + +# Generate and store a random secret key for session signing echo -n "$(openssl rand -base64 32)" | gcloud secrets create extrasuite-secret-key \ - --data-file=- --project=$PROJECT_ID + --data-file=- \ + --project=$PROJECT_ID +``` + +Replace `YOUR_CLIENT_ID` and `YOUR_CLIENT_SECRET` with the values from Step 4. + +**Grant the service account access to read these secrets:** -# Grant Cloud Run access to secrets +```bash for secret in extrasuite-client-id extrasuite-client-secret extrasuite-secret-key; do gcloud secrets add-iam-policy-binding $secret \ --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ @@ -83,7 +238,47 @@ for secret in extrasuite-client-id extrasuite-client-secret extrasuite-secret-ke done ``` -## Step 6: Deploy to Cloud Run +### Using Google Cloud Console + +**Create the secrets:** + +1. Go to **Security > Secret Manager** ([direct link](https://console.cloud.google.com/security/secret-manager)) + +2. Click **Create Secret** + - **Name:** `extrasuite-client-id` + - **Secret value:** Paste your OAuth Client ID from Step 4 + - Click **Create Secret** + +3. Click **Create Secret** again + - **Name:** `extrasuite-client-secret` + - **Secret value:** Paste your OAuth Client Secret from Step 4 + - Click **Create Secret** + +4. Click **Create Secret** again + - **Name:** `extrasuite-secret-key` + - **Secret value:** Generate a random string (you can use an online generator or run `openssl rand -base64 32` in a terminal) + - Click **Create Secret** + +**Grant access to each secret:** + +For each of the three secrets: + +1. Click the secret name to open it +2. Go to the **Permissions** tab +3. Click **Grant Access** +4. **New principals:** `extrasuite-server@YOUR_PROJECT_ID.iam.gserviceaccount.com` +5. **Role:** `Secret Manager Secret Accessor` +6. Click **Save** + +**Verify:** Each secret should show `extrasuite-server@...` in its Permissions tab. + +--- + +## Step 7: Deploy to Cloud Run + +Now deploy ExtraSuite using the pre-built Docker image. + +### Using gcloud CLI ```bash gcloud run deploy extrasuite-server \ @@ -92,29 +287,24 @@ gcloud run deploy extrasuite-server \ --region=asia-southeast1 \ --allow-unauthenticated \ --set-env-vars="GOOGLE_CLOUD_PROJECT=$PROJECT_ID" \ - --set-env-vars="BASE_DOMAIN=your-domain.com" \ + --set-env-vars="BASE_DOMAIN=placeholder.run.app" \ --set-secrets="SECRET_KEY=extrasuite-secret-key:latest" \ --set-secrets="GOOGLE_CLIENT_ID=extrasuite-client-id:latest" \ --set-secrets="GOOGLE_CLIENT_SECRET=extrasuite-client-secret:latest" \ --project=$PROJECT_ID ``` -## Step 7: Update OAuth Redirect URI - -Get your Cloud Run URL: +After deployment, get your service URL: ```bash SERVICE_URL=$(gcloud run services describe extrasuite-server \ --region=asia-southeast1 \ --project=$PROJECT_ID \ --format='value(status.url)') -echo $SERVICE_URL +echo "Your ExtraSuite URL: $SERVICE_URL" ``` -Update your OAuth credentials in Google Cloud Console to include: -`https://your-cloud-run-url/api/auth/callback` - -If not using a custom domain, update the BASE_DOMAIN: +**Update BASE_DOMAIN with your actual URL:** ```bash # Extract domain from URL (removes https://) @@ -126,107 +316,171 @@ gcloud run services update extrasuite-server \ --project=$PROJECT_ID ``` -## Step 8: Configure Email Domain Allowlist (Optional) +### Using Google Cloud Console + +1. Go to **Cloud Run** ([direct link](https://console.cloud.google.com/run)) + +2. Click **Create Service** + +3. **Container image:** Enter: + ``` + asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:latest + ``` + +4. **Service name:** `extrasuite-server` + +5. **Region:** Select a region (e.g., `asia-southeast1`) + +6. **Authentication:** Select **Allow unauthenticated invocations** + +7. Expand **Container(s), Volumes, Networking, Security** + +8. Click the **Container** tab, then **Variables & Secrets** + +9. **Add environment variables:** + - Click **Add Variable** + - Name: `GOOGLE_CLOUD_PROJECT`, Value: Your project ID + - Click **Add Variable** + - Name: `BASE_DOMAIN`, Value: `placeholder.run.app` (you'll update this after deployment) + +10. **Add secrets:** + - Click **Reference a Secret** + - Secret: `extrasuite-client-id`, Referenced as: Environment variable, Name: `GOOGLE_CLIENT_ID` + - Click **Reference a Secret** + - Secret: `extrasuite-client-secret`, Referenced as: Environment variable, Name: `GOOGLE_CLIENT_SECRET` + - Click **Reference a Secret** + - Secret: `extrasuite-secret-key`, Referenced as: Environment variable, Name: `SECRET_KEY` + +11. Click the **Security** tab + - **Service account:** Select `extrasuite-server@YOUR_PROJECT_ID.iam.gserviceaccount.com` + +12. Click **Create** + +13. After deployment completes, copy the URL shown (e.g., `https://extrasuite-server-xxxxx-as.a.run.app`) + +14. **Update BASE_DOMAIN:** + - Click **Edit & Deploy New Revision** + - Go to **Variables & Secrets** + - Update `BASE_DOMAIN` to your actual domain (without `https://`) + - Click **Deploy** + +--- + +## Step 8: Update OAuth Redirect URI + +Update your OAuth credentials with the actual Cloud Run URL. + +### Using Google Cloud Console -Restrict authentication to specific email domains: +1. Go to **APIs & Services > Credentials** ([direct link](https://console.cloud.google.com/apis/credentials)) + +2. Click your OAuth client ID (`ExtraSuite Server`) + +3. Under **Authorized redirect URIs:** + - Remove the placeholder URI + - Add your actual URI: `https://YOUR_CLOUD_RUN_URL/api/auth/callback` + +4. Click **Save** + +--- + +## Verify Your Deployment + +Test that everything is working: + +### Health Check ```bash -gcloud run services update extrasuite-server \ - --region=asia-southeast1 \ - --update-env-vars="ALLOWED_EMAIL_DOMAINS=example.com,company.org" \ - --project=$PROJECT_ID +curl https://YOUR_CLOUD_RUN_URL/api/health +``` + +Expected response: +```json +{"status":"healthy","service":"extrasuite-server"} ``` -## Step 9: Configure Domain Abbreviations (Optional) +### Login Test + +1. Open your Cloud Run URL in a browser +2. Click **Login with Google** +3. Complete the OAuth flow +4. You should see your service account email and the installation command + +**Congratulations!** ExtraSuite is now deployed. Share your Cloud Run URL with users in your organization. + +--- + +## Optional: Restrict Access by Email Domain + +Limit who can authenticate to specific email domains: -Service accounts are named using the user's email local part plus a domain abbreviation: +### Using gcloud CLI ```bash gcloud run services update extrasuite-server \ --region=asia-southeast1 \ - --update-env-vars='DOMAIN_ABBREVIATIONS={"example.com":"ex","company.org":"co"}' \ + --update-env-vars="ALLOWED_EMAIL_DOMAINS=yourcompany.com,partner.org" \ --project=$PROJECT_ID ``` -Example: `john@example.com` creates service account `john-ex@project.iam.gserviceaccount.com` +### Using Google Cloud Console -If a domain is not in the mapping, a 4-character hash is used as fallback. +1. Go to **Cloud Run > extrasuite-server > Edit & Deploy New Revision** +2. Under **Variables & Secrets**, add or update: + - Name: `ALLOWED_EMAIL_DOMAINS` + - Value: `yourcompany.com,partner.org` (comma-separated, no spaces) +3. Click **Deploy** -## Verification +--- -Test the deployment: +## Optional: Custom Domain -```bash -# Health check -curl $SERVICE_URL/api/health -# Expected: {"status":"healthy","service":"extrasuite-server"} -``` +Use your own domain instead of the auto-generated Cloud Run URL. -## Custom Domain - -To use a custom domain: +### Using gcloud CLI ```bash -# Create domain mapping gcloud beta run domain-mappings create \ --service=extrasuite-server \ --domain=extrasuite.yourdomain.com \ --region=asia-southeast1 \ --project=$PROJECT_ID - -# Configure DNS: Add CNAME record -# extrasuite.yourdomain.com -> ghs.googlehosted.com - -# Check status (SSL cert takes 15-30 minutes) -gcloud beta run domain-mappings describe \ - --domain=extrasuite.yourdomain.com \ - --region=asia-southeast1 \ - --project=$PROJECT_ID ``` -After the domain is configured, update the BASE_DOMAIN: +**Configure DNS:** Add a CNAME record pointing `extrasuite.yourdomain.com` to `ghs.googlehosted.com` + +**Wait for SSL certificate:** This can take 15-30 minutes. Check status: ```bash -gcloud run services update extrasuite-server \ +gcloud beta run domain-mappings describe \ + --domain=extrasuite.yourdomain.com \ --region=asia-southeast1 \ - --update-env-vars="BASE_DOMAIN=extrasuite.yourdomain.com" \ --project=$PROJECT_ID ``` -## Using a Specific Version +**Update BASE_DOMAIN and OAuth redirect URI** with your custom domain after SSL is provisioned. -Instead of `latest`, you can pin to a specific version: +### Using Google Cloud Console -```bash -# Use a specific release ---image=asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:v1.0.0 +1. Go to **Cloud Run > extrasuite-server** +2. Click the **Integrations** tab +3. Click **Add Integration > Custom domains** +4. Follow the prompts to add your domain and configure DNS -# Use a specific commit ---image=asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:sha-abc1234 +--- -# Use latest from main branch ---image=asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:main -``` +## Environment Variables Reference -## Production Recommendations - -1. **Enable Cloud Armor** for DDoS protection -2. **Set up Cloud Monitoring** alerts for errors -3. **Enable Cloud Audit Logs** for compliance -4. **Set minimum instances** to reduce cold starts: - ```bash - gcloud run services update extrasuite-server \ - --region=asia-southeast1 \ - --min-instances=1 \ - --project=$PROJECT_ID - ``` -5. **Configure Firestore TTL** for automatic cleanup of expired OAuth states: - ```bash - gcloud firestore fields ttls update expire_at \ - --collection-group=oauth_states \ - --enable-ttl \ - --project=$PROJECT_ID - ``` +| Variable | Required | Description | +|----------|----------|-------------| +| `GOOGLE_CLIENT_ID` | Yes | OAuth 2.0 Client ID | +| `GOOGLE_CLIENT_SECRET` | Yes | OAuth 2.0 Client Secret | +| `GOOGLE_CLOUD_PROJECT` | Yes | Your GCP project ID | +| `SECRET_KEY` | Yes | Random string for signing session tokens | +| `BASE_DOMAIN` | Yes | Your server's domain (without `https://`) | +| `ALLOWED_EMAIL_DOMAINS` | No | Comma-separated list of allowed email domains | +| `TOKEN_EXPIRY_MINUTES` | No | Access token lifetime (default: 60) | +| `SESSION_COOKIE_EXPIRY_MINUTES` | No | Session duration (default: 1440 = 24 hours) | --- @@ -238,4 +492,4 @@ You've completed the server deployment (Step 1). Return to the Organization Setu --- -**Reference:** Review [IAM Permissions](iam-permissions.md) for a complete permission reference. \ No newline at end of file +**Reference:** Review the [Operations Guide](operations.md) for monitoring and troubleshooting. \ No newline at end of file diff --git a/website/docs/deployment/iam-permissions.md b/website/docs/deployment/iam-permissions.md index 256acc61..97dc32fc 100644 --- a/website/docs/deployment/iam-permissions.md +++ b/website/docs/deployment/iam-permissions.md @@ -1,156 +1,158 @@ # IAM Permissions Reference -This document lists all IAM permissions required by the ExtraSuite server. +This reference document explains the IAM roles and permissions required by ExtraSuite. -## Cloud Run Service Account +## Summary -The service account running the ExtraSuite server needs the following roles: +ExtraSuite requires **one service account** (`extrasuite-server`) with **four roles**: -### Firestore Access +| Role | Purpose | +|------|---------| +| `roles/datastore.user` | Read/write user records in Firestore | +| `roles/iam.serviceAccountAdmin` | Create service accounts for each user | +| `roles/iam.serviceAccountTokenCreator` | Generate short-lived access tokens | +| `roles/secretmanager.secretAccessor` | Read OAuth credentials and secret key | -**Role:** `roles/datastore.user` +## Role Details -**Purpose:** Read and write session data and user records in Firestore. +### Cloud Datastore User (`roles/datastore.user`) -```bash -gcloud projects add-iam-policy-binding $PROJECT_ID \ - --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/datastore.user" -``` +**What it does:** Allows ExtraSuite to store and retrieve user records and session data in Firestore. -### Service Account Administration +**Scope:** Project-level binding -**Role:** `roles/iam.serviceAccountAdmin` +**Why needed:** When a user logs in, ExtraSuite stores their email and service account email in Firestore. This data persists across sessions. -**Purpose:** Create service accounts for users during first authentication. +--- -```bash -gcloud projects add-iam-policy-binding $PROJECT_ID \ - --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/iam.serviceAccountAdmin" -``` +### Service Account Admin (`roles/iam.serviceAccountAdmin`) -### Token Creation (Impersonation) +**What it does:** Allows ExtraSuite to create, update, and delete service accounts. -**Role:** `roles/iam.serviceAccountTokenCreator` +**Scope:** Project-level binding -**Purpose:** Generate short-lived access tokens by impersonating user service accounts. +**Why needed:** ExtraSuite creates a dedicated service account for each user on first login (e.g., `john-ex@project.iam.gserviceaccount.com`). This role permits creating these accounts. -```bash -gcloud projects add-iam-policy-binding $PROJECT_ID \ - --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/iam.serviceAccountTokenCreator" -``` +**Security note:** While this sounds like a powerful role, it is safe because **creating a service account is separate from granting it permissions**. -### Secret Manager Access +`serviceAccountAdmin` allows: -**Role:** `roles/secretmanager.secretAccessor` +- ✓ Creating service accounts +- ✓ Updating metadata (display name, description) +- ✓ Deleting service accounts +- ✗ Granting IAM roles to service accounts +- ✗ Modifying project IAM policies -**Purpose:** Read secrets for OAuth credentials and signing keys. +To grant any IAM role to a service account, you need `roles/resourcemanager.projectIamAdmin` or specific `setIamPolicy` permissions—which ExtraSuite does not have. -```bash -gcloud secrets add-iam-policy-binding $SECRET_NAME \ - --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/secretmanager.secretAccessor" -``` +**The service accounts ExtraSuite creates have zero IAM roles.** They cannot access any GCP resources (BigQuery, Cloud Storage, Compute, etc.). They can only access Google Workspace files (Sheets, Docs, Slides) that users explicitly share with them. -## End-User Permissions +--- -**Important:** End users do NOT need any GCP project access or IAM roles to use ExtraSuite. +### Service Account Token Creator (`roles/iam.serviceAccountTokenCreator`) -### How It Works +**What it does:** Allows ExtraSuite to generate short-lived access tokens by impersonating user service accounts. -1. **User authenticates** via Google OAuth (only `openid` and `userinfo.email` scopes) -2. **Server creates a service account** for the user using its own credentials -3. **Server impersonates the SA** to generate a short-lived token -4. **Token is returned** to the CLI for use with Google Workspace APIs +**Scope:** Project-level binding -The server acts as a trusted intermediary. Users only need to prove their identity - they don't need any GCP permissions. +**Why needed:** When the CLI requests a token, ExtraSuite impersonates the user's service account to create a time-limited OAuth token. This is the core functionality that makes ExtraSuite work. -### What End Users Need +**Security note:** Tokens are short-lived (default: 60 minutes) and scoped to Google Workspace APIs only. -| Requirement | Needed? | Notes | -|-------------|---------|-------| -| GCP Project membership | **No** | IAM bindings work with any Google account | -| GCP Console access | **No** | Users never interact with GCP directly | -| Any project-level IAM roles | **No** | Permissions are per-SA, granted automatically | -| Google Workspace account | **Yes** | For OAuth authentication | +--- -### Organization Rollout +### Secret Manager Secret Accessor (`roles/secretmanager.secretAccessor`) -To enable all employees to use ExtraSuite: +**What it does:** Allows ExtraSuite to read specific secrets. -1. **Configure domain allowlist** on the server: - ```bash - ALLOWED_EMAIL_DOMAINS=example.com - ``` +**Scope:** Secret-level binding (not project-level) -2. **Grant server permissions** (see sections above) +**Why needed:** ExtraSuite reads three secrets at startup: -3. **Share Google Workspace resources** with the created service accounts +- `extrasuite-client-id` - OAuth Client ID +- `extrasuite-client-secret` - OAuth Client Secret +- `extrasuite-secret-key` - Session signing key -That's it. No IAM configuration needed for individual users. +**Security note:** This role is granted only on specific secrets, not all secrets in the project. -## OAuth Scopes +--- -### Server OAuth Scopes +## End Users Do NOT Need GCP Access -The server uses Application Default Credentials with: +A common misconception is that end users need GCP permissions. They do not. -- `https://www.googleapis.com/auth/cloud-platform` - For IAM and Firestore operations +| Requirement | Needed? | +|-------------|---------| +| GCP project membership | No | +| GCP Console access | No | +| Any IAM roles | No | +| Google Workspace/Gmail account | Yes | -### User OAuth Scopes (Authentication) +**How it works:** -Users are prompted to grant minimal scopes for identity verification only: +1. User proves their identity via Google OAuth (only email scope) +2. ExtraSuite uses its own credentials to create the user's service account +3. ExtraSuite generates tokens using its own permissions +4. User receives tokens without ever touching GCP -- `openid` - OpenID Connect -- `https://www.googleapis.com/auth/userinfo.email` - Email address +--- -Users do NOT grant `cloud-platform` scope. The server uses its own credentials for all IAM operations. +## OAuth Scopes + +### User Authentication Scopes + +When users log in to ExtraSuite, they grant only: + +- `openid` - Standard OpenID Connect +- `userinfo.email` - Access to email address + +Users do NOT grant `cloud-platform` or any Google Workspace scopes during login. ### Service Account Token Scopes -The short-lived tokens include: +The short-lived tokens issued to AI agents include: + +- `https://www.googleapis.com/auth/drive.readonly` - Read Google Drive files +- `https://www.googleapis.com/auth/spreadsheets` - Read/write Google Sheets +- `https://www.googleapis.com/auth/documents` - Read/write Google Docs +- `https://www.googleapis.com/auth/presentations` - Read/write Google Slides -- `https://www.googleapis.com/auth/spreadsheets` - Google Sheets read/write -- `https://www.googleapis.com/auth/documents` - Google Docs read/write -- `https://www.googleapis.com/auth/presentations` - Google Slides read/write -- `https://www.googleapis.com/auth/drive.readonly` - Google Drive read access +--- -## Summary Table +## Least Privilege Considerations -### Runtime Service Account (`extrasuite-server`) +The default setup grants project-level permissions for simplicity. For stricter security: -| Role | Resource | Purpose | -|------|----------|---------| -| `roles/datastore.user` | Project | Read/write Firestore | -| `roles/iam.serviceAccountAdmin` | Project | Create user SAs | -| `roles/iam.serviceAccountTokenCreator` | Project | Impersonate user SAs | -| `roles/secretmanager.secretAccessor` | Specific secrets | Read OAuth config | +### Option 1: Separate Projects -## Least Privilege Recommendations +Use one project for ExtraSuite infrastructure and another for user service accounts. This isolates user service accounts from the ExtraSuite server. -For production environments: +### Option 2: Conditional Token Creator -1. **Use separate projects** for ExtraSuite infrastructure and user service accounts if needed for compliance +Instead of project-level `serviceAccountTokenCreator`, grant it only on specific service accounts: -2. **Restrict token creator scope** to specific service accounts if possible: - ```bash - gcloud iam service-accounts add-iam-policy-binding \ - username-ex@$PROJECT_ID.iam.gserviceaccount.com \ - --member="serviceAccount:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" \ - --role="roles/iam.serviceAccountTokenCreator" - ``` +```bash +gcloud iam service-accounts add-iam-policy-binding \ + USERNAME-ABBR@PROJECT.iam.gserviceaccount.com \ + --member="serviceAccount:extrasuite-server@PROJECT.iam.gserviceaccount.com" \ + --role="roles/iam.serviceAccountTokenCreator" +``` -3. **Audit regularly** using Cloud Audit Logs to monitor: - - Service account creation events - - Token generation events - - Failed authentication attempts +This requires updating bindings each time a new user is onboarded. --- -## Continue Your Setup +## Audit Logging + +Enable Cloud Audit Logs to monitor ExtraSuite activity: -If you arrived here from the Organization Setup guide, return to continue with user onboarding: +- **Admin Activity logs** (always on): Service account creation +- **Data Access logs** (must enable): Token generation, Firestore reads -[:octicons-arrow-right-24: Continue Organization Setup](../getting-started/organization-setup.md#step-2-install-your-ai-editor) \ No newline at end of file +View logs in Cloud Logging: + +```bash +gcloud logging read 'protoPayload.serviceName="iam.googleapis.com"' \ + --project=$PROJECT_ID \ + --limit=20 +``` diff --git a/website/docs/deployment/index.md b/website/docs/deployment/index.md index 5aedbd62..a0e4800f 100644 --- a/website/docs/deployment/index.md +++ b/website/docs/deployment/index.md @@ -1,179 +1,73 @@ # Deployment Guide -ExtraSuite is designed to be deployed as a self-hosted service on Google Cloud Platform. This guide covers deploying your own instance. +ExtraSuite is designed to be deployed as a self-hosted service on Google Cloud Platform. This guide walks you through deploying your own instance. -## Quick Start +## What You're Building -```bash -# Set your project ID -export PROJECT_ID=your-project-id +When you complete this guide, you'll have: -# Deploy using the pre-built image -gcloud run deploy extrasuite-server \ - --image=asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:latest \ - --region=asia-southeast1 \ - --allow-unauthenticated \ - --project=$PROJECT_ID -``` - -See [Cloud Run Deployment](cloud-run.md) for complete setup instructions including OAuth configuration. - -## Architecture Overview +- A Cloud Run service that authenticates users and issues short-lived access tokens +- A Firestore database to store user records +- OAuth credentials so users can log in with their Google accounts +- A service account with permissions to create user-specific service accounts ``` ┌─────────────┐ ┌─────────────────┐ ┌──────────────────┐ -│ CLI Tool │────▶│ ExtraSuite │────▶│ Google Cloud │ +│ AI Agent │────▶│ ExtraSuite │────▶│ Google Cloud │ │ (Claude, │ │ Server │ │ - Firestore │ │ Codex) │◀────│ (Cloud Run) │◀────│ - IAM │ └─────────────┘ └─────────────────┘ │ - OAuth │ └──────────────────┘ ``` -## Prerequisites - -### Google Cloud Platform - -1. **GCP Project** with billing enabled -2. **gcloud CLI** installed and configured -3. **Required APIs**: - - Cloud Run - - Firestore - - IAM - - IAM Credentials - - Secret Manager - -### OAuth Credentials - -1. **Google OAuth Client** configured in Cloud Console -2. **OAuth consent screen** set up (internal or external) - -### Domain (Optional) - -- Custom domain with DNS access for production deployments - -## Docker Images - -Pre-built Docker images are available from Google Artifact Registry: - -``` -asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server -``` - -**Available tags:** - -| Tag | Description | -|-----|-------------| -| `latest` | Latest stable release | -| `v1.0.0` | Specific version | -| `main` | Latest from main branch | -| `sha-abc1234` | Specific commit | - -## Environment Variables - -### Required - -| Variable | Description | -|----------|-------------| -| `GOOGLE_CLIENT_ID` | OAuth 2.0 Client ID | -| `GOOGLE_CLIENT_SECRET` | OAuth 2.0 Client Secret | -| `GOOGLE_CLOUD_PROJECT` | GCP Project ID for service accounts and Firestore | -| `SECRET_KEY` | Session signing key (use a long random string) | -| `BASE_DOMAIN` | Domain of your server (e.g., `extrasuite.example.com`) | +## Prerequisites Checklist -### Optional +Before you begin, ensure you have: -| Variable | Description | Default | -|----------|-------------|---------| -| `SERVER_URL` | Full base URL for server | Derived from `BASE_DOMAIN` as `https://{BASE_DOMAIN}` | -| `FIRESTORE_DATABASE` | Firestore database name | `(default)` | -| `ALLOWED_EMAIL_DOMAINS` | Comma-separated allowed domains | All domains | -| `DOMAIN_ABBREVIATIONS` | JSON mapping for SA naming | Hash-based | +| Requirement | Details | +|-------------|---------| +| Google Cloud Project | A project where you have **Owner** or **Editor** role, with billing enabled | +| Google Workspace or Gmail | To test authentication after deployment | -### Session Cookie Settings +**Optional but recommended:** -| Variable | Description | Default | -|----------|-------------|---------| -| `SESSION_COOKIE_NAME` | Cookie name | `session` | -| `SESSION_COOKIE_EXPIRY_MINUTES` | Session duration | `1440` (24 hours) | -| `SESSION_COOKIE_SAME_SITE` | SameSite policy | `lax` | -| `SESSION_COOKIE_HTTPS_ONLY` | HTTPS-only cookies | `true` | -| `SESSION_COOKIE_DOMAIN` | Cookie domain | Value of `BASE_DOMAIN` | +| Requirement | Details | +|-------------|---------| +| gcloud CLI | Makes deployment faster. [Install gcloud CLI](https://cloud.google.com/sdk/docs/install) | +| Custom domain | For a professional URL instead of the auto-generated Cloud Run URL | -### Token Settings +## Deployment Steps Overview -| Variable | Description | Default | -|----------|-------------|---------| -| `TOKEN_EXPIRY_MINUTES` | Access token lifetime | `60` (1 hour) | +The deployment process has 8 steps: -## Security Considerations +| Step | What You'll Do | Time | +|------|----------------|------| +| 1 | Enable Google Cloud APIs | 2 min | +| 2 | Create a Firestore database | 2 min | +| 3 | Configure OAuth consent screen | 5 min | +| 4 | Create OAuth credentials | 3 min | +| 5 | Create a service account for ExtraSuite | 3 min | +| 6 | Store secrets securely | 3 min | +| 7 | Deploy to Cloud Run | 5 min | +| 8 | Verify the deployment | 2 min | -### IAM Permissions +**[Start the Deployment Guide →](cloud-run.md)** -The ExtraSuite server requires specific IAM roles. See [IAM Permissions](iam-permissions.md) for details. +## Docker Image -**Key principle:** End users do NOT need any GCP project access. The server acts as a trusted intermediary. +ExtraSuite publishes official Docker images to Google Artifact Registry: -### Secret Management - -- Store OAuth credentials in Secret Manager -- Generate strong random keys for session signing -- Rotate secrets periodically - -### Network Security - -- Always use HTTPS in production -- Consider Cloud Armor for DDoS protection -- Enable VPC Service Controls for sensitive environments - -## Deployment Guides - -- **[Cloud Run Deployment](cloud-run.md)** - Step-by-step deployment guide -- **[IAM Permissions](iam-permissions.md)** - Complete IAM role reference -- **[Operations](operations.md)** - Troubleshooting and common issues - -## Local Development - -For development and testing: - -```bash -cd extrasuite-server -cp .env.template .env -# Edit .env with your configuration - -uv sync -uv run uvicorn extrasuite_server.main:app --reload --port 8001 ``` - -Set `SERVER_URL=http://localhost:8001` for local development. - -## Building from Source - -If you prefer to build your own image: - -```bash -git clone https://github.com/think41/extrasuite.git -cd extrasuite - -# Build the image -docker build -t my-extrasuite-server:latest . - -# Push to your registry -docker tag my-extrasuite-server:latest gcr.io/$PROJECT_ID/extrasuite-server:latest -docker push gcr.io/$PROJECT_ID/extrasuite-server:latest +asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server ``` -## Support - -For deployment issues: - -1. Check the [Operations guide](operations.md) for common issues -2. Review Cloud Run logs for errors -3. Open an issue on [GitHub](https://github.com/think41/extrasuite/issues) - ---- - -## Continue Your Setup +| Tag | When to Use | +|-----|-------------| +| `latest` | Production deployments (latest stable release) | +| `v1.0.0` | Pin to a specific version | +| `main` | Testing latest changes (may be unstable) | -If you arrived here from the Organization Setup guide, return to continue with user onboarding: +## Additional Resources -[:octicons-arrow-right-24: Continue Organization Setup](../getting-started/organization-setup.md#step-2-install-your-ai-editor) +- **[IAM Permissions Reference](iam-permissions.md)** - Detailed explanation of required permissions +- **[Operations Guide](operations.md)** - Monitoring, troubleshooting, and maintenance diff --git a/website/docs/deployment/operations.md b/website/docs/deployment/operations.md index d0e92c48..ca3abff6 100644 --- a/website/docs/deployment/operations.md +++ b/website/docs/deployment/operations.md @@ -1,98 +1,152 @@ # Operations Guide -This document covers monitoring, debugging, and troubleshooting for ExtraSuite deployments. +This guide covers monitoring, updating, and troubleshooting your ExtraSuite deployment. ## Monitoring ### View Logs +#### Using gcloud CLI + ```bash -# Recent logs +# Recent logs (last 50 entries) gcloud run services logs read extrasuite-server \ --region=asia-southeast1 \ --project=$PROJECT_ID \ --limit=50 -# Error logs only +# Errors only gcloud logging read \ 'resource.type="cloud_run_revision" AND resource.labels.service_name="extrasuite-server" AND severity>=ERROR' \ --project=$PROJECT_ID \ - --limit=20 \ - --format="json" + --limit=20 ``` -### Health Checks +#### Using Google Cloud Console + +1. Go to **Cloud Run > extrasuite-server** +2. Click the **Logs** tab + +### Health Check ```bash -# Basic health check -curl https://your-domain.com/api/health -# Expected: {"status":"healthy","service":"extrasuite-server"} +curl https://YOUR_DOMAIN/api/health +``` + +Expected response: +```json +{"status":"healthy","service":"extrasuite-server"} ``` ### List User Service Accounts +See how many users have been onboarded: + ```bash gcloud iam service-accounts list --project=$PROJECT_ID \ --format="table(email,displayName)" ``` -### Check IAM Permissions +--- + +## Updating ExtraSuite + +### Update to Latest Version + +#### Using gcloud CLI ```bash -gcloud projects get-iam-policy $PROJECT_ID \ - --flatten="bindings[].members" \ - --format="table(bindings.role,bindings.members)" \ - --filter="bindings.members:extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com" +gcloud run services update extrasuite-server \ + --region=asia-southeast1 \ + --image=asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:latest \ + --project=$PROJECT_ID ``` -## Common Issues +#### Using Google Cloud Console -### OAuth Scopes Showing Too Many Permissions +1. Go to **Cloud Run > extrasuite-server** +2. Click **Edit & Deploy New Revision** +3. Update the container image URL to use `latest` or a specific version +4. Click **Deploy** -**Symptom:** OAuth consent screen asks for "See, edit, configure and delete your Google Cloud data" instead of just email access. +### Pin to a Specific Version -**Cause:** Using `include_granted_scopes="true"` causes Google to include any previously granted scopes. +For production stability, pin to a version tag: -**Solution:** Users may need to revoke existing app permissions at [myaccount.google.com/permissions](https://myaccount.google.com/permissions) before re-authenticating. +```bash +gcloud run services update extrasuite-server \ + --region=asia-southeast1 \ + --image=asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:v1.0.0 \ + --project=$PROJECT_ID +``` -### Firestore "No document to update" Error +### Rollback to Previous Version -**Symptom:** +```bash +# List available revisions +gcloud run revisions list \ + --service=extrasuite-server \ + --region=asia-southeast1 \ + --project=$PROJECT_ID + +# Route all traffic to a previous revision +gcloud run services update-traffic extrasuite-server \ + --region=asia-southeast1 \ + --to-revisions=extrasuite-server-00001-abc=100 \ + --project=$PROJECT_ID +``` + +--- + +## Troubleshooting + +### OAuth Consent Shows Too Many Permissions + +**Symptom:** Users see "See, edit, configure and delete your Google Cloud data" instead of just email access. + +**Cause:** Google includes previously granted scopes when `include_granted_scopes` is enabled. + +**Solution:** Users should revoke ExtraSuite's access at [myaccount.google.com/permissions](https://myaccount.google.com/permissions), then log in again. + +--- + +### 404 Error: "No document to update" + +**Symptom:** Error in logs: ``` NotFound: 404 No document to update: projects/.../documents/users/email@domain.com ``` -**Cause:** Code is using Firestore `update()` method on a document that doesn't exist. +**Cause:** This is a code bug where `update()` is used instead of `set(merge=True)`. -**Solution:** Use `set(merge=True)` instead of `update()` for upsert behavior. +**Solution:** Update to the latest version of ExtraSuite which fixes this issue. + +--- ### Email Domain Allowlist Not Working -**Symptom:** Valid email domains are rejected. +**Symptom:** Valid email domains are rejected during login. -**Cause:** The gcloud CLI may interpret commas incorrectly in some shells. +**Cause:** The `ALLOWED_EMAIL_DOMAINS` environment variable may be malformed. -**Solution:** Verify the environment variable is set correctly: +**Diagnosis:** ```bash gcloud run services describe extrasuite-server \ --region=asia-southeast1 \ - --format="value(spec.template.spec.containers[0].env)" + --project=$PROJECT_ID \ + --format="yaml(spec.template.spec.containers[0].env)" ``` -### Firestore Database Recreation Delay - -**Symptom:** After deleting a Firestore database, recreating fails: +**Solution:** Ensure domains are comma-separated with no spaces: ``` -FAILED_PRECONDITION: Database ID '(default)' is not available. Please retry in X seconds. +ALLOWED_EMAIL_DOMAINS=example.com,company.org ``` -**Cause:** Firestore requires a cooldown period (~4-5 minutes) after database deletion. - -**Solution:** Wait for the cooldown period before recreating. +--- -### Custom Domain SSL Certificate Pending +### Custom Domain SSL Certificate Not Working -**Symptom:** Custom domain doesn't work with HTTPS. +**Symptom:** HTTPS doesn't work on your custom domain. **Cause:** Google-managed SSL certificates take 15-30 minutes to provision. @@ -101,14 +155,17 @@ FAILED_PRECONDITION: Database ID '(default)' is not available. Please retry in X gcloud beta run domain-mappings describe \ --domain=your-domain.com \ --region=asia-southeast1 \ + --project=$PROJECT_ID \ --format="yaml(status.conditions)" -# Verify DNS +# Verify DNS is configured dig your-domain.com CNAME +short -# Should return: ghs.googlehosted.com. +# Expected: ghs.googlehosted.com. ``` -**Solution:** Wait for certificate provisioning. When `CertificateProvisioned` shows `status: 'True'`, HTTPS will work. +**Solution:** Wait for certificate provisioning. Once `CertificateProvisioned` shows `status: 'True'`, HTTPS will work. + +--- ### IAM Policy Binding Fails @@ -127,103 +184,106 @@ gcloud projects add-iam-policy-binding $PROJECT_ID \ --condition=None ``` -## Updating the Deployment +--- -### Update to Latest Version +### Firestore Database Recreation Fails -```bash -gcloud run services update extrasuite-server \ - --region=asia-southeast1 \ - --image=asia-southeast1-docker.pkg.dev/thinker41/extrasuite/server:latest \ - --project=$PROJECT_ID +**Symptom:** After deleting a Firestore database, recreating fails with: +``` +FAILED_PRECONDITION: Database ID '(default)' is not available. Please retry in X seconds. ``` -### Update Environment Variables +**Cause:** Firestore requires a cooldown period (~5 minutes) after database deletion. -```bash -gcloud run services update extrasuite-server \ - --region=asia-southeast1 \ - --update-env-vars="KEY=value" \ - --project=$PROJECT_ID -``` +**Solution:** Wait 5 minutes, then try again. -### Rollback to Previous Version +--- -```bash -# List revisions -gcloud run revisions list \ - --service=extrasuite-server \ - --region=asia-southeast1 \ - --project=$PROJECT_ID +### Service Account Quota Exceeded -# Route traffic to a specific revision -gcloud run services update-traffic extrasuite-server \ - --region=asia-southeast1 \ - --to-revisions=extrasuite-server-00001-abc=100 \ - --project=$PROJECT_ID -``` +**Symptom:** New users cannot onboard. -## Testing Authentication Flow +**Cause:** GCP default quota is 100 service accounts per project. +**Diagnosis:** ```bash -# Clear token cache -rm -f ~/.config/extrasuite/token.json - -# Run test -cd /path/to/extrasuite -PYTHONPATH=extrasuite-client/src python3 extrasuite-client/examples/basic_usage.py \ - --server https://your-domain.com +gcloud iam service-accounts list --project=$PROJECT_ID | wc -l ``` -**Three test scenarios:** +**Solution:** [Request a quota increase](https://console.cloud.google.com/iam-admin/quotas) for "Service Accounts per Project". -1. **First run (no cache):** Token file missing, browser opens, user authenticates -2. **Cached token:** Token file present and valid, no browser needed -3. **Session reuse:** Delete token cache, browser opens, SSO/session skips login prompt +--- ## Secrets Management -### Rotate Secrets +### Rotate a Secret ```bash -# Update secret with new value -echo -n "new-value" | gcloud secrets versions add extrasuite-secret-key --data-file=- +# Add new version +echo -n "new-secret-value" | gcloud secrets versions add extrasuite-secret-key \ + --data-file=- \ + --project=$PROJECT_ID -# Deploy with new version (Cloud Run auto-updates on next deploy) +# Redeploy to pick up new version gcloud run services update extrasuite-server \ --region=asia-southeast1 \ --update-secrets="SECRET_KEY=extrasuite-secret-key:latest" \ --project=$PROJECT_ID ``` -## Service Account Quota +### View Secret Versions + +```bash +gcloud secrets versions list extrasuite-secret-key --project=$PROJECT_ID +``` + +--- + +## Testing the Authentication Flow -ExtraSuite creates one service account per user. Monitor usage: +Use the included test script to verify authentication works: ```bash -# Count service accounts -gcloud iam service-accounts list --project=$PROJECT_ID | wc -l +# Clear any cached token +rm -f ~/.config/extrasuite/token.json + +# Run the test +cd /path/to/extrasuite +PYTHONPATH=extrasuite-client/src python3 extrasuite-client/examples/basic_usage.py \ + --server https://YOUR_DOMAIN ``` -GCP default quota is 100 service accounts per project. [Request an increase](https://console.cloud.google.com/iam-admin/quotas) if needed. +**Expected behavior:** -## Clean Up +1. Browser opens to the login page +2. User authenticates with Google +3. Browser redirects back and closes +4. Script prints a valid access token -To remove all ExtraSuite resources: +--- + +## Cleanup + +To completely remove ExtraSuite from your project: ```bash # Delete Cloud Run service gcloud run services delete extrasuite-server \ --region=asia-southeast1 \ - --project=$PROJECT_ID + --project=$PROJECT_ID \ + --quiet -# Delete user service accounts (adjust pattern for your domains) +# Delete user service accounts (adjust pattern for your domain abbreviations) gcloud iam service-accounts list --project=$PROJECT_ID --format="value(email)" | \ grep -E '-(ex|co)@' | \ xargs -I {} gcloud iam service-accounts delete {} --project=$PROJECT_ID --quiet +# Delete the server service account +gcloud iam service-accounts delete extrasuite-server@$PROJECT_ID.iam.gserviceaccount.com \ + --project=$PROJECT_ID --quiet + # Delete Firestore database -gcloud firestore databases delete --database="(default)" --project=$PROJECT_ID +gcloud firestore databases delete --database="(default)" --project=$PROJECT_ID --quiet # Delete secrets for secret in extrasuite-client-id extrasuite-client-secret extrasuite-secret-key; do @@ -233,8 +293,13 @@ done --- -## Continue Your Setup +## Getting Help -If you arrived here from the Organization Setup guide, return to continue with user onboarding: +If you encounter issues not covered here: -[:octicons-arrow-right-24: Continue Organization Setup](../getting-started/organization-setup.md#step-2-install-your-ai-editor) +1. Check [Cloud Run logs](#view-logs) for error details +2. Search [GitHub Issues](https://github.com/think41/extrasuite/issues) +3. Open a new issue with: + - Error message from logs + - Steps to reproduce + - Your configuration (without secrets) diff --git a/website/docs/index.html b/website/docs/index.html index d75334ae..3962330c 100644 --- a/website/docs/index.html +++ b/website/docs/index.html @@ -898,94 +898,121 @@

- -
+ +
-
-

- How It Works +
+

+ Setup in 3 Steps

-

- Set up once, use everywhere +

+ A one-time setup that unlocks unlimited productivity gains

- +
- - - -
-
-
- 0 -
-

- Install an AI Agent -

-

- Get Claude Code, Gemini CLI, or Codex CLI on your machine. -

-
-
-
- 1 -
-

- Connect Your AI -

-

- Works with Claude, ChatGPT, Gemini, and other AI assistants. -

-
-
-
- 2 + + + + +
+ + +
+
+ 1 +
+
+

+ Install for Yourself or Your Team +

+

+ Deploy ExtraSuite with individual setup (5 min) or organization deployment. +

+
+ + + + Saves hours every week +
+
+ Start Setup + + + +
+
-

- Authenticate Once -

-

- Sign in with Google. Your AI gets a secure, time-limited token. -

-
-
-
- 3 + + + + +
+
+ 2 +
+
+

+ Install the Skill in Your AI Agent +

+

+ Run a single command to add Google Workspace capabilities. +

+
+ Claude Code + Cursor + Codex + Gemini +
+
+ View Guides + + + +
+
-

- Share Documents -

-

- Share specific files with your AI, just like sharing with a - colleague. -

-
-
-
- 4 + + + + +
+
+ 3 +
+
+

+ Share Documents with Your AI +

+

+ Share your Google Sheet, Doc, or Slide with your service account email. +

+
+
+ + Sheets +
+
+ + Docs +
+
+ + Slides +
+
+
+ Learn More + + + +
+
-

- Get Work Done -

-

- Describe what you need. Your AI reads, writes, and updates your - docs. -

-
+
+