Skip to content

feat(mining): Add Google Contacts sync as default-on advanced option in Gmail mining #2679

@baderdean

Description

@baderdean

Context

We want contact mining to include Google Contacts, not only Gmail emails, while keeping the current mining UX simple.

Decision

Implement a lightweight version of Approach 2:

  • Keep Gmail mining as the primary flow.
  • Add Sync Google Contacts under Advanced mining options.
  • Toggle is enabled by default for Google sources.
  • Google Contacts sync runs as a byproduct of the Gmail mining run.

Architecture

High-Level Flow

User clicks \"Start Mining\"
       │
       ▼
┌─────────────────────────────────┐
│  Existing Gmail mining starts   │
│  (TasksManager / fetch/extract) │
└─────────────────────────────────┘
       │
       ├──────────────────────────────┐
       │ (if Google source + toggle) │
       ▼                              ▼
┌─────────────────────┐    ┌─────────────────────────┐
│ Continue Gmail flow │    │ Fetch Google Contacts  │
│ (no change)         │    │ via People API         │
└─────────────────────┘    └─────────────────────────┘
                                    │
                                    ▼
                         ┌─────────────────────┐
                         │ Normalize & persist │
                         │ via contacts.create │
                         └─────────────────────┘
                                    │
                                    ▼
                         ┌─────────────────────┐
                         │ Dedupe + refine     │
                         │ (existing pipeline) │
                         └─────────────────────┘

Key Design Points

  1. No separate task manager - reuse existing TasksManager
  2. No new task type - lightweight internal path handled in fetch phase
  3. Same persistence path - contacts flow through contacts.create() so dedupe/refine works automatically
  4. Source metadata - set person.source = 'google-contacts:<email>' to track origin
  5. Failure isolation - wrap Google Contacts sync in try/catch; if it fails, mining succeeds with warning

Files to Change

Frontend

File Change
frontend/src/components/mining/stepper-panels/mine/MiningSettingsDialog.vue Add toggle for "Sync Google Contacts" (Google sources only, default ON)
frontend/src/types/mining.ts Add googleContactsSync?: boolean to mining task or extend mining options
frontend/src/stores/leadminer.ts Add googleContactsSync state, pass in mining start payload
frontend/src/utils/sources.ts (Optional) Update types if needed

Backend

File Change
backend/src/controllers/mining.controller.ts Pass googleContactsSync option to task creation; if true, trigger contacts sync after fetch starts
backend/src/services/tasks-manager/TasksManager.ts Add googleContactsSync to task details; trigger sync in startMining
backend/src/services/tasks-manager/types.ts Add googleContactsSync?: boolean to mining source details
backend/src/services/extractors/Extractor.ts Add google_contacts extractor type
backend/src/services/extractors/engines/ Create GoogleContacts.ts - fetches from Google People API, normalizes to contact format
backend/src/db/types.ts Add GOOGLE_CONTACTS mining type if needed
backend/src/workers/email-message/emailMessageHandlers.ts Add handler for Google Contacts data ingestion
backend/src/services/OAuth2/google.ts Ensure reusable token refresh for People API calls

Supabase Edge Functions

File Change
supabase/functions/fetch-mining-source/ May need to extend to support People API scope (already has contacts scope in mining.helpers.ts)

Implementation Order

  1. Backend extractor - Create Google Contacts fetcher/normalizer
  2. Backend integration - Wire into mining controller (trigger after fetch)
  3. Frontend toggle - Add advanced option (default ON)
  4. End-to-end test - Verify sync runs and contacts appear

Out of Scope

  • Separate standalone Google Contacts mining workflow
  • Large task-manager redesign
  • DB schema redesign
  • User-facing source selector for Google Contacts

Acceptance Criteria

  • Google source shows "Sync Google Contacts" in advanced options only
  • Toggle default is ON
  • Mining with toggle ON syncs Gmail + Google Contacts in one run
  • Mining with toggle OFF behaves exactly like current Gmail mining
  • Google Contacts sync failure does not fail the full mining run
  • Contacts remain deduplicated and visible through existing contacts table/view
  • Unit/integration tests cover ON/OFF behavior and failure isolation

Related Code Context

  • Current Google OAuth scopes in backend/src/controllers/mining.helpers.ts already include https://www.googleapis.com/auth/contacts
  • Existing Google People API code in backend/src/services/export/exports/google/contacts-api.ts for export - can reuse patterns
  • Mining sources stored in private.mining_sources with OAuth credentials

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions