Skip to content

feat: implement database sharding strategy (#602)#704

Open
amankoli09 wants to merge 1 commit into
rinafcode:mainfrom
amankoli09:feature/602-database-sharding-strategy
Open

feat: implement database sharding strategy (#602)#704
amankoli09 wants to merge 1 commit into
rinafcode:mainfrom
amankoli09:feature/602-database-sharding-strategy

Conversation

@amankoli09
Copy link
Copy Markdown
Contributor

Closes #602

Overview

This PR introduces the horizontal database sharding subsystem for the TeachLink backend, It enables our application to scale horizontally by distributing PostgreSQL load across multiple independent nodes.

Key Features & Acceptance Criteria Met

  • ✅ Sharding Strategy Documented: Comprehensive architecture and runbook documentation added to docs/SHARDING_STRATEGY.md.
  • ✅ Router Implementation: ShardRouter supports 4 strategies:
    • Hash-Based: Consistent-hash ring (150 virtual nodes/shard) for dynamic key distribution.
    • Tenant-Based: Normalizes keys to guarantee multi-tenant SaaS data locality.
    • Range-Based: Explicit bucketing for numeric ranges.
    • Read-Replica: Read-only query distribution via weighted random selection.
  • ✅ Migration Strategy: ShardMigrationService performs batched, zero-downtime, idempotent cross-shard migrations (INSERT ... ON CONFLICT DO NOTHING), including a dry-run phase and rollback support.
  • ✅ Rebalancing Procedure: ShardRebalanceService monitors pg pool utilization across shards, detecting when thresholds (e.g. SHARD_REBALANCE_HIGH_WATERMARK=80%) are breached to automatically orchestrate migration plans.
  • ✅ Health Monitoring: ShardHealthService tracks query latencies, active connections, pool utilization, and error rates per shard.
  • ✅ Connection Management: ShardConnectionManager lazily initializes and caches TypeORM DataSource instances for memory-efficient resource sharing.

Changes Made

  • src/sharding/*: All core sharding logic, controllers, and services.
  • src/app.module.ts: Wired ShardingModule into core application providers.
  • src/config/env.validation.ts & .env.example: Added validation logic and examples for SHARD_COUNT and rebalance thresholds.
  • docs/SHARDING_STRATEGY.md: The complete operator runbook and architecture specification.

Verification

  • Wrote Jest unit tests for ShardRouter validating ring distribution, deterministic routing, and replica fallback mechanisms.
  • All pre-commit hooks (lint-staged, eslint, prettier) and tsc --noEmit checks passed successfully.

Closes #602

@drips-wave
Copy link
Copy Markdown

drips-wave Bot commented May 30, 2026

@amankoli09 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@RUKAYAT-CODER
Copy link
Copy Markdown
Contributor

kindly resolve conflict and fix workflow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Implement database sharding strategy

2 participants