updating param desc

[sebastianwebber]

Documentation Improvements (rules.yml)

Memory Parameters

• shared_buffers: Updated guidance to start with 25% of RAM and use pg_buffercache extension for optimal tuning based on workload analysis
• effective_cache_size: Clarified that it doesn't allocate memory, only informs planner. Added formula: RAM - shared_buffers
• work_mem: Added critical OOM warnings with worst-case calculation example (work_mem × operations × parallel_workers × connections). Emphasized per-session tuning over global settings
• maintenance_work_mem: Added PostgreSQL 17+ information about parallel CREATE INDEX improvements and version-specific optimizations

Write-Ahead Log (WAL) Parameters

• min_wal_size / max_wal_size: Enhanced with workload-specific recommendations and monitoring guidance
• checkpoint_completion_target: Added practical example showing impact of spreading checkpoints
• wal_buffers: Clarified auto-tuning behavior and when to set explicit values

Network Parameters

• listen_addresses: Added security considerations and Docker/cloud environment guidance
• max_connections: Added connection pooling recommendations (PgBouncer, pgpool-II) with trade-offs

Storage Parameters (PostgreSQL 18+)

• effective_io_concurrency: Clarified it only affects bitmap heap scans, added PostgreSQL 18 default change (1→16)
• maintenance_io_concurrency: Explained relationship with VACUUM, CREATE INDEX, ANALYZE operations
• random_page_cost: Documented 2025 debate on SSD values, explained sequential vs index scan trade-offs
• io_method: Detailed differences between worker, io_uring, and sync methods with platform-specific guidance
• io_workers: Added workload-specific sizing guidance (10-40% of CPU cores)
• io_combine_limit / io_max_combine_limit: Explained I/O batching benefits for sequential scans
• io_max_concurrency: Documented read-ahead formula and memory pressure considerations
• file_copy_method: Added recommendation for clone method on supported filesystems (Btrfs, XFS, APFS, ZFS)

Worker Parameters

• max_worker_processes: Explained worker pool concept and consumers (parallel query, replication, extensions)
• max_parallel_workers: Clarified system-wide limit and relationship with max_worker_processes
• max_parallel_workers_per_gather: Documented resource multiplication effect (N workers = N+1x resources)

Code Optimizations

Checkpoint Tuning (checkpoint.go)

WAL Buffers Optimization:

• DW (Data Warehouse): 64MB for write-heavy batch operations
• OLTP with large shared_buffers (>8GB): 32MB for high concurrent writes
• Other profiles: Auto-tuning (-1)

WAL Size Optimization per Profile:

• DW: min=4GB, max=16GB (write-heavy with batch jobs)
• OLTP: min=2GB, max=8GB (frequent transactions)
• Web: min=1GB, max=4GB (moderate writes)
• Mixed: min=2GB, max=6GB (balanced workload)
• Desktop: min=512MB, max=2GB (low activity)

Storage Tuning (storage.go)

Random Page Cost Optimization:

• DW profile with SSD/SAN: 1.8 (favors sequential scans for analytical queries touching >10% of rows)
• Other profiles with SSD/SAN: 1.1 (favors index scans)
• HDD: 4.0 (PostgreSQL default)

I/O Concurrency (by disk type):

• SSD: 200 (effective_io_concurrency and maintenance_io_concurrency)
• SAN: 300
• HDD: 2

Async I/O Tuning (aio.go)

I/O Workers (by profile and disk type):

• Base factors: Desktop 10%, Web 20%, Mixed 25%, OLTP 30%, DW 40%
• HDD adds +10% (higher latency benefits from more workers)
• Minimum 2 workers, capped at TotalCPU

I/O Max Combine Limit:

• DW: 128 blocks (1MB) for large sequential scans
• Other profiles: 16 blocks (128KB, default)

I/O Max Concurrency:

• DW: 256 (high read-ahead for analytics)
• OLTP: 128 (balanced concurrency)
• Other profiles: 64 (default)

Worker Configuration (worker.go)

Dynamic Scaling with CPU Cores:

• max_worker_processes: max(8, TotalCPU)
• max_parallel_workers: max(8, TotalCPU)
• max_parallel_workers_per_gather:
  • Desktop/Web/Mixed/OLTP: 2 (transactional workloads don't benefit from high parallelism)
  • DW: min(TotalCPU/2, max_parallel_workers) with minimum of 2 (analytical queries benefit from parallelism)

Testing

• Added comprehensive table-driven tests for:
  • WAL configuration (TestComputeWalSizes, TestComputeWalBuffers)
  • Storage configuration (Test_computeStorageRandomPageCost)
  • Async I/O configuration (Test_computeAIO with 10 test cases)
  • Worker configuration (TestNewWorkerCfg with 9 test cases covering various CPU counts and profiles)

References Added

All documentation updates include authoritative references from:

• PostgreSQL official documentation
• Cybertec PostgreSQL blog
• Crunchy Data blog
• pganalyze blog and tools
• Percona blog
• AWS, Neon, and other cloud provider resources
• 2025 performance tuning guides

Key Design Principles

1. Workload-Specific Optimization: Different configurations for OLTP vs DW workloads
2. Hardware-Aware Scaling: Parameters scale with CPU cores and disk types
3. Conservative Defaults: Maintain safe defaults while enabling performance for capable systems
4. Modern Best Practices: Incorporate PostgreSQL 17-18 improvements and 2025 recommendations
5. Resource Safety: Prevent memory exhaustion with appropriate limits and warnings