Skip to content

Migrate to Private Docker Image & Add Federated Queries Test Suite#21

Open
mkottakota1 wants to merge 13 commits intomainfrom
Build_vertica_private
Open

Migrate to Private Docker Image & Add Federated Queries Test Suite#21
mkottakota1 wants to merge 13 commits intomainfrom
Build_vertica_private

Conversation

@mkottakota1
Copy link
Collaborator

Description

This PR adds comprehensive test coverage for the ODBCLoader's federated query capabilities while migrating the CI/CD pipeline from the public Vertica Docker image to a private image for enhanced security, control, and customization.

Changes

Test Suite

  • New test file: tests/federated_queries.sql - Contains four test scenarios demonstrating federated query functionality:

    1. Predicate Pushdown - Validates that WHERE conditions are pushed down to the external database
    2. Column Pruning - Verifies that only required columns are fetched from the external source
    3. Federated Joins - Tests joining external tables with Vertica-managed tables
    4. Complex Predicates - Ensures multiple WHERE conditions are properly optimized
  • Expected output file: tests/expected/federated_queries.out - Contains baseline expected results for test validation

  • Updated Makefile:

    • Modified test target to run federated queries test suite
    • Added test_debug target for easier troubleshooting
    • Enhanced test output with dynamic progress indicators and validation status

Infrastructure

  • Docker Image Migration:
    • Migrated from public vertica/vertica-ce image to private custom Docker image
    • Updated GitHub Actions workflow to use private image registry
    • Added Docker authentication for private image access
    • Improved build consistency and security posture

Testing

  • All tests pass successfully with live output display
  • Test validation compares actual results against expected output
  • ODBC driver compatibility ensured across different MySQL ODBC versions
  • Tests run against private Vertica image with enhanced configuration

Motivation

  1. Federated Queries: Essential feature for joining Vertica tables with external databases while optimizing data movement through predicate pushdown and column pruning
  2. Private Image: Provides better control over dependencies, security patches, and custom configurations for CI/CD pipeline stability

Related Issues

Closes #[issue-number]

Checklist

  • Federated queries tests added/updated
  • Expected output files created
  • Makefile updated with new test targets
  • All tests passing with private image
  • GitHub Actions workflow updated for private image registry
  • Docker authentication configured
  • CI/CD pipeline validated

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant