The connector currently hardcodes MAX_CONCURRENT_BULK_OPS = 5 in bulk_job_manager.py and uses all 5 slots Shopify allows. Customers who have other systems submitting bulk queries to the same store have no way to reserve slots for those systems.
Proposed change:
Add a max_concurrent_bulk_ops field to the Endpoint Config -> Advanced section (1, 5) and pass it through to the connector.
The connector currently hardcodes
MAX_CONCURRENT_BULK_OPS = 5inbulk_job_manager.pyand uses all 5 slots Shopify allows. Customers who have other systems submitting bulk queries to the same store have no way to reserve slots for those systems.Proposed change:
Add a
max_concurrent_bulk_opsfield to the Endpoint Config -> Advanced section (1, 5) and pass it through to the connector.