Skip to content

feat: retry option added in gcp upload #40

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
tr-emp-042 merged 2 commits into from
May 9, 2026
Merged

feat: retry option added in gcp upload #40

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7751a58
feat: retry option added in gcp upload
tr-emp-042 May 9, 2026
dc2d37b
Update cloudstorage.js
tr-emp-042 May 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "tracker-utils",
"version": "1.6.11",
"version": "1.6.12",
"description": "Tracker Utils",
"main": "index.js",
"scripts": {
Expand All @@ -17,7 +17,7 @@
},
"dependencies": {
"@google-cloud/pubsub": "^4.7.2",
"@google-cloud/storage": "^5.16.1",
"@google-cloud/storage": "^7.19.0",
"axios": "^1.7.7",
"bluebird": "^3.5.0",
"ejs": "^3.1.6",
Expand Down
9 changes: 8 additions & 1 deletion src/cloudstorage.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
// External and built-in deps
const { Storage } = require('@google-cloud/storage');
const gcpUploadRetryOpt = {
autoRetry: true,
maxRetries: 5,
retryDelayMultiplier: 2,
totalTimeout: 120000, // 120 seconds
maxRetryDelay: 30000 // 30 seconds
};
Comment on lines +3 to +9
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🌐 Web query:

@google-cloud/storage version 7 retryOptions configuration fields and structure

💡 Result:

In @google-cloud/storage (Node.js) v7, the retry configuration is provided via the Storage constructor option retryOptions (a plain object). Its fields include (with the structure shown below): 1) autoRetry (boolean) - Whether requests are automatically retried. If false, requests will not retry and the other parameters below will not affect retry behavior. [1] 2) retryDelayMultiplier (number) - Multiplier for increasing delay between completion of failed requests and initiation of subsequent retries. [1] 3) totalTimeout (number) - Total time (from initial request sent to a timeout). After timeout, an error is returned regardless of any retry attempts during that period. [1] 4) maxRetryDelay (number) - Maximum delay time between requests; when reached, retryDelayMultiplier is no longer used to increase delay time. [1] 5) maxRetries (number) - Maximum number of automatic retries attempted before returning the error. [1] 6) idempotencyStrategy (IdempotencyStrategy) - Strategy controlling idempotency-based retry behavior; the docs example uses IdempotencyStrategy.RetryAlways. [1] Example structure (as shown in Google’s Cloud Storage retry sample): const {Storage} = require('@google-cloud/storage'); const storage = new Storage({ retryOptions: { autoRetry: true, retryDelayMultiplier: 3, totalTimeout: 500, maxRetryDelay: 60, maxRetries: 5, idempotencyStrategy: IdempotencyStrategy.RetryAlways, }, }); [1] Additional context on where this config plugs in: - When you initialize Cloud Storage, a retryOptions config file is initialized as well; unless overridden, options in the config are set to default values. To modify default behavior, pass retryOptions into the storage constructor upon initialization. [2] IdempotencyStrategy values (used by idempotencyStrategy): - RetryAlways, RetryConditional, RetryNever. [3] Notes / scope - The fields above are the ones explicitly documented in Google’s v7-era Node.js Cloud Storage retry configuration sample and the related reference pages. [1][2]

Citations:

🏁 Script executed:

#!/bin/bash
# Search for existing retry configuration examples or documentation in the codebase

rg -n -C5 'retryOptions|Storage.*retry' --type=js

Repository: CloudStuffTech/tracker-utils

Length of output: 671

Correct units in retry configuration: use milliseconds, not seconds.

Field names are correct for v7, but the code has a critical unit error:

  • totalTimeout: 300 with comment "seconds" will actually timeout after 300 milliseconds (0.3 seconds), not 5 minutes as likely intended. Should be totalTimeout: 300000 for 300 seconds.
  • maxRetryDelay: 60 with comment "seconds" will cap retries at 60 milliseconds, not 60 seconds. Should be maxRetryDelay: 60000 for 60 seconds.

Additionally, the v7 API example includes idempotencyStrategy: IdempotencyStrategy.RetryAlways in the retryOptions, which should be added here. This requires importing IdempotencyStrategy from @google-cloud/storage.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/cloudstorage.js` around lines 3 - 9, The retry configuration object
gcpUploadRetryOpt uses seconds but the v7 client expects milliseconds: update
totalTimeout from 300 to 300000 and maxRetryDelay from 60 to 60000 (and adjust
any comments accordingly), and add the v7 idempotency setting by importing
IdempotencyStrategy from '@google-cloud/storage' and adding idempotencyStrategy:
IdempotencyStrategy.RetryAlways into the retry options (i.e., inside
gcpUploadRetryOpt or the retryOptions object you pass to the storage
client/upload).


/**
* Note: Environment should have GOOGLE_APPLICATION_CREDENTIALS set up. This has the path of service
Expand All @@ -21,7 +28,7 @@ class CloudStorage {
this.type = type;
this.baseUrl = baseUrl;
// TODO: Update this in future when aws will be used.
this.storage = new Storage();
this.storage = new Storage({retryOptions: gcpUploadRetryOpt});
}

/**
Expand Down
Loading