Conversation
Align bucket prefixes with audio/visual layout to reduce listing scope.
|
|
||
| if (data.nextPageToken) { | ||
| return getMediaAssets(bucketName, whitelist, taskName, language, data.nextPageToken, categorizedObjects); | ||
| return getMediaAssets(bucketName, whitelist, language, taskName, data.nextPageToken, categorizedObjects); |
There was a problem hiding this comment.
Looks like the arguments are flipped here. @digital-pro did you run the tasks locally to check if this PR works?
| return `${bucket}/${taskName}`; | ||
| } | ||
|
|
||
| return `${bucket}/${taskName}`; |
There was a problem hiding this comment.
I think this:
if (assetType === 'visual') {
return `${bucket}/${taskName}`;
}
is unnecessary since the default return is the same as well.
|
|
||
| return `${bucket}/${assetType === 'audio' ? language : taskName}`; | ||
| if (assetType === 'audio') { | ||
| return language ? `${bucket}/${language}` : bucket; |
There was a problem hiding this comment.
So previously if the language was not present it was adding an extra / and that was causing issues?
|
Sorry, I posted to the -engineering channel when I found the Sentry report over the weekend that this was mostly a placeholder and that someone who knows the code better should figure out if this is the right approach to fixing it. I didn't feel competent to know, but wanted to flag that the issue wasn't the initially blamed large asset, but large object name retrievals. |
|
@zwatson2001 do you think we can look into this? |
|
It was asking for a list of all assets in the bucket without filtering by language, etc. So the returned list was getting flagged as it is over 800K (just for the list), and doing it multiple times. We talked about it at standup and Zach thinks he can sort it out and presumably run the fix by you.
From: asengupta3 ***@***.***>
Sent: Monday, February 2, 2026 9:11 AM
To: levante-framework/core-tasks ***@***.***>
Cc: David Cardinal ***@***.***>; Mention ***@***.***>
Subject: Re: [levante-framework/core-tasks] Optimize media asset bucket listing (PR #415)
@asengupta3 commented on this pull request.
________________________________
In task-launcher/src/tasks/shared/helpers/getBucketName.ts<https://urldefense.com/v3/__https:/github.com/levante-framework/core-tasks/pull/415*discussion_r2755390806__;Iw!!G92We9drHetJ8EofZw!dHsdAlLSGai08CSFWJUxgSGA3kJDc441YRnqK7qLvXYgJpztx5G5T75ekbxSV3JMQNjuM1tT7yJYrqajX6QXQn3Av9Wi$>:
@@ -10,5 +10,13 @@ export function getBucketName(
? TASK_BUCKET_NAMES_DEV[assetType as keyof typeof TASK_BUCKET_NAMES_DEV]
: TASK_BUCKET_NAMES_PROD[assetType as keyof typeof TASK_BUCKET_NAMES_PROD];
- return `${bucket}/${assetType === 'audio' ? language : taskName}`;
+ if (assetType === 'audio') {
+ return language ? `${bucket}/${language}` : bucket;
So previously if the language was not present it was adding an extra / and that was causing issues?
—
Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/levante-framework/core-tasks/pull/415*pullrequestreview-3740815292__;Iw!!G92We9drHetJ8EofZw!dHsdAlLSGai08CSFWJUxgSGA3kJDc441YRnqK7qLvXYgJpztx5G5T75ekbxSV3JMQNjuM1tT7yJYrqajX6QXQpRGSfry$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ABJ4N3FLD77Y4MGHKO3GWM34J6AJBAVCNFSM6AAAAACTTZFUYCVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTONBQHAYTKMRZGI__;!!G92We9drHetJ8EofZw!dHsdAlLSGai08CSFWJUxgSGA3kJDc441YRnqK7qLvXYgJpztx5G5T75ekbxSV3JMQNjuM1tT7yJYrqajX6QXQm9YgSvE$>.
You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>
|
Summary
Issue
Listing could include unrelated paths when bucket prefixes or validation didn’t align with the current bucket structure, causing extra listing work and risk of pulling unrelated assets.
Fix
Bucket name generation and prefix selection now mirror the actual layout, and validation explicitly enforces audio//... and visual//....
Savings
Estimated list-traffic reduction when 6 tasks run:
Test plan