Skip to content

Add: decompress csv function in utils.py#15

Merged
scottbw merged 2 commits into
mainfrom
CDSPT-15585-Move-the-decompress_csv-function-from-the-Athena-DAG-into-Mario
Jun 19, 2026
Merged

Add: decompress csv function in utils.py#15
scottbw merged 2 commits into
mainfrom
CDSPT-15585-Move-the-decompress_csv-function-from-the-Athena-DAG-into-Mario

Conversation

@TiffanyCheng27

Copy link
Copy Markdown
Contributor

No description provided.

@scottbw scottbw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a couple of cases worth checking:

  1. If called with a path that doesn't end in .gz, it will silently overwrite the original file! So best check it does end in '.gz' and raise an error if it doesn't
  2. if the output folder doesn't exist it will raise an error, maybe add a makedirs first?
  3. It will silently overwrite the output file it it already exists, maybe again something to check and raise, or at least warn in the logs

@scottbw scottbw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting those guards in, approved.

@scottbw scottbw merged commit 639e951 into main Jun 19, 2026
3 checks passed
@scottbw scottbw deleted the CDSPT-15585-Move-the-decompress_csv-function-from-the-Athena-DAG-into-Mario branch June 19, 2026 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants