Skip to content

CDSPT-15588 Implemented module for producing type-strict dataframes#17

Merged
TiffanyCheng27 merged 2 commits into
mainfrom
CDSPT-15588-create-datatype-mapping-function-in-mario
Jun 22, 2026
Merged

CDSPT-15588 Implemented module for producing type-strict dataframes#17
TiffanyCheng27 merged 2 commits into
mainfrom
CDSPT-15588-create-datatype-mapping-function-in-mario

Conversation

@scottbw

@scottbw scottbw commented Jun 18, 2026

Copy link
Copy Markdown
Member

📌 Summary

Implemented module for producing type-strict dataframes from CSV and Hyper

🧪 What Changed?

  • New feature
  • Bug fix
  • Refactor
  • Documentation only
  • CI/CD or tooling
  • Other (please explain):

✅ Checklist

Code Quality

  • Code is clean, commented, and follows our conventions
  • No hardcoded values or secrets
  • Reusable and modular where possible

Testing

  • I have tested this change locally
  • Relevant tests have been added/updated
  • All existing tests still pass

Documentation

  • Code comments or inline explanations added where needed
  • README / documentation updated (if applicable)
  • Any environment variables added are documented

Security & Deployment

  • No sensitive data in code or logs
  • No breaking changes introduced
  • Deploy scripts / jobs updated (if needed)

⚙️ Actions Required After Pulling

  • Yes.
  • No.

If you checked Yes, please provide instructions below:


🧠 Context or Screenshots

https://jiscdev.atlassian.net/browse/CDSPT-15588


🚨 Risk & Impact

  • No user-facing changes
  • Minimal risk — small changes isolated to specific areas
  • Medium risk — changes affect multiple areas
  • High risk — could impact critical paths or user flows

Notes for reviewers (optional):

Comment thread mario/dataframe_utils.py
return True
if x_str in {"false", "0"}:
return False
return pd.NA

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the bool data x=1.0 or 0.0 then this convert function will return pd.NA
There might be a risk that csv save bool as float type and cause this error.
Can also consider to add trim value function for x_str

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed some changes to make boolean handling more robust.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked. All good

…cluding native bools, floats, and string floats. Added test cases to confirm.
@TiffanyCheng27 TiffanyCheng27 merged commit 8416938 into main Jun 22, 2026
3 checks passed
@TiffanyCheng27 TiffanyCheng27 deleted the CDSPT-15588-create-datatype-mapping-function-in-mario branch June 22, 2026 09:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants