Skip to content

Feat/imputation improvements#79

Merged
jorgeMFS merged 2 commits into
mainfrom
feat/imputation-improvements
Aug 14, 2025
Merged

Feat/imputation improvements#79
jorgeMFS merged 2 commits into
mainfrom
feat/imputation-improvements

Conversation

@jorgeMFS

@jorgeMFS jorgeMFS commented Aug 14, 2025

Copy link
Copy Markdown
Owner

Summary by Sourcery

Introduce a flexible, strategy-agnostic imputation framework with global and per-column settings and mask-and-score tuning; enhance GUI, CLI scripts, and reporting diagnostics; update documentation; bump version to 1.2.0; fix GUI metric defaults and typing compatibility.

New Features:

  • Add strategy-agnostic imputation panel supporting global and per-column configuration with optional mask-and-score tuning
  • Add clinical_all_features_e2e.py script for comprehensive clinical pipeline runs
  • Include categorical PSI and Cramér’s V metrics in bias diagnostics with per-variable trigger reporting

Bug Fixes:

  • Fix GUI quality metrics picker defaults for dict-style configuration
  • Resolve Python 3.9 typing compatibility issue in the GUI

Enhancements:

  • Make minor tuning UI improvements

Build:

  • Bump project version to 1.2.0

Documentation:

  • Update README and usage docs to reflect new strategy-agnostic imputation and tuning options

…reporting triggers include PSI/Cramér’s V; add clinical_all_features_e2e script; misc compatibility
…ggers, new clinical_all_features_e2e script, CLI flags)
@sourcery-ai

sourcery-ai Bot commented Aug 14, 2025

Copy link
Copy Markdown
Contributor
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

This PR overhauls imputation to be strategy-agnostic with global and per-column tuning, refreshes documentation and reporting (including categorical bias metrics), adds a comprehensive clinical E2E script, bumps version to 1.2.0, and delivers GUI bug fixes and compatibility improvements.

Class diagram for imputation configuration and strategy selection

classDiagram
    class ImputationConfig {
      +strategy: str
      +params: dict
      +per_column: dict
      +tuning: bool
    }
    class Imputer {
      +fit_transform(data)
      +set_strategy(strategy)
      +set_params(params)
      +set_per_column(per_column)
      +run_tuning()
    }
    ImputationConfig <|-- Imputer
    Imputer o-- "1..*" ImputationStrategy
    class ImputationStrategy {
      +name: str
      +apply(data, params)
    }
    ImputationStrategy <|-- MeanImputer
    ImputationStrategy <|-- MedianImputer
    ImputationStrategy <|-- ModeImputer
    ImputationStrategy <|-- KNNImputer
    ImputationStrategy <|-- MICEImputer
    ImputationStrategy <|-- SVDImputer
    ImputationStrategy <|-- NoneImputer
Loading

Class diagram for bias metrics reporting

classDiagram
    class BiasMetrics {
      +numeric_metrics: list
      +categorical_metrics: list
      +report_triggers(variable)
    }
    BiasMetrics <|-- NumericBiasMetrics
    BiasMetrics <|-- CategoricalBiasMetrics
    class NumericBiasMetrics {
      +SMD
      +VarianceRatio
      +KS
    }
    class CategoricalBiasMetrics {
      +PSI
      +CramersV
    }
Loading

File-Level Changes

Change Details Files
Implement strategy-agnostic imputation and tuning
  • Revamped README to support global strategy with optional per-column overrides
  • Extended mask-and-score tuning to all imputation methods
  • Adjusted config YAML examples for new parameters
README.md
Add comprehensive clinical end-to-end script
  • Documented new scripts/clinical_all_features_e2e.py in README
  • Noted the addition in CHANGELOG
README.md
CHANGELOG.md
Enhance bias reporting with categorical metrics
  • Included PSI and Cramér’s V metrics with per-variable trigger indicators
  • Updated bias description in CLI workflow bullets
README.md
CHANGELOG.md
Bump project version to 1.2.0
  • Updated version in pyproject.toml and src/phenoqc/init.py
  • Added new 1.2.0 section in CHANGELOG
pyproject.toml
src/phenoqc/__init__.py
CHANGELOG.md
Apply GUI bug fixes and compatibility tweaks
  • Fixed quality metrics picker defaults for dict-style metrics
  • Resolved Python 3.9 typing issues with minor UI tuning improvements
CHANGELOG.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@codecov-commenter

codecov-commenter commented Aug 14, 2025

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Files with missing lines Coverage Δ
src/phenoqc/__init__.py 100.00% <100.00%> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@jorgeMFS jorgeMFS merged commit 883aabe into main Aug 14, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants