chore : remove pyproject.toml, add library namespace by EncHawk · Pull Request #147 · kubeflow/pipelines-components

EncHawk · 2026-04-21T17:32:13Z

pyproject.toml was removed to include the repository into kubeflow-sdk as part of pipelines. it is followed here

the pipelines, components directories are included under a common library/ namespace, this is to ensure that import command contians library in the sdk as followed kubeflow/sdk#125.

issues : #80

Checklist:

Pre-Submission Checklist

All tests and CI checks pass
Pre-commit hooks pass without errors
You have signed off your commits
The title for your pull request (PR) should follow our title convention.
Learn more about the pull request title convention used in this repository.

… pipelines and components. Signed-off-by: D.Leap <dilipkumar2000.r@gmail.com>

google-oss-prow · 2026-04-21T17:32:19Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mprahl for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

google-oss-prow · 2026-04-21T17:32:33Z

The following users are mentioned in OWNERS file(s) but are untrusted for the following reasons. One way to make the user trusted is to add them as members of the kubeflow org. You can then trigger verification by writing /verify-owners in a comment.

test-reviewer
- User is not a member of the org. Satisfy at least one of these conditions to make the user trusted.
test-owner
- User is not a member of the org. Satisfy at least one of these conditions to make the user trusted.

EncHawk · 2026-04-21T17:34:46Z

@kramaranya just added the changes

@manaswinidas could you verify if all your concerns are fixed from here

Copilot

Pull request overview

This PR restructures the repository to support inclusion as a git submodule in kubeflow-sdk, introducing a top-level library/ namespace for components/pipelines and removing the Python packaging configuration (pyproject.toml).

Changes:

Move components and pipelines under a shared library/ namespace and adjust imports accordingly.
Remove pyproject.toml (previously used for packaging + uv extras + tool configs).
Add/relocate tests and asset metadata/ownership files under the new library/ layout.

Reviewed changes

Copilot reviewed 38 out of 61 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
pyproject.toml	Removes packaging/tooling configuration previously used by CI and local dev.
pipelines/training/init.py	Removes old pipelines namespace package init.
pipelines/evaluation/init.py	Removes old pipelines namespace package init.
pipelines/deployment/init.py	Removes old pipelines namespace package init.
pipelines/data_processing/init.py	Removes old pipelines namespace package init.
pipelines/init.py	Removes old top-level pipelines aggregation module.
pipelines/data_processing/sdg/tests/data/prompts/generate_question.yaml	Removes old SDG pipeline test prompt asset (moved under `library/`).
pipelines/data_processing/sdg/tests/data/llm_test_flow.yaml	Removes old SDG pipeline test flow asset (moved under `library/`).
library/init.py	Adds `library` top-level Python package.
library/pipelines/init.py	Adds convenience imports for pipeline categories under `library`.
library/pipelines/data_processing/init.py	Adds data_processing pipelines namespace under `library`.
library/pipelines/data_processing/README.md	Adds category README for data_processing pipelines under `library`.
library/pipelines/data_processing/sdg/pipeline.py	Updates pipeline to import SDG component from `library.components...`; adjusts `__main__` compilation import.
library/pipelines/data_processing/sdg/tests/init.py	Adds tests package marker for SDG pipeline tests.
library/pipelines/data_processing/sdg/tests/test_pipeline.py	Adds compilation test for the SDG pipeline under the `library` namespace.
library/pipelines/data_processing/sdg/README.md	Adds SDG pipeline README under `library`.
library/pipelines/data_processing/sdg/metadata.yaml	Adds SDG pipeline metadata under `library`.
library/pipelines/data_processing/sdg/OWNERS	Adds SDG pipeline OWNERS under `library`.
library/pipelines/training/init.py	Adds training pipelines namespace under `library`.
library/pipelines/evaluation/init.py	Adds evaluation pipelines namespace under `library`.
library/pipelines/deployment/init.py	Adds deployment pipelines namespace under `library`.
library/components/init.py	Adds convenience imports for component categories under `library`.
library/components/data_processing/init.py	Adds data_processing components namespace under `library`.
library/components/data_processing/README.md	Adds category README for data_processing components under `library`.
library/components/data_processing/sdg/component.py	Removes `kfp.compiler` import; adjusts `__main__` compilation import.
library/components/data_processing/sdg/README.md	Adds SDG component README under `library`.
library/components/data_processing/sdg/metadata.yaml	Adds SDG component metadata under `library`.
library/components/data_processing/sdg/OWNERS	Adds SDG component OWNERS under `library`.
library/components/data_processing/sdg/tests/test_component_unit.py	Adds extensive unit tests for SDG component behavior.
library/components/data_processing/sdg/tests/test_component_local.py	Adds local runner tests (including optional real-LLM E2E tests).
library/components/data_processing/sdg/tests/test_data/transform_test_flow.yaml	Adds transform-only flow test fixture.
library/components/data_processing/sdg/tests/test_data/llm_test_flow.yaml	Adds LLM flow test fixture.
library/components/data_processing/sdg/tests/test_data/sample_input.jsonl	Adds sample JSONL input fixture.
library/components/data_processing/sdg/tests/test_data/prompts/generate_question.yaml	Adds prompt fixture for LLM flow tests.
library/components/data_processing/sdg/shared/run_local.py	Adds a local-run script that patches KFP LocalRunner behavior for optional artifacts.
library/components/data_processing/yoda_data_processor/component.py	Refactors prompt prefix helper and updates `__main__` compilation import.
library/components/data_processing/yoda_data_processor/init.py	Exposes the yoda component via package exports.
library/components/data_processing/yoda_data_processor/README.md	Updates README to reference metadata.yaml for details.
library/components/data_processing/yoda_data_processor/metadata.yaml	Adds component metadata under `library`.
library/components/data_processing/yoda_data_processor/OWNERS	Adds component OWNERS under `library`.
library/components/data_processing/yoda_data_processor/tests/support.py	Adds in-memory dataset test doubles/helpers.
library/components/data_processing/yoda_data_processor/tests/test_component_unit.py	Adds unit tests for yoda component logic and outputs.
library/components/data_processing/yoda_data_processor/tests/test_component_local.py	Adds local execution tests for yoda component.
library/components/deployment/init.py	Adds deployment components namespace under `library`.
library/components/deployment/test_data/component_valid/metadata.yaml	Adds metadata test fixture under `library`.
library/components/deployment/test_data/component_valid/OWNERS	Adds OWNERS test fixture under `library`.
library/components/evaluation/init.py	Adds evaluation components namespace under `library`.
library/components/training/init.py	Adds training components namespace under `library`.
components/training/init.py	Removes old components namespace package init.
components/evaluation/init.py	Removes old components namespace package init.
components/deployment/init.py	Removes old components namespace package init.
components/data_processing/init.py	Removes old components namespace package init.
components/init.py	Removes old top-level components aggregation module.
components/deployment/component_valid/OWNERS	Removes old OWNERS fixture from prior location.
components/data_processing/yoda_data_processor/tests/test_component_unit.py	Removes old yoda unit tests from prior location.
components/data_processing/yoda_data_processor/tests/test_component_local.py	Removes old yoda local tests from prior location.
init.py	Updates stated import usage to `library.*` (but still contains broken imports).

Comments suppressed due to low confidence (1)

library/components/data_processing/sdg/component.py:9

This component uses pandas (imported inside sdg()), but the component’s packages_to_install only includes sdg-hub. Unless the base image/transitive deps guarantee pandas, the component can fail at runtime. Consider adding pandas explicitly to packages_to_install to match the component’s direct imports.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-04-21T17:45:40Z

 """Kubeflow Pipelines Components

 A collection of reusable components and pipelines for Kubeflow Pipelines.

 Usage:
-    from kfp_components import components, pipelines
-    from kfp_components.components import training
-    from kfp_components.pipelines import evaluation
+    from library import components, pipelines
+    from library.components import training
+    from library.pipelines import evaluation
 """


With pyproject.toml removed, the repo's CI/dev setup likely breaks: .github/actions/setup-python-ci runs uv sync --extra ..., which requires a pyproject defining dependency groups/extras (lint/test). If the intent is to keep this repo buildable/testable standalone, keep a minimal pyproject for uv/tooling config or update the CI workflows/action to install dependencies without pyproject.

Copilot · 2026-04-21T17:45:40Z

 try:
    # Try relative imports first (works when installed as package)
    from . import components, pipelines
 except ImportError:
    # Fallback to absolute imports (works during testing with sys.path modification)
    import components  # noqa: F401
    import pipelines  # noqa: F401


__init__.py still imports components/pipelines, but those directories no longer exist after the move under library/, so importing this module will always raise ImportError. Either update these imports to from .library import components, pipelines (and similarly in the fallback) or remove this file if it’s no longer intended to be importable as a package entrypoint.

Copilot · 2026-04-21T17:45:40Z

    import pipelines  # noqa: F401
-
-__all__ = ["components", "pipelines"]
+#since the 


Line 20 has an incomplete comment (#since the) which looks accidental and will fail style/lint checks. Please remove it or replace it with a complete explanation.

Suggested change

#since the

# Re-export the package's public submodules.

EncHawk · 2026-04-21T23:09:44Z

how should the CI runs be changed since it fails due to not being able to install packages without the pyproject.toml file,
also the __init__.py file in the root, should pipelines and components still be part of all[]?
@manaswinidas @kramaranya

manaswinidas · 2026-04-25T05:15:55Z

/ok-to-test

EncHawk · 2026-05-01T19:03:40Z

hi @manaswinidas @HumairAK could you suggest a change for these failing tests?

manaswinidas · 2026-05-11T05:33:56Z

All 9 failing CI checks fail with the same root cause — not a test issue:

error: No `pyproject.toml` found in current directory or any parent directory

The CI workflows (build, lint, compile-check, targeted-tests, validate-metadata, README validation, YAML lint, scripts tests) all run uv sync --extra test / uv sync --extra lint, which requires a pyproject.toml to resolve dependency groups. Since this PR deletes it entirely, every job crashes before any test even runs.

Since #80 intends this repo to become a git submodule inside kubeflow/sdk, removing pyproject.toml is the right call. The fix should be on the CI side — update the workflow files (.github/workflows/ and .github/actions/setup-python-ci/) to install dependencies without uv sync --extra ... (e.g., use uv pip install with explicit deps or a requirements-ci.txt), or remove/disable the CI checks that will move to the parent SDK repo.

added library namespace, removed pyproject.toml addressing changes in…

385bc01

… pipelines and components. Signed-off-by: D.Leap <dilipkumar2000.r@gmail.com>

Copilot AI review requested due to automatic review settings April 21, 2026 17:32

google-oss-prow Bot requested review from HumairAK and shivchander April 21, 2026 17:32

google-oss-prow Bot added size/XL do-not-merge/invalid-owners-file labels Apr 21, 2026

Copilot started reviewing on behalf of EncHawk April 21, 2026 17:35 View session

Copilot AI reviewed Apr 21, 2026

View reviewed changes

google-oss-prow Bot added the ok-to-test label Apr 25, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore : remove pyproject.toml, add library namespace #147

chore : remove pyproject.toml, add library namespace #147
EncHawk wants to merge 1 commit into
kubeflow:mainfrom
EncHawk:main

EncHawk commented Apr 21, 2026

Uh oh!

google-oss-prow Bot commented Apr 21, 2026

Uh oh!

google-oss-prow Bot commented Apr 21, 2026

Uh oh!

EncHawk commented Apr 21, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

Copilot AI Apr 21, 2026

Uh oh!

EncHawk commented Apr 21, 2026 •

edited

Loading

Uh oh!

manaswinidas commented Apr 25, 2026

Uh oh!

EncHawk commented May 1, 2026

Uh oh!

manaswinidas commented May 11, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

EncHawk commented Apr 21, 2026

Pre-Submission Checklist

Uh oh!

google-oss-prow Bot commented Apr 21, 2026

Uh oh!

google-oss-prow Bot commented Apr 21, 2026

Uh oh!

EncHawk commented Apr 21, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

EncHawk commented Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

manaswinidas commented Apr 25, 2026

Uh oh!

EncHawk commented May 1, 2026

Uh oh!

manaswinidas commented May 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

EncHawk commented Apr 21, 2026 •

edited

Loading

manaswinidas commented May 11, 2026 •

edited

Loading