Skip to content

Conversation

@BrendanWalsh
Copy link
Collaborator

Quality of life improvements for the build process:

  1. Pipeline parameterization (pipeline.yaml):

    • Add toggle parameters for each CI job type: testStyle, testUnit,
      testPython, testR, testE2E, testWebsiteSamples, websiteAutoDeploy,
      publishArtifacts
    • Jobs now check their corresponding parameter before running
    • Allows developers to skip expensive jobs during PR iteration
  2. Pip install optimization (project/CodegenPlugin.scala):

    • Add --no-deps flag to pip install in installPipPackage task
    • Speeds up wheel installation by skipping dependency resolution
  3. Verification scripts (tools/):

    • verify_python_mapping.py: Validates Java→Python class mapping
      for sentinel stages that have historically exposed export issues
    • verify_snapshot_alignment.py: Ensures wheel versions match JAR
      versions in local Maven cache (auto-detects Scala version)

@github-actions
Copy link

Hey @BrendanWalsh 👋!
Thank you so much for contributing to our repository 🙌.
Someone from SynapseML Team will be reviewing this pull request soon.

We use semantic commit messages to streamline the release process.
Before your pull request can be merged, you should make sure your first commit and PR title start with a semantic prefix.
This helps us to create release messages and credit you for your hard work!

Examples of commit messages with semantic prefixes:

  • fix: Fix LightGBM crashes with empty partitions
  • feat: Make HTTP on Spark back-offs configurable
  • docs: Update Spark Serving usage
  • build: Add codecov support
  • perf: improve LightGBM memory usage
  • refactor: make python code generation rely on classes
  • style: Remove nulls from CNTKModel
  • test: Add test coverage for CNTKModel

To test your commit locally, please follow our guild on building from source.
Check out the developer guide for additional guidance on testing your change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant