Skip to content

Jim dev#240

Merged
kazewong merged 585 commits into
mainfrom
jim-dev
Jul 23, 2025
Merged

Jim dev#240
kazewong merged 585 commits into
mainfrom
jim-dev

Conversation

@kazewong

@kazewong kazewong commented Jul 23, 2025

Copy link
Copy Markdown
Owner

This pull request includes many development changes and bug fixes since the last revamp of the prior system.

The major updates in this PR involves the now more standardized runinterface, which should be used for production-related task. Another major feature updates is the incorporation of orchestration tools like Dagster, in order to provide an interface for fully automated deployment. Note that the orchestration example and tools are not directly within the jim package, but we provide examples of deploying such system. The run interface is relatively stable going forward, while the orchestration interface is still in development mode.

Another major update in this PR is the restructuring of the single event likelihood interface, specially standardizing the class hierarchy of the likelihood classes. Different marginalization schemes are now in their own class, such that parameters are standardized as class variables instead of run-time dictionary values.

Once we hit production level stable, we should start keeping a change log and define the release schedule of jim.

Summary by CodeRabbit

  • New Features

    • Introduced a modular, extensible framework for gravitational wave parameter estimation, including new core modules for data handling, detectors, likelihoods, priors, transformations, and run management.
    • Added advanced support for single-event and population analyses, with improved configuration, serialization, and workflow orchestration.
    • Integrated new command-line tools, configuration files, and example scripts for streamlined execution and reproducibility.
    • Enhanced support for Docker containerization and continuous integration workflows.
  • Bug Fixes

    • Improved numerical stability and JIT compatibility for mathematical and transformation utilities.
    • Refined prior and transform logic for greater correctness and flexibility.
  • Documentation

    • Added new and updated documentation files describing abstraction layers, workflows, and usage examples.
  • Style

    • Refactored codebase for improved readability, consistent formatting, and updated import paths.
  • Tests

    • Expanded and modernized unit and integration test coverage for priors, transforms, likelihoods, run management, and core utilities.
    • Added new test scripts and notebooks for cross-validation with external tools.
  • Chores

    • Updated dependencies, Python version requirements, and project configuration for enhanced compatibility and maintainability.
    • Improved .gitignore patterns and development tooling configurations.
  • Revert

    • Removed deprecated or redundant modules, scripts, and configuration files, consolidating functionality into the new core structure.

kazewong and others added 28 commits July 12, 2025 08:28
Add placeholder unit test for likelihood module
- Reformat long function calls and asset specs for clarity - Use
consistent string quoting and indentation - Move assert error messages
to multi-line format for readability - Add properties to
SingleEventLikelihood for duration and detector names - Refactor
likelihood classes for modularity and clarity
Add abstract _likelihood method to SingleEventLikelihood and refactor
subclasses to implement core likelihood logic in _likelihood. Move
parameter updates and fixed values to evaluate methods for clarity.
Expand unit tests for likelihood classes and add fixtures
Fix HeterodynedTransientLikelihoodFD initialization
…-in-single_eventlikelihoodpy

208 replacing keyword arguments in single event likelihood
@kazewong kazewong merged commit 41005eb into main Jul 23, 2025
2 of 7 checks passed
@coderabbitai

coderabbitai Bot commented Jul 23, 2025

Copy link
Copy Markdown

Caution

Review failed

The pull request is closed.

Walkthrough

This update is a comprehensive refactor and expansion of the codebase, introducing new container build files, workflows, and a major reorganization of the core modules for gravitational wave parameter estimation. It adds new abstractions for priors, transforms, likelihoods, data handling, detectors, and run management, while removing legacy single_event modules. Extensive new tests, documentation, and Dagster pipelines are included.

Changes

Files / Groups Change Summary
.github/workflows/container.yml, .github/workflows/pre-commit.yml Added/updated GitHub Actions workflows for Docker image publishing and pre-commit checks.
.gitignore, example/.gitignore Updated and deleted ignore files to broaden patterns and remove local ignores.
.pre-commit-config.yaml, pyproject.toml Upgraded tool versions, dependencies, and added new optional/development dependencies and configurations.
containers/Containerfile.cpu, containers/Containerfile.cuda Added new container build files for CPU and CUDA environments.
docs/gen_ref_pages.py, docs/workflow.md, example/examples.md Added documentation on workflow abstraction; removed old example docs; cleaned up whitespace.
example/GW150914_IMRPhenomD.py, example/GW170817_IMRPhenomD.py, example/notebooks/*.ipynb,
example/configs/*.yaml, example/examples.md, src/jimgw/__init__.py,
src/jimgw/constants.py, src/jimgw/single_event/*.py, src/jimgw/jim.py
Deleted legacy example scripts, notebooks, configs, and old single_event modules.
example/GW150914_IMRPhenomPV2.py, example/GW170817_IMRPhenomPV2.py Refactored examples to use new core modules, updated imports, and expanded sampler configuration.
example/GW150914_RunAPI.py, example/GW150914_like_injection_Pv2.py,
example/workbench.py, example/workbench_bilby.py
Added new example scripts demonstrating the new API and workflow.
example/example_GW150914_config.yaml Added new YAML config for GW150914 inference.
jim_dagster/InjectionRecovery/*, jim_dagster/RealDataCatalog/*, jim_dagster/SingleRun/* Added new Dagster asset pipelines, definitions, and diagnostic asset stubs.
src/jimgw/core/__init__.py, src/jimgw/core/constants.py Added core package init and new constants module.
src/jimgw/core/base.py Removed RunManager base class.
src/jimgw/core/jim.py Added new Jim class as the main inference interface.
src/jimgw/core/population/injection_util.py Added utility to generate fiducial population samples for injection studies.
src/jimgw/core/prior.py Refactored priors to composite pattern, added Gaussian and Rayleigh priors, unified transforms.
src/jimgw/core/single_event/data.py Added new Data and PowerSpectrum classes for GW data and PSD handling.
src/jimgw/core/single_event/detector.py Added new detector framework for ground-based GW detectors.
src/jimgw/core/single_event/gps_times.py Added JAX-compatible GPS/UTC/Julian time and sidereal time conversion utilities.
src/jimgw/core/single_event/likelihood.py Added new likelihood classes for transient GW events, including marginalizations and heterodyned methods.
src/jimgw/core/single_event/prior.py Added new chirp mass prior class.
src/jimgw/core/single_event/transforms.py Refactored and expanded parameter transformation classes, improved invertibility and stability.
src/jimgw/core/single_event/utils.py Added new utility functions for inner products, mass, coordinate, and spin conversions.
src/jimgw/core/single_event/wave.py, src/jimgw/core/single_event/waveform.py Minor refactoring for einsum usage and whitespace.
src/jimgw/core/transforms.py Refactored and expanded transform classes, added new transforms and improved reverse logic.
src/jimgw/core/utils.py Added numerically stable math utilities for use with JAX.
src/jimgw/run/cli/execute_single_run.py Added CLI script to execute a single run and save results.
src/jimgw/run/library/IMRPhenomPv2_standard_cbc.py, src/jimgw/run/library/class_definitions.py Added run definition classes and enums for CBC parameter estimation.
src/jimgw/run/run_definition.py, src/jimgw/run/single_event_run_definition.py Added abstract and single-event run definition classes for deterministic, extensible run configs.
src/jimgw/run/run_manager.py, src/jimgw/run/single_event_run_manager.py Added run manager classes for orchestrating and analyzing runs, with plotting utilities.
test/integration/*, test/unit/*, test/verify_jit_gmst.py Refactored and expanded tests to cover new modules, transforms, priors, likelihoods, and run management.
test/unit/source_files/* Added scripts to generate reference data for spin and sky location transforms.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant RunManager
    participant RunDefinition
    participant Jim
    participant Likelihood
    participant Prior
    participant Transforms

    User->>RunManager: Instantiate with RunDefinition or config file
    RunManager->>RunDefinition: Load or initialize run config
    RunManager->>Jim: Initialize with Likelihood, Prior, Transforms, and sampler params
    User->>RunManager: sample()
    RunManager->>Jim: sample()
    Jim->>Prior: sample_initial_condition()
    Jim->>Likelihood: evaluate_posterior()
    Jim->>Transforms: apply sample/likelihood transforms
    Jim->>Jim: Run sampler (training/production)
    Jim-->>RunManager: Return samples, log-prob, loss, diagnostics
    RunManager-->>User: Provide results, plots, summary
Loading

Estimated code review effort

🎯 5 (Critical) | ⏱️ ~120+ minutes

This is a major refactor and expansion involving new core abstractions, removal of legacy code, extensive new features, and broad test coverage. It touches nearly every aspect of the codebase and will require careful, multi-disciplinary review.

Possibly related PRs

  • kazewong/jim#194: Adds a unit test for transform functionality, particularly enhancing reverse_bijective_transform—directly related as the main PR also updates this function.
  • kazewong/jim#177: Introduces RayleighPrior and PeriodicTransform classes, matching additions in this PR's prior and transform modules.
  • kazewong/jim#201: Fixes rotation matrix usage in SkyFrameToDetectorFrameSkyPositionTransform, which is refactored and improved in this PR.

Suggested labels

enhancement

Suggested reviewers

  • tsunhopang
  • thomasckng
  • CharmaineWONG2
  • SSL32081

Poem

In a warren of code, a rabbit did dwell,
Refactoring modules, and testing as well.
With priors and transforms, detectors anew,
Samplers and likelihoods, all shiny and true.
Pipelines now hop, containers now run—
This carrot of progress is second to none!
🐇✨

Note

⚡️ Unit Test Generation - Beta

CodeRabbit's unit test generation is now available in Beta! Automatically generate comprehensive unit tests for your code changes, ensuring better test coverage and catching edge cases you might miss. Our AI analyzes your code structure and creates tests that follow best practices and your project's testing patterns. Learn more here, or just try it under ✨ Finishing Touches.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch jim-dev

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants