Skip to content

Add lightweight experiment management and versioning #12

@novis10813

Description

@novis10813

Summary

Add basic experiment tracking to ResearchSession, allowing users to save/load experiments and track factor evolution over time.

Background

Factor research involves many iterations. Without experiment tracking, it's easy to lose track of what worked and what didn't. This provides a lightweight solution before full Knowledge Graph integration.

Goals

  • Save experiment configurations and results
  • Load and compare historical experiments
  • Track factor performance over time

Proposed Tasks

  • Add ResearchSession.save_experiment(name: str, output_dir: str) method
    • Saves factor definition, analysis results, backtest results
    • Creates timestamped directory with config.json
    • Reuses FactorAnalysisResult.save() structure
  • Add ResearchSession.load_experiment(path: str) method
    • Loads experiment configuration
    • Returns Experiment object with metadata
  • Add ResearchSession.list_experiments(output_dir: str) method
    • Lists all saved experiments
    • Returns summary (name, date, key metrics)
  • Create simple SQLite-based experiment registry (optional, for faster querying)
  • Add unit tests for experiment save/load
  • Update documentation with examples

Notes

  • Should integrate with existing FactorAnalysisResult.save() structure
  • Keep it lightweight (file-based) - full Knowledge Graph comes later
  • Consider experiment comparison utilities (diff two experiments)
  • Should support both manual naming and auto-generated names

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions