Skip to content

refactor: eliminate code duplication in precompute engine + bench_precompute_sketch binary#233

Merged
milindsrivastava1997 merged 4 commits into
mainfrom
pr/F-refactor-bench-binary
Apr 7, 2026
Merged

refactor: eliminate code duplication in precompute engine + bench_precompute_sketch binary#233
milindsrivastava1997 merged 4 commits into
mainfrom
pr/F-refactor-bench-binary

Conversation

@zzylol
Copy link
Copy Markdown
Contributor

@zzylol zzylol commented Mar 25, 2026

Summary

  • Refactors precompute engine to eliminate code and data duplication across worker/engine modules
  • Fixes design doc: corrects second-tier merge worker design (had correctness issues)
  • Adds bench_precompute_sketch binary for standalone precompute engine benchmarking
  • Adds integration test: e2e_precompute_equivalence in tests/

This is PR F of 6 stacked PRs splitting #162

Stacking order:

  • PR A → main: Core engine (merge first)
  • PR B → PR A: E2E tests + sliding window fix
  • PR C → PR B: Multi-connector ingest + pane-based sliding window
  • PR D → PR C: Benchmarks + scalability tests
  • PR E → PR D: Correctness tests + comparison test
  • PR F (this) → PR E: Refactoring + benchmarking binary (final)

Test plan

  • cargo build --bin bench_precompute_sketch succeeds
  • cargo test integration test in tests/e2e_precompute_equivalence.rs passes

🤖 Generated with Claude Code

@zzylol zzylol force-pushed the pr/F-refactor-bench-binary branch from 2372141 to decbcc8 Compare March 25, 2026 11:32
@zzylol zzylol force-pushed the pr/E-correctness-tests-comparison branch from 8c04bc5 to 47d279e Compare March 25, 2026 11:32
@zzylol zzylol force-pushed the pr/E-correctness-tests-comparison branch 2 times, most recently from 443c7e0 to 31906ef Compare April 6, 2026 21:49
@zzylol zzylol force-pushed the pr/F-refactor-bench-binary branch from decbcc8 to 86a5c69 Compare April 6, 2026 21:58
Base automatically changed from pr/E-correctness-tests-comparison to main April 7, 2026 01:06
@zzylol zzylol force-pushed the pr/F-refactor-bench-binary branch from 86a5c69 to 480ae78 Compare April 7, 2026 01:46
zz_y and others added 4 commits April 7, 2026 08:26
…nary

- Refactor Worker to use WorkerRuntimeConfig struct and Arc<AggregationConfig>
- Add bench_precompute_sketch binary for precompute engine benchmarking
- Add e2e_precompute_equivalence integration test
- Expand design doc with second-tier merge worker design

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Define WorkerRuntimeConfig struct and destructure it in Worker::new
- Update engine.rs to construct WorkerRuntimeConfig when creating workers
- Fix make_worker test helper to use WorkerRuntimeConfig
- Fix AggregationConfig::new calls in accumulator_factory tests (17-arg)
- Fix .inner.sketch.get_n() → .inner.count() in e2e test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…nary

- Add missing 17th arg (value_column: None) to AggregationConfig::new
  in bench_precompute_sketch.rs and e2e_precompute_equivalence.rs
- Add bench_precompute_sketch.rs stub to Dockerfile dependency cache layer

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@zzylol zzylol force-pushed the pr/F-refactor-bench-binary branch from 1c8c533 to 14ceb05 Compare April 7, 2026 13:27
@milindsrivastava1997 milindsrivastava1997 merged commit 2aa78d3 into main Apr 7, 2026
6 checks passed
@milindsrivastava1997 milindsrivastava1997 deleted the pr/F-refactor-bench-binary branch April 7, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants