Speed up integration-test fixtures on Windows (shrink copied transcripts; skip symlink tests when unavailable)#249
Conversation
The realistic-integration tests exercise behavior (hierarchy processing, caching, CLI flags, regeneration), not raw data volume, yet rendering cost scales with transcript size. Two fixture session files are ~5 MB each and get re-rendered in every function-scoped copy, dominating suite wall-time — on Windows especially, where the per-message render work (not copytree, which measured at ~0.15s) is the bottleneck. Prefix-truncate oversized session files in the *copy* only, leaving the source tree pristine so volume-sensitive tests (test_performance, test_json_real_projects, test_dag*) keep the full data. A leading prefix is DAG-safe: an entry's parentUuid always refers to an earlier line, so dropping the tail removes only leaf descendants. agent-/ subagents files and the teammates/JSSoundRecorder fixtures are left intact (the 600 KB threshold sits above their largest files). Before/after (Windows, this file, serial): 150.3s -> 93.2s (-38%) Full unit suite (-n auto --dist=worksteal): 185.7s -> 146.9s (-21%) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TestIndexInlineAggregateLoopCharacterization creates a symlink to discover a synthetic project. On Windows without elevation/Developer Mode, os.symlink raises OSError [WinError 1314], failing all three tests. Probe symlink capability once at import and skipif the class, so the suite is green on stock Windows while still running the tests everywhere symlinks work (Linux/macOS/elevated Windows). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughIntegration test fixtures now shrink oversized standalone JSONL transcripts after copying project trees, and characterization tests now skip symlink-dependent coverage when symlink creation is unavailable. ChangesTest Stability Improvements
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/test_integration_realistic.py`:
- Around line 71-77: The truncation logic in the JSONL fixture copy path can
still write a partial record when no complete line exists within the kept
prefix. Update the truncation step around the `cut`/`last_newline` handling so
that `load_transcript()` only receives complete JSONL lines: if `last_newline`
is not found within the kept prefix, skip writing the truncated file (or
otherwise avoid writing any partial object) before `jsonl_file.write_bytes` is
called.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ba425536-a218-4d4d-8a49-0e92d235b323
📒 Files selected for processing (2)
test/test_integration_realistic.pytest/test_session_scan_characterization.py
In _shrink_large_transcripts, if the kept prefix contains no newline (the first record alone exceeds the keep size), skip truncating that file rather than writing a partial JSON record that would parse as malformed JSONL. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Part of #248.
Problem
The test suite is far slower on Windows than Linux. Profiling the worst
offenders — the
test_integration_realistic.pyfamily (18 of the 25slowest tests, 30–45s each) — showed the cost is not the
shutil.copytreeof the fixture data (~0.15s) but the repeatedrendering + cache build of the large fixture transcripts, which every
function-scoped test re-does. A couple of fixture session files are ~5 MB
each and dominate.
Change
(
_shrink_large_transcripts), leaving the source tree pristine sovolume-sensitive tests (
test_performance,test_dag*, …) keep thefull data. The truncation is a leading-prefix cut, which is DAG-safe:
an entry's
parentUuidalways refers to an earlier line, so droppingthe tail removes only leaf descendants.
agent-/subagentsfiles andcontent-sensitive fixtures stay below the 600 KB threshold.
available. On Windows without Developer Mode/elevation,
os.symlinkraises
OSError [WinError 1314]; the tests now probe capability onceand
skipif, so they still run everywhere symlinks work(Linux/macOS/elevated Windows).
Results (local Windows)
test_integration_realistic.py(serial)-n auto)No production code is touched; the coverage trade-off is documented in the
fixture module. This is an absolute speedup (it helps Linux
proportionally too); the Windows/Linux ratio is addressed by the
companion PRs tracked in #248.
Summary by CodeRabbit