Skip to content

test(campaign): add boundary value analysis and test gaps for decomposer #535

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
theRebelliousNerd wants to merge 1 commit into
base: main
Choose a base branch
from
Open

test(campaign): add boundary value analysis and test gaps for decomposer #535

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
450 changes: 450 additions & 0 deletions .quality_assurance/2026_05_23_decomposer_boundary_analysis.md
View file
Open in desktop

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions cmd/nerd/chat/.nerd/session.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"session_id": "sess_1779316969672792758",
"session_id": "sess_1779511068319050636",
"started_at": "2026-05-20T22:42:49.693359663Z",
"last_active_at": "2026-05-20T22:42:49.693359833Z",
"last_active_at": "2026-05-23T04:37:48.322127917Z",
"turn_count": 0,
"suspended": false,
"history_file": "sess_1779316969672792758.json"
"history_file": "sess_1779511068319050636.json"
}
17 changes: 17 additions & 0 deletions cmd/nerd/chat/.nerd/sessions/sess_1779510744853008096.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"session_id": "sess_1779510744853008096",
"messages": [
{
"role": "assistant",
"content": "System Ready",
"time": "2026-05-23T04:32:24.852250536Z"
},
{
"role": "assistant",
"content": "**Scan complete**\n\n| Metric | Value |\n|--------|-------|\n| Files indexed | 10 |\n| Directories | 0 |\n| Facts generated | 100 |\n| Duration | 1.00s |\n\nThe kernel has been updated with fresh codebase facts.",
"time": "2026-05-23T04:32:24.858288332Z"
}
],
"created_at": "2026-05-23T04:32:24.859958874Z",
"updated_at": "2026-05-23T04:32:24.859958983Z"
}
17 changes: 17 additions & 0 deletions cmd/nerd/chat/.nerd/sessions/sess_1779511068319050636.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
{
"session_id": "sess_1779511068319050636",
"messages": [
{
"role": "assistant",
"content": "System Ready",
"time": "2026-05-23T04:37:48.318445809Z"
},
{
"role": "assistant",
"content": "**Scan complete**\n\n| Metric | Value |\n|--------|-------|\n| Files indexed | 10 |\n| Directories | 0 |\n| Facts generated | 100 |\n| Duration | 1.00s |\n\nThe kernel has been updated with fresh codebase facts.",
"time": "2026-05-23T04:37:48.321118556Z"
}
],
"created_at": "2026-05-23T04:37:48.322433015Z",
"updated_at": "2026-05-23T04:37:48.322433105Z"
}
2 changes: 1 addition & 1 deletion internal/campaign/.nerd/campaigns/c_test_triage/assault/triage/latest.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"metadata": {
"campaign_id": "/c_test_triage",
"failures": 0,
"generated_at": "2026-05-20T22:42:54Z",
"generated_at": "2026-05-23T04:37:52Z",
"results_dir": ".nerd/campaigns/c_test_triage/assault/results",
"success": 0,
"total": 0
Expand Down
12 changes: 12 additions & 0 deletions internal/campaign/.nerd/campaigns/c_test_triage/assault/triage/triage_20260523T043250.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"summary": "total_results=0 success=0 failures=0\nNo failures detected.\n",
"recommended_tasks": [],
"metadata": {
"campaign_id": "/c_test_triage",
"failures": 0,
"generated_at": "2026-05-23T04:32:50Z",
"results_dir": ".nerd/campaigns/c_test_triage/assault/results",
"success": 0,
"total": 0
}
}
12 changes: 12 additions & 0 deletions internal/campaign/.nerd/campaigns/c_test_triage/assault/triage/triage_20260523T043752.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"summary": "total_results=0 success=0 failures=0\nNo failures detected.\n",
"recommended_tasks": [],
"metadata": {
"campaign_id": "/c_test_triage",
"failures": 0,
"generated_at": "2026-05-23T04:37:52Z",
"results_dir": ".nerd/campaigns/c_test_triage/assault/results",
"success": 0,
"total": 0
}
}
10 changes: 10 additions & 0 deletions internal/campaign/decomposer_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,16 @@ func (m *mockLLMClient) SchemaCapable() bool {
return m.schemaCapable
}

// TODO: TEST_GAP: [Null/Undefined/Empty] Verify NewDecomposer handles an empty workspace string without defaulting to root or polluting unintended locations.
// TODO: TEST_GAP: [Null/Undefined/Empty] Verify Decompose handles Decomposer instances with nil optional dependencies (advisoryBoard, intelligenceGatherer, edgeCaseDetector).
// TODO: TEST_GAP: [Null/Undefined/Empty] Verify readDocumentsFromPath behaves correctly when an element in SourcePaths is an empty string `[""]`.
// TODO: TEST_GAP: [Type Coercion] Verify cleanJSONResponse cleanly handles partially valid JSON with incorrect nested types (e.g., string instead of int for phase budget).
// TODO: TEST_GAP: [Type Coercion] Verify seedDocFacts asserts strictly typed Mangle Atoms instead of Strings when creating campaign facts.
// TODO: TEST_GAP: [User Request Extremes] Verify readDocumentsFromDir gracefully handles a directory tree with 1,000,000 nested files without exhausting memory or file descriptors.
// TODO: TEST_GAP: [User Request Extremes] Verify the Decomposer can process a simulated 50 million line monorepo by using streaming logic/sparse retrieval instead of full RAM loading.
// TODO: TEST_GAP: [User Request Extremes] Verify validation logic prevents the LLM from hallucinating non-existent subsystems, tools, or coding languages in the plan.
// TODO: TEST_GAP: [State Conflicts] Verify Decomposer handles concurrent calls to setter methods (SetPromptProvider, SetShardLister) while Decompose is running without data races.
// TODO: TEST_GAP: [State Conflicts] Verify readDocumentsFromDir handles Time-of-Check/Time-of-Use (TOC/TOU) race conditions where a file is deleted after metadata is gathered but before reading content.
Comment on lines +100 to +109
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Narrow TODOs to uncovered subcases to avoid stale test-gap markers.

Several TODOs overlap tests already present in this file (e.g., empty source paths, JSON coercion paths, nil intelligence handling, deleted-file behavior). Please rewrite these as precise missing subcases instead of broad gaps, so the backlog reflects true coverage holes.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/campaign/decomposer_test.go` around lines 100 - 109, The TODOs list
in decomposer_test.go is too broad and duplicates existing tests; replace each
high-level marker with narrow, precise missing subcases tied to specific
functions: for NewDecomposer add a test case for an explicit empty workspace
string (verify it returns an error or a non-root workspace and does not write
files), for Decompose add a test where
advisoryBoard/intelligenceGatherer/edgeCaseDetector are nil to assert graceful
handling, for readDocumentsFromPath add a case where SourcePaths contains an
empty string element to assert it skips or returns a clear error, for
cleanJSONResponse add a case with partially valid JSON that has wrong nested
types (e.g., phase budget as string) to assert strict coercion behavior, for
seedDocFacts assert Mangle atom types are preserved (not coerced to strings),
and for readDocumentsFromDir add focused TOC/TOU and deleted-file subcases (file
removed after stat but before read) and a resource-limit simulation (streaming
fallback) rather than a million-file stress test; also add a concurrency test
exercising SetPromptProvider/SetShardLister called while Decompose runs to
ensure no data races.

func TestNewDecomposer(t *testing.T) {
mockKernel := &core.RealKernel{} // Minimal struct
mockClient := &mockLLMClient{}
Expand Down