Skip to content

Fix/Add ReasoningResult confidence_tier/is_actionable computed fields + boundary tests#118

Open
ArokyaMatthew wants to merge 2 commits into
Devnil434:mainfrom
ArokyaMatthew:fix/reasoningresult-confidence-tier-117
Open

Fix/Add ReasoningResult confidence_tier/is_actionable computed fields + boundary tests#118
ArokyaMatthew wants to merge 2 commits into
Devnil434:mainfrom
ArokyaMatthew:fix/reasoningresult-confidence-tier-117

Conversation

@ArokyaMatthew
Copy link
Copy Markdown
Contributor

@ArokyaMatthew ArokyaMatthew commented May 26, 2026

Fixes #117

What changed

  • Added two computed fields on ReasoningResult:
    • confidence_tier: "high" (>= 0.75), "medium" (>= 0.50), otherwise "low"
    • is_actionable: True only when label == "Suspicious" and confidence >= 0.65
  • Implemented them using Pydantic v2 @computed_field so they are included in model_dump(mode="json").
  • Added tests/test_reasoning_schema.py with boundary + serialization coverage.

Tests

  • Added unit tests covering tier boundaries: 0.49, 0.50, 0.74, 0.75, 1.0
  • Added tests for is_actionable threshold + label behavior
  • Verified both fields are serialized in model_dump(mode="json")

Summary by CodeRabbit

  • Refactor

    • Enhanced reasoning result schema to properly expose computed fields for confidence assessment in API responses
  • Tests

    • Added comprehensive test coverage for reasoning schema field computation, confidence tier boundaries, and serialization validation

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 26, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6ab89b39-c807-4ffa-a16f-604e2dec0091

📥 Commits

Reviewing files that changed from the base of the PR and between eadd39f and 7d563e3.

📒 Files selected for processing (2)
  • libs/schemas/reasoning.py
  • tests/test_reasoning_schema.py

📝 Walkthrough

Walkthrough

This PR refactors ReasoningResult to use Pydantic's @computed_field decorator for exposing confidence_tier and is_actionable derived properties. The custom model_dump_json() override is removed, and comprehensive unit tests validate the threshold logic and JSON serialization behavior.

Changes

Computed fields implementation and testing

Layer / File(s) Summary
Computed fields implementation
libs/schemas/reasoning.py
Pydantic imports updated to include computed_field. ReasoningResult adds two computed properties: confidence_tier (derived from confidence thresholds: ≥0.75 → high, ≥0.50 → medium, otherwise low) and is_actionable (true when label=="Suspicious" AND confidence≥0.65). Custom JSON serialization method removed.
Computed field unit tests
tests/test_reasoning_schema.py
Three parameterized pytest tests cover confidence-tier boundary values (0.49, 0.50, 0.74, 0.75, 1.0), is_actionable across label and confidence combinations, and verification that model_dump(mode="json") includes both computed fields with expected values.

🎯 2 (Simple) | ⏱️ ~10 minutes

🐰 Confidence tiers now computed, not hard-coded,
Actionability logic flows through schemas encoded,
Pydantic's decorator makes serialization clean,
Dashboard dashboards dashingly styled, pristine! 🎨✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and specifically summarizes the main changes: adding computed fields (confidence_tier and is_actionable) to ReasoningResult and adding boundary tests.
Linked Issues check ✅ Passed The PR implements all requirements from issue #117: confidence_tier with correct thresholds (0.75, 0.50), is_actionable property with label and confidence checks, Pydantic @computed_field implementation for serialization, and comprehensive tests covering boundary values and serialization.
Out of Scope Changes check ✅ Passed All changes are directly scoped to issue #117 requirements: modifications to libs/schemas/reasoning.py for computed fields and new test file tests/test_reasoning_schema.py with comprehensive tests.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ArokyaMatthew
Copy link
Copy Markdown
Contributor Author

@Devnil434 , please review

@kunal-9090
Copy link
Copy Markdown

I checked both failing CI jobs on this PR. Phase 3 failed because services/init.py eagerly imports tracking, which pulls unrelated tracking imports into memory tests. Phase 4 failed on Ruff errors in the reasoning package.\n\nI prepared a fix branch here: https://github.com/kunal-9090/Eagle/tree/codex/fix-pr-118-ci\n\nFix included:\n- stop eager tracking imports in services/init.py\n- correct services/tracking/tracker.py to import settings from libs.config.settings\n- clean Ruff issues in services/reasoning/vlm.py, services/reasoning/pipeline.py, and services/reasoning/scene_graph.py\n\nLocal verification passes for both lint commands:\n- python -m ruff check services/memory/ libs/schemas/memory.py services/init.py services/tracking/tracker.py\n- python -m ruff check services/reasoning/ libs/schemas/reasoning.py libs/schemas/memory.py services/init.py services/tracking/tracker.py\n\nLocal pytest is blocked only because this machine lacks akeredis, which the GitHub workflow installs. I could not push directly to this contributor fork; GitHub returned 403 for kunal-9090.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Issue — Add confidence_tier computed property to ReasoningResult

3 participants