Skip to content

Improvement/238 tests for evaluation metrics#242

Open
dkkdark wants to merge 47 commits intomainfrom
improvement/238-tests-for-evaluation-metrics
Open

Improvement/238 tests for evaluation metrics#242
dkkdark wants to merge 47 commits intomainfrom
improvement/238-tests-for-evaluation-metrics

Conversation

@dkkdark
Copy link
Copy Markdown
Collaborator

@dkkdark dkkdark commented Mar 24, 2026

No description provided.

@dkkdark dkkdark self-assigned this Mar 24, 2026
@dkkdark dkkdark requested a review from NoB0 March 24, 2026 11:01
@github-actions
Copy link
Copy Markdown

Current Branch Main Branch
Coverage Badge Coverage Badge

Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/evaluation/test_satisfaction_metric.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_satisfaction_metric.py Outdated
@dkkdark dkkdark requested a review from NoB0 March 24, 2026 16:39
Copy link
Copy Markdown
Collaborator

@NoB0 NoB0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General comment on format. In the codebase we typically use types from the typing package. For docstrings the verb is conjugated. I commented on a few of places to change (please check for others that I may have skipped).

Left some comments with clarification questions.

Comment thread tests/conftest.py Outdated
Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/conftest.py Outdated
Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
Comment thread tests/evaluation/test_utility_metrics.py Outdated
@dkkdark dkkdark requested a review from NoB0 March 31, 2026 08:55
Comment thread tests/evaluation/test_quality_metric.py Outdated
Comment thread tests/evaluation/test_success_rate_metric.py Outdated
Comment thread tests/evaluation/test_successful_recommendation_round_ratio_metric.py Outdated
) == pytest.approx(1 / 2)


def test_successful_round_ratio_rejects_unannotated_dialogue(
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

new test with unannotated dialogue

@dkkdark dkkdark requested a review from NoB0 April 14, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants