Customer Service Agent Evaluation

This directory contains the customer service agent and its evaluation tests.

Running Evaluations

There are two ways to run evaluations:

1. Recommended: Programmatic Evaluation (Pytest)

This method is more robust and avoids naming conflicts with the ADK CLI.

From the project root directory (adk_eval/):

PYTHONPATH=. uv run pytest customer_service_agent/test_agent_eval.py

From this directory (customer_service_agent/):

PYTHONPATH=.. uv run pytest test_agent_eval.py

2. ADK CLI Evaluation

From the project root directory (adk_eval/):

uv run adk eval customer_service_agent customer_service_agent/eval.test.json --config_file_path customer_service_agent/test_config.json

From this directory (customer_service_agent/): Note: This may encounter naming conflicts if the file is named agent.py. It is recommended to use the Pytest method above.

uv run adk eval . eval.test.json --config_file_path test_config.json

3. Golden Dataset Evaluation

You can also run evaluation using the golden dataset generated by the ADK web UI.

CLI Method:

uv run adk eval customer_service_agent customer_service_agent/evalset780045.evalset.json --config_file_path customer_service_agent/test_config.json

Pytest Method:

PYTHONPATH=. uv run pytest customer_service_agent/test_golden_eval.py