Skip to content

Model validation#409

Open
2cylu2 wants to merge 2 commits intodevelopfrom
model-validation
Open

Model validation#409
2cylu2 wants to merge 2 commits intodevelopfrom
model-validation

Conversation

@2cylu2
Copy link
Copy Markdown
Contributor

@2cylu2 2cylu2 commented Apr 4, 2026

This work is for issue #404.

What’s included:

  • shared model registry for HARP models
  • HARP featured model dropdown now reads from the registry
  • pytest-based validation harness
  • local PyHARP smoke tests
  • first-pass remote validation for HARP dropdown models

Run locally:
python3 -m pip install -r requirements-model-validation.txt
python3 -m pytest tests/model_validation -rA --run-network-validation

For Stability validation:
export HARP_STABILITY_API_KEY=...
python3 -m pytest tests/model_validation -rA --run-network-validation

Current status:

  • local example validation works
  • some remote dropdown models pass
  • several remote Hugging Face models fail with 503, timeouts, or remote runtime errors
  • Stability validation reaches the API but currently returns 400 Bad Request
  • more provider/model-specific fixes are still needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant