Skip to content

Make live scraper integration tests optional or less flaky in CI #16

@Metrohan

Description

@Metrohan

What to build / Ne yapılacak

EN: The repository includes integration tests (tests/integration/test_scrapers_real.py) that call live scraping functions directly against external sites. These tests depend on third-party availability and DOM structure, so they fail for reasons unrelated to local changes — making CI noisy and unreliable.

TR: tests/integration/test_scrapers_real.py dosyası scraper fonksiyonlarını doğrudan canlı sitelere karşı çalıştırıyor. Bu testler üçüncü taraf sitelerin erişilebilirliğine ve DOM yapısına bağlı olduğundan, yerel değişikliklerle ilgisi olmayan sebeplerle başarısız olabiliyor; bu da CI'ı güvenilmez hale getiriyor.

Acceptance criteria

  • Live scraper integration tests are either skipped in CI by default (e.g. -m "not integration" guard already in place) or replaced with deterministic mocked equivalents.
  • pytest (no flags) runs without making real HTTP/browser requests and exits green on a clean checkout.
  • A make test-live target or pytest -m integration is documented for manual runs that actually hit external sites.

Blocked by

None — can start immediately.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciCI/CD and test infrastructureenhancementNew feature or requestscraperRelated to scraper modules

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions