Skip to content

Add CodSpeed performance benchmarks#93

Merged
tarasko merged 12 commits into
masterfrom
codspeed-wizard-1775515249753
Apr 7, 2026
Merged

Add CodSpeed performance benchmarks#93
tarasko merged 12 commits into
masterfrom
codspeed-wizard-1775515249753

Conversation

@codspeed-hq
Copy link
Copy Markdown
Contributor

@codspeed-hq codspeed-hq Bot commented Apr 6, 2026

Summary

This PR integrates CodSpeed for continuous performance monitoring of picows.

Changes

  • Benchmark suite (tests/test_benchmarks.py): 6 benchmarks using pytest-codspeed covering the library's critical paths:

    • URL parsing (simple and with query parameters)
    • Echo roundtrip at small (64B), medium (4KB), and large (64KB) payload sizes
    • Frame sending throughput (1000 small frames)
  • CI workflow (.github/workflows/codspeed.yml): GitHub Actions workflow that compiles the Cython extensions and runs benchmarks with CodSpeed's simulation instrument on every push to master and on pull requests.

  • README badge: CodSpeed badge added to the badge section.

How it works

The benchmarks exercise the full WebSocket stack through loopback connections -- server creation, client handshake, frame building, sending, parsing, and echoing -- which captures the end-to-end performance characteristics of picows. URL parsing benchmarks cover the pure-Python entry point.

CodSpeed's simulation mode provides deterministic, low-variance measurements that are independent of CI runner hardware, making it reliable for detecting regressions.

Note on CI runners

Since this is a personal repository (not an organization), codspeed-macro runners are not available. The workflow uses standard ubuntu-latest runners. Performance variance will be higher than with dedicated macro runners. If the repository is moved to an organization in the future, switching to macro runners is recommended for more stable walltime measurements. See: https://codspeed.io/docs/instruments/walltime#usage-on-personal-github-accounts

Next steps

  • Review the initial benchmark results on the CodSpeed dashboard after the first CI run
  • Consider adding more granular benchmarks as the library evolves (e.g., TLS handshake, auto-ping/pong)

@codspeed-hq
Copy link
Copy Markdown
Contributor Author

codspeed-hq Bot commented Apr 6, 2026

Congrats! CodSpeed is installed 🎉

🆕 3 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Detected benchmarks


Open in CodSpeed

@codspeed-hq codspeed-hq Bot marked this pull request as ready for review April 6, 2026 22:47
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 7, 2026

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.28%. Comparing base (2aea022) to head (2b07a63).
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #93      +/-   ##
==========================================
- Coverage   91.38%   91.28%   -0.10%     
==========================================
  Files           5        5              
  Lines        1079     1079              
  Branches       41       41              
==========================================
- Hits          986      985       -1     
- Misses         82       83       +1     
  Partials       11       11              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@tarasko tarasko changed the title Add CodSpeed performance benchmarks and CI workflow Add CodSpeed performance benchmarks Apr 7, 2026
@tarasko tarasko merged commit da98d5d into master Apr 7, 2026
31 checks passed
@tarasko tarasko deleted the codspeed-wizard-1775515249753 branch April 7, 2026 17:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants