Skip to content

docs(benchmarks): add honest-framing README for the manual benchmark layer (B1)#94

Merged
DemchaAV merged 1 commit into
developfrom
docs/benchmarks-readme-roles
May 31, 2026
Merged

docs(benchmarks): add honest-framing README for the manual benchmark layer (B1)#94
DemchaAV merged 1 commit into
developfrom
docs/benchmarks-readme-roles

Conversation

@DemchaAV

Copy link
Copy Markdown
Owner

Summary

Wires up Track B1 — the honest-claims prerequisite before the 1.6.6 Maven Central debut. The benchmarks/ module has no README.md today; this PR adds one and pins the framing.

The page's load-bearing message: the manual benchmark harness is a smoke / diff / endurance tool, not a JMH-grade benchmark. Once GraphCompose ships on Maven Central, downstream callers will read the repo before deciding whether to adopt it; any number quoted from this layer in marketing copy would set false expectations and get torn apart on Hacker News. This page exists to make it embarrassing for any maintainer (or AI agent) to do that without thinking.

What the page covers

Section What it answers
What this is / is not Manual harness, not JMH-grade. No warmup control, no forked JVM, no GC profiling rigour.
When to use Smoke before release, pre/post diff on one machine, stress / endurance overnight runs.
When not to use Publishable "X% faster than Y" claims. Architectural decisions (pick the right invariant, not a number). Cross-library comparisons that read too much into a single point.
Files in this module Role table for each *.javaCurrentSpeedBenchmark is the CI smoke runner, ComparativeBenchmark is the iText / openHTMLToPDF / JasperReports comparison (rough local only), etc.
Running Exact mvnw exec:java invocations matching the CI perf-smoke job — taken straight from ci.yml after PR-7.1 / PR-83 cleanups.
How to read a report What avgMs / p95Ms / docsPerSec / avgKB / peakMB mean and the limits of each.
Roadmap Cross-links Track C (B3 → B4 → B5 → B6) in the 1.7.0 plan as the JMH layer that will replace this for rigorous measurements.

Why now (and why this scope)

The B1 row in the readiness taskboard is a soft prereq for the 1.6.6 cut — explicitly because honest framing has to be in place before Maven Central traffic starts looking at the repo. The taskboard's exact wording:

B1 (честные claims) — soft-prereq публичного 1.6.6 (первый Central publish, нужно убрать преувеличения «faster than X» из публичной коммуникации).

Scope is intentionally one file plus a CHANGELOG entry — no measurement-code changes, no fixture restructuring. Those land in Track C (1.7.0).

Verification

$ ./mvnw -B -ntp test -pl . -Dtest='CanonicalSurfaceGuardTest,DocumentationCoverageTest'
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0

benchmarks/README.md contains no legacy-API tokens so no
CanonicalSurfaceGuardTest allowlist entry is needed. (The legacy
inventory in G1 / G3 was the explicit reason for those allowlist
entries; this page doesn't name legacy types — it talks about benchmark
methodology.)

CHANGELOG entry added to v1.6.6 — Planned under ### Build.

Test plan

  • Doc guards green locally
  • CI green on PR (Guards / JDK 17/21/25 / Examples Smoke / no-poi-suite / binary-compat)
  • Reviewer skim — "What this is / is not" + "When not to use" are the load-bearing sections; the rest is mechanical role / invocation documentation
  • (Out of scope) follow-up: once Track C B3-B6 ships the JMH layer, update the "Roadmap" section to point at it as the supersedure path

…layer (B1)

Wires up Track B1 from the v1.6.5->1.7 readiness taskboard - the honest-claims prerequisite before the 1.6.6 Maven Central debut. There is no README in the benchmarks module today; this PR adds one and explicitly positions the harness as a smoke / diff / endurance tool, not a JMH-grade benchmark.

Sections: what this is and is not, when to use, when NOT to use (publishable claims, architectural decisions, cross-library comparisons), file-by-file role table, exact CI smoke invocation, how to read a report, JMH roadmap (Track C B3 to B6 in 1.7.0).

Tone: senior-defensive. Numbers from this harness are 'rough local comparisons suitable for did this change regress something obviously', not 'X percent faster than Y'. The aim is to make sure no maintainer or downstream caller quotes a number from this layer in a public communication once Central artefacts start shipping.

Verification: ./mvnw test -pl . -Dtest='CanonicalSurfaceGuardTest,DocumentationCoverageTest' - 7 tests, 0 failures (~18s). benchmarks/ has no legacy API tokens so no allowlist entry needed.
@DemchaAV DemchaAV merged commit 2ba7458 into develop May 31, 2026
11 checks passed
@DemchaAV DemchaAV deleted the docs/benchmarks-readme-roles branch May 31, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant