docs(benchmarks): add honest-framing README for the manual benchmark layer (B1)#94
Merged
Merged
Conversation
…layer (B1) Wires up Track B1 from the v1.6.5->1.7 readiness taskboard - the honest-claims prerequisite before the 1.6.6 Maven Central debut. There is no README in the benchmarks module today; this PR adds one and explicitly positions the harness as a smoke / diff / endurance tool, not a JMH-grade benchmark. Sections: what this is and is not, when to use, when NOT to use (publishable claims, architectural decisions, cross-library comparisons), file-by-file role table, exact CI smoke invocation, how to read a report, JMH roadmap (Track C B3 to B6 in 1.7.0). Tone: senior-defensive. Numbers from this harness are 'rough local comparisons suitable for did this change regress something obviously', not 'X percent faster than Y'. The aim is to make sure no maintainer or downstream caller quotes a number from this layer in a public communication once Central artefacts start shipping. Verification: ./mvnw test -pl . -Dtest='CanonicalSurfaceGuardTest,DocumentationCoverageTest' - 7 tests, 0 failures (~18s). benchmarks/ has no legacy API tokens so no allowlist entry needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Wires up Track B1 — the honest-claims prerequisite before the 1.6.6 Maven Central debut. The
benchmarks/module has noREADME.mdtoday; this PR adds one and pins the framing.The page's load-bearing message: the manual benchmark harness is a smoke / diff / endurance tool, not a JMH-grade benchmark. Once GraphCompose ships on Maven Central, downstream callers will read the repo before deciding whether to adopt it; any number quoted from this layer in marketing copy would set false expectations and get torn apart on Hacker News. This page exists to make it embarrassing for any maintainer (or AI agent) to do that without thinking.
What the page covers
*.java—CurrentSpeedBenchmarkis the CI smoke runner,ComparativeBenchmarkis the iText / openHTMLToPDF / JasperReports comparison (rough local only), etc.mvnw exec:javainvocations matching the CIperf-smokejob — taken straight fromci.ymlafter PR-7.1 / PR-83 cleanups.avgMs/p95Ms/docsPerSec/avgKB/peakMBmean and the limits of each.Why now (and why this scope)
The B1 row in the readiness taskboard is a soft prereq for the 1.6.6 cut — explicitly because honest framing has to be in place before Maven Central traffic starts looking at the repo. The taskboard's exact wording:
Scope is intentionally one file plus a CHANGELOG entry — no measurement-code changes, no fixture restructuring. Those land in Track C (1.7.0).
Verification
benchmarks/README.mdcontains no legacy-API tokens so noCanonicalSurfaceGuardTestallowlist entry is needed. (The legacyinventory in G1 / G3 was the explicit reason for those allowlist
entries; this page doesn't name legacy types — it talks about benchmark
methodology.)
CHANGELOG entry added to
v1.6.6 — Plannedunder### Build.Test plan
no-poi-suite/binary-compat)