testmark is a Go tool and library for enhancing and automating benchmark workflows.
It provides a CLI for formatting go test -bench output and a library with utilities for custom benchmarking.
To install the latest version of testmark, run:
go install github.com/rah-0/testmark@latestRequirements:
- Go 1.18 or later (for module support)
Once installed, you can use the testmark CLI to process Go benchmark output and make it more readable.
To format your benchmark results using testmark, you can pipe the output from go test directly into the tool:
go test -run=^$ -bench="^BenchmarkFunction$" -benchmem | testmark
This will take the benchmark results and output them in a more readable format, where time units like ns/op and B/op will be converted to friendly forms separated by \t.
Before:
Sample/Count10-8 100000 114430 ns/op 808 B/op 18 allocs/op
Sample/Count100-8 100000 126958 ns/op 840 B/op 18 allocs/op
Sample/Count1000-8 100000 248419 ns/op 1048 B/op 18 allocs/op
Sample/Count10000-8 100000 797351 ns/op 1048 B/op 18 allocs/op
Sample/Count100000-8 100000 3651349 ns/op 1128 B/op 18 allocs/op
Sample/Count1000000-8 100000 37593152 ns/op 1176 B/op 18 allocs/op
After:
Sample/Count10-8 100000 114430 ns/op 808 B/op 18 allocs/op CPU[114µs 430ns]
Sample/Count100-8 100000 126958 ns/op 840 B/op 18 allocs/op CPU[126µs 958ns]
Sample/Count1000-8 100000 248419 ns/op 1048 B/op 18 allocs/op CPU[248µs 419ns] MEM[1KiB 24B]
Sample/Count10000-8 100000 797351 ns/op 1048 B/op 18 allocs/op CPU[797µs 351ns] MEM[1KiB 24B]
Sample/Count100000-8 100000 3651349 ns/op 1128 B/op 18 allocs/op CPU[3ms 651µs 349ns] MEM[1KiB 104B]
Sample/Count1000000-8 100000 37593152 ns/op 1176 B/op 18 allocs/op CPU[37ms 593µs 152ns] MEM[1KiB 128B]
While testmark enhances Go benchmark output by converting raw values into readable formats, be cautious when using it in automated toolchains or with other CLI tools.
- Formatting changes: The tool adds readable units (e.g.,
s,KiB) to the benchmark output, which may break downstream tools expecting a specific format (e.g., exact tab or space separation). - Overwriting output files: If redirecting output, ensure files are not unintentionally overwritten by
testmark. - Unmatched input:
testmarkwill leave non-benchmark lines or lines not matching the expected Go benchmark format unchanged, ensuring no unexpected alterations occur.
Always test the full integration if you plan to use testmark as part of a larger automation pipeline.
In addition to the CLI formatting tool, testmark provides several packages that you can import into your projects:
- benchutil: Lightweight benchmarking utilities for measuring performance without Go's testing framework
- testutil: Test helpers for managing resources and capturing panics
The benchutil package provides a lightweight, self-contained benchmarking utility to measure performance and memory usage without relying on go test.
package main
import (
"fmt"
"github.com/rah-0/testmark/benchutil"
)
func allocLarge() []byte {
return make([]byte, 2*int64(1 << 20)) // 2MiB
}
func allocSmall() []byte {
return make([]byte, int64(1 << 20)) // 1MiB
}
func main() {
b := benchutil.NewBench().SetRuns(1000)
slow := b.Measure(func(){_ = allocSmall()})
fast := b.Measure(func(){_ = allocLarge()})
nsDiff := fast.DeltaNsPct(slow)
bDiff := fast.DeltaBPct(slow)
fmt.Printf("Time diff: %.2f%%\n", nsDiff)
fmt.Printf("Memory diff: %.2f%%\n", bDiff)
}Naturally, there are precision issues when benching outside of *testing.B.
func BenchmarkAllocSmall(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
allocSmall()
}
}
//Outputs:
//BenchmarkAllocSmall-8 15234 81454 ns/op 1048579 B/op 1 allocs/op
func BenchmarkAllocLarge(b *testing.B) {
b.ResetTimer()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
allocLarge()
}
}
//Outputs:
//BenchmarkAllocLarge-8 3946 286477 ns/op 2097160 B/op 1 allocs/op
func TestMeasure_Allocations(t *testing.T) {
tuner := NewBench().SetRuns(10000)
small := tuner.Measure(func() {
_ = allocSmall()
})
large := tuner.Measure(func() {
_ = allocLarge()
})
fmt.Println(small.NsPerOp) //84120
fmt.Println(small.BytesPerOp) //1048596
fmt.Println(large.NsPerOp) //216284
fmt.Println(large.BytesPerOp) //2097165
}So, what's the drift?
| Source | Type | Ns/Op | B/Op | Drift Ns | Drift B |
|---|---|---|---|---|---|
| go test | Small | 81454 | 1048579 | — | — |
| benchutil | Small | 84120 | 1048596 | +3.27% | +0.00% |
| go test | Large | 286477 | 2097160 | — | — |
| benchutil | Large | 216284 | 2097165 | −24.51% | +0.00% |
- Memory (
B/op) is nearly identical in both tools, confirming accuracy of the GC-based measurement. - Time (
ns/op) is less consistent,benchutilreports higher time for small allocs and lower for large allocs. This is somewhat expected due to GC noise, scheduling, and lack of internal runtime calibration (b.ResetTimer(), etc.).
The testutil package offers wrappers to:
- Catch and report panics in tests or benchmarks
- Wrap TestMain() with setup/teardown hooks
func TestMain(m *testing.M) {
testutil.TestMainWrapper(testutil.TestConfig{
M: m,
LoadResources: func() error {
// Setup before tests
return nil
},
UnloadResources: func() error {
// Cleanup after tests
return nil
},
})
}func TestSafe(t *testing.T) {
testutil.RunTestWithRecover(t, func(t *testing.T) {
panic("unexpected")
})
}UnloadResourcesruns even if tests fail or panic- Keeps tests alive even if panics occur
- Prints full stack traces for debug
If this saved you time or brought value to your project, feel free to show some support. Every bit is appreciated 🙂
