Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
73a0ba4
feat: Add per-message sampling with adaptive strategies and profile v…
willibrandon Aug 23, 2025
4527571
fix: resolve golangci-lint issues in sampling implementation
willibrandon Aug 23, 2025
a5991da
fix: add missing sampling methods to otel adapter mock loggers
willibrandon Aug 23, 2025
d8d69c6
fix: resolve adaptive sampling hash bias on Windows
willibrandon Aug 23, 2025
7b3150f
fix: add missing sampling methods to middleware noOpLogger
willibrandon Aug 23, 2025
22d753a
refactor: address Copilot code review feedback
willibrandon Aug 23, 2025
3789fef
docs: split sampling documentation into separate guide
willibrandon Aug 23, 2025
90f8042
docs: add sampling guide link to documentation section
willibrandon Aug 23, 2025
a7a6e2c
feat: add sampling observability and fix flaky timing test
willibrandon Aug 23, 2025
e5e203b
feat: add String() and PrometheusMetrics() methods to SamplingMetrics
willibrandon Aug 23, 2025
00e522d
fix: resolve race condition in sampling debug flag
willibrandon Aug 23, 2025
414984d
docs: add monitoring & observability section to sampling guide
willibrandon Aug 23, 2025
2daf9d3
feat: add sampling observability examples and fix profile stats
willibrandon Aug 23, 2025
ca2cbdb
feat: add sampling observability examples and fix profile stats
willibrandon Aug 23, 2025
1a27b90
fix: increase debounce test timeout to prevent CI flakiness
willibrandon Aug 23, 2025
2686449
fix: correct name field assignment in AddCustomProfile functions and …
willibrandon Aug 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,40 @@ orderLogger := mtlog.ForType[OrderService](log)
orderLogger.Information("Processing order") // SourceContext: "OrderService"
```

## Per-Message Sampling

Efficient log volume management through intelligent per-message sampling. mtlog provides comprehensive sampling capabilities to help control log volume in production while preserving important events.

### Quick Examples

```go
// Basic sampling methods
logger.Sample(10).Info("Every 10th message")
logger.SampleRate(0.2).Info("20% of messages")
logger.SampleDuration(time.Second).Info("Once per second")
logger.SampleFirst(100).Info("First 100 only")

// Adaptive sampling - maintains target events/second
logger.SampleAdaptive(100).Info("Auto-adjusting rate")

// Use predefined profiles
logger.SampleProfile("HighTrafficAPI").Info("API call")
logger.SampleProfile("ProductionErrors").Error("Error occurred")
```

### Key Features

- **Multiple Strategies**: Counter, rate, time-based, first-N, group, conditional, and exponential backoff sampling
- **Adaptive Sampling**: Automatically adjusts rates to maintain target throughput with hysteresis and dampening
- **Predefined Profiles**: Ready-to-use configurations for common scenarios
- **Custom Profiles**: Define and register your own reusable sampling configurations
- **Version Management**: Support for versioned profiles with auto-migration
- **Zero Allocations**: Optimized for minimal performance impact

### Learn More

For comprehensive documentation including advanced strategies, configuration options, and best practices, see the [**Sampling Guide**](docs/sampling-guide.md).

## Structured Fields with With()

The `With()` method provides a convenient way to add structured fields to log events, following the slog convention of accepting variadic key-value pairs:
Expand Down Expand Up @@ -888,6 +922,13 @@ See the [examples](./examples) directory and [OTEL examples](./adapters/otel/exa
- [Type-based logging](./examples/fortype/main.go)
- [LogContext scoped properties](./examples/logcontext/main.go)
- [Advanced filtering](./examples/filtering/main.go)
- [Conditional logging](./examples/conditional/main.go)
- [Sampling basics](./examples/sampling/main.go)
- [Advanced sampling](./examples/sampling-advanced/main.go)
- [Sampling monitoring](./examples/sampling-monitoring/main.go)
- [Sampling debug](./examples/sampling-debug/main.go)
- [Sampling profiles](./examples/sampling-profiles/main.go)
- [Router sinks](./examples/router/main.go)
- [Capturing](./examples/capturing/main.go)
- [LogValue interface](./examples/logvalue/main.go)
- [Console themes](./examples/themes/main.go)
Expand All @@ -906,6 +947,8 @@ See the [examples](./examples) directory and [OTEL examples](./adapters/otel/exa
- [Dynamic levels](./examples/dynamic-levels/main.go)
- [Configuration](./examples/configuration/main.go)
- [Generics usage](./examples/generics/main.go)
- [With properties](./examples/with/main.go)
- [Showcase](./examples/showcase/main.go)
- [HTTP middleware](./adapters/middleware/examples/) - net/http, Gin, Echo, Fiber, Chi

## Ecosystem Compatibility
Expand Down Expand Up @@ -1280,6 +1323,7 @@ For comprehensive guides and examples, see the [docs](./docs) directory:

- **[Quick Reference](./docs/quick-reference.md)** - Quick reference for all features
- **[Template Syntax](./docs/template-syntax.md)** - Guide to message template syntaxes
- **[Sampling Guide](./docs/sampling-guide.md)** - Comprehensive per-message sampling documentation
- **[Sinks Guide](./docs/sinks.md)** - Complete guide to all output destinations
- **[Routing Patterns](./docs/routing-patterns.md)** - Advanced event routing patterns and best practices
- **[Dynamic Level Control](./docs/dynamic-levels.md)** - Runtime level management
Expand Down
21 changes: 20 additions & 1 deletion adapters/middleware/helpers.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"net/http"
"strings"
"time"

"github.com/willibrandon/mtlog/core"
)
Expand Down Expand Up @@ -342,4 +343,22 @@ func (n *noOpLogger) D(template string, args ...any) {}
func (n *noOpLogger) I(template string, args ...any) {}
func (n *noOpLogger) W(template string, args ...any) {}
func (n *noOpLogger) E(template string, args ...any) {}
func (n *noOpLogger) F(template string, args ...any) {}
func (n *noOpLogger) F(template string, args ...any) {}

// Sampling methods - all return self for no-op
func (n *noOpLogger) Sample(num uint64) core.Logger { return n }
func (n *noOpLogger) SampleDuration(duration time.Duration) core.Logger { return n }
func (n *noOpLogger) SampleRate(rate float32) core.Logger { return n }
func (n *noOpLogger) SampleFirst(num uint64) core.Logger { return n }
func (n *noOpLogger) SampleGroup(groupName string, num uint64) core.Logger { return n }
func (n *noOpLogger) SampleWhen(predicate func() bool, num uint64) core.Logger { return n }
func (n *noOpLogger) SampleBackoff(key string, factor float64) core.Logger { return n }
func (n *noOpLogger) ResetSampling() {}
func (n *noOpLogger) ResetSamplingGroup(groupName string) {}
func (n *noOpLogger) EnableSamplingSummary(period time.Duration) core.Logger { return n }
func (n *noOpLogger) GetSamplingStats() (sampled uint64, skipped uint64) { return 0, 0 }
func (n *noOpLogger) SampleProfile(profileName string) core.Logger { return n }
func (n *noOpLogger) SampleAdaptive(targetEventsPerSecond uint64) core.Logger { return n }
func (n *noOpLogger) SampleAdaptiveWithOptions(targetEventsPerSecond uint64, minRate, maxRate float64, adjustmentInterval time.Duration) core.Logger {
return n
}
6 changes: 4 additions & 2 deletions adapters/middleware/metrics_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -469,8 +469,10 @@ func TestMetricsMiddleware(t *testing.T) {
}

// Duration should be reasonable (not too much overhead)
if duration > sleepDuration+10*time.Millisecond {
t.Errorf("Duration seems too long: %v (expected ~%v)", duration, sleepDuration)
// Allow more tolerance on CI systems which may have timing variations
maxOverhead := 20 * time.Millisecond
if duration > sleepDuration+maxOverhead {
t.Errorf("Duration seems too long: %v (expected ~%v with max %v overhead)", duration, sleepDuration, maxOverhead)
}
})
}
Expand Down
18 changes: 18 additions & 0 deletions adapters/otel/bridge_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,24 @@ func (m *mockLogger) Warn(template string, args ...any) {
m.Warning(template, args...)
}

// Sampling methods - all return self for testing
func (m *mockLogger) Sample(n uint64) core.Logger { return m }
func (m *mockLogger) SampleDuration(duration time.Duration) core.Logger { return m }
func (m *mockLogger) SampleRate(rate float32) core.Logger { return m }
func (m *mockLogger) SampleFirst(n uint64) core.Logger { return m }
func (m *mockLogger) SampleGroup(groupName string, n uint64) core.Logger { return m }
func (m *mockLogger) SampleWhen(predicate func() bool, n uint64) core.Logger { return m }
func (m *mockLogger) SampleBackoff(key string, factor float64) core.Logger { return m }
func (m *mockLogger) ResetSampling() {}
func (m *mockLogger) ResetSamplingGroup(groupName string) {}
func (m *mockLogger) EnableSamplingSummary(period time.Duration) core.Logger { return m }
func (m *mockLogger) GetSamplingStats() (sampled uint64, skipped uint64) { return 0, 0 }
func (m *mockLogger) SampleProfile(profileName string) core.Logger { return m }
func (m *mockLogger) SampleAdaptive(targetEventsPerSecond uint64) core.Logger { return m }
func (m *mockLogger) SampleAdaptiveWithOptions(targetEventsPerSecond uint64, minRate, maxRate float64, adjustmentInterval time.Duration) core.Logger {
return m
}

func TestBridge(t *testing.T) {
// Create a mock logger
logger := &mockLogger{level: core.InformationLevel}
Expand Down
20 changes: 19 additions & 1 deletion adapters/otel/fuzz_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -286,4 +286,22 @@ func (m *mockFuzzLogger) ForContext(propertyName string, value any) core.Logger
func (m *mockFuzzLogger) WithContext(ctx context.Context) core.Logger { return m }
func (m *mockFuzzLogger) With(args ...any) core.Logger { return m }
func (m *mockFuzzLogger) Info(template string, args ...any) {}
func (m *mockFuzzLogger) Warn(template string, args ...any) {}
func (m *mockFuzzLogger) Warn(template string, args ...any) {}

// Sampling methods - all return self for testing
func (m *mockFuzzLogger) Sample(n uint64) core.Logger { return m }
func (m *mockFuzzLogger) SampleDuration(duration time.Duration) core.Logger { return m }
func (m *mockFuzzLogger) SampleRate(rate float32) core.Logger { return m }
func (m *mockFuzzLogger) SampleFirst(n uint64) core.Logger { return m }
func (m *mockFuzzLogger) SampleGroup(groupName string, n uint64) core.Logger { return m }
func (m *mockFuzzLogger) SampleWhen(predicate func() bool, n uint64) core.Logger { return m }
func (m *mockFuzzLogger) SampleBackoff(key string, factor float64) core.Logger { return m }
func (m *mockFuzzLogger) ResetSampling() {}
func (m *mockFuzzLogger) ResetSamplingGroup(groupName string) {}
func (m *mockFuzzLogger) EnableSamplingSummary(period time.Duration) core.Logger { return m }
func (m *mockFuzzLogger) GetSamplingStats() (sampled uint64, skipped uint64) { return 0, 0 }
func (m *mockFuzzLogger) SampleProfile(profileName string) core.Logger { return m }
func (m *mockFuzzLogger) SampleAdaptive(targetEventsPerSecond uint64) core.Logger { return m }
func (m *mockFuzzLogger) SampleAdaptiveWithOptions(targetEventsPerSecond uint64, minRate, maxRate float64, adjustmentInterval time.Duration) core.Logger {
return m
}
Loading
Loading