diff --git a/docs/.agent/docs_inventory.md b/docs/.agent/docs_inventory.md index b5ade76..4498ea4 100644 --- a/docs/.agent/docs_inventory.md +++ b/docs/.agent/docs_inventory.md @@ -9,6 +9,7 @@ | `docs/cookbook/src/recipes/file_uploads.md` | Recipe for File Uploads | Docs | Updated (Buffered) | | `docs/cookbook/src/recipes/websockets.md` | Recipe for Real-time Chat | Docs | Updated (Extractors) | | `docs/cookbook/src/recipes/background_jobs.md` | Recipe for Background Jobs | Docs | OK | +| `docs/cookbook/src/recipes/structured_logging.md` | Recipe for Structured Logging | Docs | New | | `docs/cookbook/src/recipes/tuning.md` | Performance Tuning | Docs | DELETED | | `docs/cookbook/src/recipes/new_feature.md` | New Feature Guide | Docs | DELETED | | `docs/cookbook/src/architecture/action_pattern.md` | Action Pattern Guide | Docs | DELETED | diff --git a/docs/.agent/last_run.json b/docs/.agent/last_run.json index b2bb51d..9d1515b 100644 --- a/docs/.agent/last_run.json +++ b/docs/.agent/last_run.json @@ -1,5 +1,5 @@ { "last_processed_ref": "v0.1.335", - "date": "2026-02-19", - "notes": "Deleted incorrect recipes (tuning, action pattern). Updated File Uploads and WebSockets recipes for accuracy. Expanded Learning Path with mini projects." + "date": "2026-02-26", + "notes": "Updated File Uploads recipe to correct buffering details and API usage. Added Structured Logging recipe. Improved Learning Path." } diff --git a/docs/.agent/run_report_2026-02-26.md b/docs/.agent/run_report_2026-02-26.md new file mode 100644 index 0000000..b36b71c --- /dev/null +++ b/docs/.agent/run_report_2026-02-26.md @@ -0,0 +1,36 @@ +# Run Report: 2026-02-26 + +**Agent:** Documentation & Cookbook Maintainer +**Target Version:** v0.1.335 +**Status:** Success + +## 📝 Summary +Performed maintenance on the Cookbook and Learning Path. No new code version was detected, but several improvements were made to existing documentation based on codebase analysis. + +## 🔍 Codebase Analysis +- **File Uploads**: Confirmed that `rustapi-core`'s `Multipart` extractor buffers the entire request body. Corrected documentation that incorrectly suggested streaming capabilities and fixed API usage for configuring body limits. +- **Structured Logging**: Identified `rustapi-extras` support for structured logging (JSON, Datadog, Splunk) and added a dedicated recipe. + +## 🛠️ Changes + +### 📚 Cookbook & Learning Path +1. **Fixed Recipe: File Uploads** (`docs/cookbook/src/recipes/file_uploads.md`) + - Corrected `RustApi` builder usage (replaced `.layer()` with `.body_limit()`). + - Removed incorrect references to `DefaultBodyLimit` as a type. + - Clarified memory buffering behavior. + +2. **New Recipe: Structured Logging** (`docs/cookbook/src/recipes/structured_logging.md`) + - Added comprehensive guide for `StructuredLoggingLayer`. + - Included configuration examples for Development, Production (JSON), and Datadog. + +3. **Learning Path Updates** (`docs/cookbook/src/learning/curriculum.md`) + - **Module 6.5**: Removed streaming pitfalls, added body limit configuration. + - **Module 9**: Added checks for blocking synchronous tasks in WebSockets. + - **Phase 4 Capstone**: Aligned job storage requirements. + +4. **Navigation** (`docs/cookbook/src/SUMMARY.md`) + - Added "Structured Logging" to the recipe list. + +## 🚀 Next Steps +- Consider adding a "Streaming Uploads" feature to `rustapi-core` in the future to handle large files efficiently. +- Expand "Observability" section with more on OpenTelemetry integration. diff --git a/docs/cookbook/src/SUMMARY.md b/docs/cookbook/src/SUMMARY.md index 12c2678..0e72524 100644 --- a/docs/cookbook/src/SUMMARY.md +++ b/docs/cookbook/src/SUMMARY.md @@ -46,6 +46,7 @@ - [Production Tuning](recipes/high_performance.md) - [Response Compression](recipes/compression.md) - [Resilience Patterns](recipes/resilience.md) + - [Structured Logging](recipes/structured_logging.md) - [Audit Logging](recipes/audit_logging.md) - [Time-Travel Debugging (Replay)](recipes/replay.md) - [Deployment](recipes/deployment.md) diff --git a/docs/cookbook/src/learning/curriculum.md b/docs/cookbook/src/learning/curriculum.md index 9cae23d..8de9263 100644 --- a/docs/cookbook/src/learning/curriculum.md +++ b/docs/cookbook/src/learning/curriculum.md @@ -131,12 +131,12 @@ Create a `POST /register` endpoint that accepts a JSON body `{"username": "...", - **Reading:** [File Uploads](../recipes/file_uploads.md). - **Task:** Create an endpoint `POST /upload` that accepts a file and saves it to disk. - **Expected Output:** `curl -F file=@image.png` uploads the file. -- **Pitfalls:** Loading large files entirely into memory (use streaming). +- **Pitfalls:** Multipart uploads are buffered into memory by default. Increase the body limit for large files but be aware of memory usage. #### 🧠 Knowledge Check 1. Which extractor is used for file uploads? -2. Why should you use `field.chunk()` instead of `field.bytes()`? -3. How do you increase the request body size limit? +2. Why is it important to set a reasonable `body_limit`? +3. What happens if you try to upload a file larger than the configured limit? ### 🏆 Phase 2 Capstone: "The Secure Blog Engine" **Objective:** Enhance the Todo API into a Blog Engine. @@ -191,7 +191,7 @@ Create a `POST /register` endpoint that accepts a JSON body `{"username": "...", #### 🧠 Knowledge Check 1. How do you upgrade an HTTP request to a WebSocket connection? -2. Can you share state between HTTP handlers and WebSocket handlers? +2. Why must you avoid long-running synchronous operations in the message loop? 3. What happens if a WebSocket handler panics? ### Module 10: Production Readiness & Deployment @@ -287,7 +287,7 @@ Create a `POST /register` endpoint that accepts a JSON body `{"username": "...", **Requirements:** - **Ingestion:** HTTP/3 endpoint receiving JSON events. - **Processing:** Push events to a `rustapi-jobs` queue (Redis backend). -- **Storage:** Workers process events and store aggregates in a database. +- **Storage:** Workers process events (simulate archive or aggregation) and store results. - **Observability:** Full tracing from ingestion to storage. - **Audit:** Log all configuration changes to the system. - **Resilience:** Circuit breakers on database writes. diff --git a/docs/cookbook/src/recipes/file_uploads.md b/docs/cookbook/src/recipes/file_uploads.md index bb5d67c..f7ededa 100644 --- a/docs/cookbook/src/recipes/file_uploads.md +++ b/docs/cookbook/src/recipes/file_uploads.md @@ -19,7 +19,7 @@ RustAPI's `Multipart` extractor currently buffers the entire request body into m ```rust use rustapi_rs::prelude::*; -use rustapi_rs::extract::{Multipart, DefaultBodyLimit}; +use rustapi_rs::extract::Multipart; use std::path::Path; #[tokio::main] @@ -30,13 +30,11 @@ async fn main() -> Result<(), Box> { println!("Starting Upload Server at http://127.0.0.1:8080"); RustApi::new() - // Increase body limit to 1GB (default is usually 1MB) - .body_limit(1024 * 1024 * 1024) - .route("/upload", post(upload_handler)) - // Increase body limit to 50MB (default is usually 2MB) + // Increase body limit to 50MB (default is usually 1MB) // ⚠️ IMPORTANT: Since Multipart buffers the whole body, // setting this too high can exhaust server memory. - .layer(DefaultBodyLimit::max(50 * 1024 * 1024)) + .body_limit(50 * 1024 * 1024) + .route("/upload", post(upload_handler)) .run("127.0.0.1:8080") .await } @@ -107,7 +105,7 @@ RustAPI loads the entire `multipart/form-data` body into memory. - **Mitigation**: Set a reasonable `DefaultBodyLimit` (e.g., 10MB - 100MB) to prevent DoS attacks. ### 2. Body Limits -The default request body limit is small (2MB) to prevent attacks. You **must** explicitly increase this limit for file upload routes using `.layer(DefaultBodyLimit::max(size_in_bytes))`. +The default request body limit is small (1MB) to prevent attacks. You **must** explicitly increase this limit for file upload routes using `.body_limit(size_in_bytes)`. ### 3. Security - **Path Traversal**: Malicious users can send filenames like `../../system32/cmd.exe`. Always rename files or sanitize filenames strictly. diff --git a/docs/cookbook/src/recipes/structured_logging.md b/docs/cookbook/src/recipes/structured_logging.md new file mode 100644 index 0000000..a9a36c9 --- /dev/null +++ b/docs/cookbook/src/recipes/structured_logging.md @@ -0,0 +1,115 @@ +# Structured Logging & Tracing + +RustAPI provides a powerful structured logging system via `rustapi-extras` that integrates with the `tracing` ecosystem. This allows you to output logs in various formats (JSON, Datadog, Splunk) with correlation IDs, request timing, and automatic field redaction. + +## Dependencies + +Ensure `rustapi-extras` with the `structured-logging` feature is enabled. + +```toml +[dependencies] +rustapi-rs = { version = "0.1.335", features = ["extras-structured-logging"] } +# OR +rustapi-extras = { version = "0.1.335", features = ["structured-logging"] } +``` + +## Basic Usage + +To enable structured logging, add the `StructuredLoggingLayer` to your application. + +```rust +use rustapi_rs::prelude::*; +use rustapi_rs::extras::structured_logging::{StructuredLoggingConfig, StructuredLoggingLayer}; + +#[tokio::main] +async fn main() -> Result<(), Box> { + // 1. Create configuration (default is JSON format) + let config = StructuredLoggingConfig::default(); + + // 2. Add the layer to your app + RustApi::new() + .layer(StructuredLoggingLayer::new(config)) + .route("/", get(handler)) + .run("127.0.0.1:8080") + .await +} + +async fn handler() -> &'static str { + // These logs will be formatted as JSON + tracing::info!("Handling request"); + "Hello World" +} +``` + +## Configuration Presets + +The configuration builder provides several presets for common environments. + +### Development (Pretty Printing) +Useful for local debugging with human-readable logs. + +```rust +let config = StructuredLoggingConfig::development(); +// Outputs: "2023-10-01T12:00:00Z INFO [rustapi] Request completed path=/ status=200 duration_ms=2" +``` + +### Production (JSON) +Optimized for log aggregators like ELK, CloudWatch, etc. + +```rust +let config = StructuredLoggingConfig::production_json(); +// Outputs: {"timestamp":"...","level":"INFO","message":"Request completed","path":"/","status":200,"duration_ms":2} +``` + +### Datadog +Formats logs specifically for Datadog ingestion. + +```rust +let config = StructuredLoggingConfig::datadog(); +``` + +## Advanced Configuration + +You can customize almost every aspect of the logging behavior using the builder. + +```rust +use rustapi_rs::extras::structured_logging::{StructuredLoggingConfig, LogOutputFormat}; + +let config = StructuredLoggingConfig::builder() + .format(LogOutputFormat::Json) + .service_name("payment-service") + .service_version("1.2.0") + .environment("production") + + // Include extra request details + .include_request_headers(true) + .include_caller_info(true) // File and line number + + // Redact sensitive headers + .redact_header("x-api-key") + .redact_header("authorization") + + // Add static fields to every log + .static_field("region", "us-east-1") + + // Correlation ID + .correlation_id_header("x-request-id") + .generate_correlation_id(true) + + .build(); +``` + +## Key Features + +### Correlation IDs +The layer automatically extracts a correlation ID from the request header (default `x-correlation-id`) or generates a new UUID if missing. This ID is attached to every log message generated within the request scope, allowing you to trace a request across microservices. + +### Field Redaction +Security is critical. The logger automatically redacts sensitive headers like `Authorization`, `Cookie`, `X-Api-Key` by default. You can add more headers to the redaction list via configuration. + +### Performance +The logging layer uses the `tracing` ecosystem which is designed for high performance. JSON serialization is handled efficiently. However, enabling `include_request_body` or `include_response_body` can have a performance impact and should generally be avoided in production unless debugging. + +## Integration with OpenTelemetry + +Structured logging works well alongside OpenTelemetry (Otel). While structured logging handles the "logs" signal, Otel handles "traces" and "metrics". You can enable both `extras-structured-logging` and `extras-otel` features for a complete observability stack.