feat: structured JSON logger with per-request id and access log#156
Open
Shawnaldinho wants to merge 1 commit into
Open
feat: structured JSON logger with per-request id and access log#156Shawnaldinho wants to merge 1 commit into
Shawnaldinho wants to merge 1 commit into
Conversation
Previously every backend log was a free-form console.log/error, so
there was no way to (a) correlate the streaming error you see in a
client with the server-side line that produced it, or (b) filter on
status / route / user_id from a log aggregator. The blog post about
mikeoss called out the absence of structured logging, request
correlation, and error context, so this PR puts the floor in.
Three pieces:
1. backend/src/lib/logger.ts — ~80-line zero-dep JSON line logger.
- logger.{debug,info,warn,error}({ fields }, msg) emits one JSON
object per call to stdout/stderr.
- level is controlled by LOG_LEVEL env var, defaulting to "info"
in production and "debug" elsewhere.
- logger.child({ fields }) returns a logger that prefixes every
event with the given fields — used by the request middleware to
bind request_id.
- errFields(err) helper extracts name/message/stack from an Error.
- Kept hand-written instead of pulling in pino because we only need
~30 lines today; the call sites won't change if it's later swapped.
2. backend/src/middleware/requestContext.ts — assigns a per-request id
(from inbound X-Request-Id if present, else a UUID), echoes it on
the response, parks a child logger on res.locals.log, and on
response close emits one access-log line with method/route/status/
elapsed_ms/user_id. Wired into index.ts before the rate limiters.
3. Demonstrate adoption: replace console.error in the two streaming
error paths (POST /chat, POST /projects/:projectId/chat) and the
generate-title handler. Each unhandled error log now ships with the
request_id so it correlates to the access log line. Left the
debug-style console.log calls in chatTools.ts / documents.ts alone
— they're a separate, broader cleanup.
The X-Request-Id header is echoed on every response so a curl user
or a frontend retry can grep their request out of the logs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Backend logs were all free-form `console.log`/`console.error`, so an error you see on the client can't be correlated to the line that produced it on the server, and there's no way to filter by route/status/user_id from a log aggregator. This PR puts the floor in: a tiny structured logger, a per-request id, and an access log line per response.
Changes
Why
Concrete things this enables that weren't possible before:
Addresses the observability concern from https://insights.flank.ai/where-mikeoss-falls-short.html (gap 14). Doesn't try to also tackle tracing / metrics / Sentry — those are reasonable follow-ups built on this foundation.
Testing