Skip to content

PBM-1720 Improve documentation and implementation for S3 logging #355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
radoslawszulgo merged 7 commits into from
Apr 8, 2026
Merged

PBM-1720 Improve documentation and implementation for S3 logging #355

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
6354c90
PBM-1720 Improve documentation and implementation for S3 storage debu…
radoslawszulgo Apr 8, 2026
a42e404
Update docs/reference/configuration-options.md
radoslawszulgo Apr 8, 2026
7a9da10
Update docs/reference/configuration-options.md
radoslawszulgo Apr 8, 2026
b37c565
Update docs/reference/configuration-options.md
radoslawszulgo Apr 8, 2026
12d9538
Update s3-storage file to consistency and cross-reference.
radoslawszulgo Apr 8, 2026
3a88a00
Apply suggestions from code review
radoslawszulgo Apr 8, 2026
99323df
fix whitespaces
radoslawszulgo Apr 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion docs/details/s3-storage.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,10 @@ serverSideEncryption:

You can enable debug logging for different types of S3 requests in Percona Backup for MongoDB. Percona Backup for MongoDB prints S3 log messages in the `pbm logs` output so that you can debug and diagnose S3 request issues or failures.

To enable S3 debug logging, set the `storage.s3.DebugLogLevel` option in Percona Backup for MongoDB configuration. The supported values are: `LogDebug`, `Signing`, `HTTPBody`, `RequestRetries`, `RequestErrors`, `EventStreamBody`.
To enable S3 debug logging, set the `storage.s3.DebugLogLevel` option in Percona Backup for MongoDB configuration. The supported values are: `Signing`, `Retries`, `Request`, `RequestWithBody`, `Response`, `ResponseWithBody`, `DeprecatedUsage`, `RequestEventMessage`. Additionally, for backward compatibility with PBM versions older than 2.10.0, deprecated values are: `LogDebug`, `HTTPBody`, `RequestRetries`, `RequestErrors`, `EventStreamBody`.

See [AWS S3 storage options](../reference/configuration-options.md#storage-s3-debugloglevel) for details and an example.


### Storage classes

Expand Down
46 changes: 33 additions & 13 deletions docs/reference/configuration-options.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,25 +155,45 @@
*Type*: string <br>
*Required*: NO

Enables S3 debug logging for different types of S3 requests. S3 log messages are printed in the `pbm logs` output.
Enables AWS S3 debug logging for different types of AWS S3 requests. AWS S3 log messages are printed in the `pbm logs` output. Possible values:

Starting with version 2.10.0, PBM uses AWS SDK v2. The AWS SDK v1 values are deprecated. They are kept for backward compatibility.
- `Signing` - logs the request signing process
- `Retries` - logs each retry attempt when a request fails with a retryable error, including the retry count

Check warning on line 161 in docs/reference/configuration-options.md

View workflow job for this annotation

GitHub Actions / vale

[vale] docs/reference/configuration-options.md#L161 

[Vale.Spelling] Did you really mean 'retryable'?
Raw output 
{"message": "[Vale.Spelling] Did you really mean 'retryable'?", "location": {"path": "docs/reference/configuration-options.md", "range": {"start": {"line": 161, "column": 67}}}, "severity": "WARNING"}
- `Request` - logs outgoing HTTP request metadata (method, URL, headers) without the body
- `RequestWithBody` - logs outgoing HTTP requests, including the full request body; may expose sensitive data such as object contents, credentials, tokens, or headers
- `Response` - logs incoming HTTP response metadata (status code, headers) without the body
- `ResponseWithBody` - logs incoming HTTP responses, including the full response body; may expose sensitive data such as object contents, credentials, tokens, or headers
- `DeprecatedUsage` - logs deprecated usage of AWS S3 endpoints
- `RequestEventMessage` - logs individual event stream messages written/sent from the client to the AWS S3 (request events)
- `ResponseEventMessage` - logs individual event stream messages read/received by the client from the AWS S3 (response events)

Please find the mapping table below:
!!! warning

| AWS SDK v1 value | AWS SDK v2 value |
|------------------|------------------|
| `LogDebug` | `Request` <br> `Response`|
| `Signing` | `Signing`|
| `HTTPBody` | `RequestWithBody` <br> `ResponseWithBody`|
| `RequestRetries` | `DebugWithRequestRetries`|
| `RequestErrors` | `DebugWithRequestErrors`|
| `EventStreamBody`| `RequestWithBody` <br> `ResponseWithBody`|
`RequestWithBody` and `ResponseWithBody` can log full HTTP payloads and related sensitive information. Enable them only temporarily for troubleshooting, and only in controlled environments where exposing object contents, credentials, tokens, or headers is acceptable.

Additionally, for backward compatibility with PBM versions older than 2.10.0, the following deprecated options are supported:

To specify several event types, separate them by comma. To lean more about the event types, see [the documentation :octicons-link-external-16:](https://pkg.go.dev/github.com/aws/aws-sdk-go@v1.40.7/aws#LogLevelType)
| Deprecated value | Automatically translated into |
|------------------|---------------------------------------|
| `LogDebug` | `Request`,`Response` |
| `HTTPBody` | `RequestWithBody`, `ResponseWithBody` |
| `RequestRetries` | `Retries` |
| `RequestErrors` | `Response` |
| `EventStreamBody`| `RequestWithBody`, `ResponseWithBody` |

When undefined, no S3 debug logging is performed.
To specify several event types, separate them by comma. When undefined, no S3 debug logging is performed.

Note, this setting may result in excessive logging. By default, log entries are stored in a capped collection in your database. To redirect logs into a separate file, see [Logging configuration definition](../manage/logpath.md#logging-configuration-options).

#### Example
Here's an example and recommended configuration when troubleshooting AWS S3 communication:

```yaml
storage:
type: s3
s3:
debugLogLevels: RequestWithBody, ResponseWithBody
```

### storage.s3.insecureSkipTLSVerify

Expand Down
Loading