Skip to content

Feature: implement S3-compatible backend (AWS S3, Backblaze B2, Cloudflare R2) #8

Description

@piranhap

Summary

src/backend/s3.rs is currently a stub returning NotImplemented. An S3-compatible backend would cover AWS S3, Backblaze B2, Cloudflare R2, and MinIO — making opendotsync viable without rclone or GitHub for users who already have cloud storage.

Config

backend:
  type: s3
  bucket: my-dotsync-bucket
  region: us-east-1           # optional, defaults to us-east-1
  prefix: opendotsync/        # optional path prefix within bucket
  endpoint: https://...       # optional, for R2/B2/MinIO custom endpoints

Credentials via standard AWS env vars: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, or ~/.aws/credentials.

Implementation Notes

  • Add aws-sdk-s3 behind the existing backend-s3 feature flag in Cargo.toml
  • Add prefix and endpoint fields to BackendConfig::S3 in src/config.rs
  • Object layout mirrors the local backend: <prefix>/<profile>/manifest.json, <prefix>/<profile>/v001.tar.zst
  • push: PutObject with the snapshot bytes
  • pull: GetObject
  • prune: DeleteObject for each old version
  • load_manifest / save_manifest: GetObject / PutObject on manifest.json; return default manifest on NoSuchKey
  • Use aws_sdk_s3::Client::from_env() for zero-config credential resolution
  • Wire in create_backend() and cmd_init() in src/sync.rs

Affected Files

  • Cargo.toml — add aws-sdk-s3, aws-config, tokio behind backend-s3 feature
  • src/config.rs — add prefix, endpoint fields to BackendConfig::S3
  • src/backend/s3.rs — full implementation
  • src/sync.rs — wire S3Backend in create_backend() and cmd_init()

Note on Async

aws-sdk-s3 is async. StorageBackend is currently sync. Options:

  1. Add a tokio runtime and block inside the S3 backend methods (tokio::runtime::Handle::current().block_on(...))
  2. Make StorageBackend async (larger refactor, better long-term)

Option 1 is recommended for now to avoid touching the trait and all existing backends.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions