Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
130 commits
Select commit Hold shift + click to select a range
ba11040
s3: detect looping when using gcs and versions
ncw Nov 21, 2023
5fba502
http: enable methods used with WebDAV - fixes #7444
alensiljak Nov 22, 2023
a259226
Add Alen Šiljak to contributors
ncw Nov 24, 2023
251a8e3
hash: allow runtime configuration of supported hashes for testing
ncw Nov 23, 2023
5d5473c
random: speed up String function for generating larger blocks
ncw Nov 23, 2023
94ccc95
random: stop using deprecated rand.Seed in go1.20 and later
ncw Nov 24, 2023
d5d28a7
operations: fix overwrite of destination when multi-thread transfer f…
ncw Nov 22, 2023
c27977d
fstest: factor chunked copy tests from b2 and use them in s3 and oos
ncw Nov 24, 2023
fabeb8e
b2: fix server side chunked copy when file size was exactly --b2-copy…
ncw Nov 24, 2023
cc2a4c2
fstest: factor chunked streaming tests from b2 and use in all backends
ncw Nov 24, 2023
8f47b67
b2: fix streaming chunked files an exact multiple of chunk size
ncw Nov 24, 2023
d8855b2
serve s3: document multipart copy doesn't work #7454
ncw Nov 24, 2023
4eed3ae
s3: ensure we can set upload cutoff that we use for Rclone provider
ncw Nov 24, 2023
fd2322c
fs/fshttp: fix --contimeout being ignored
rkonfj Nov 24, 2023
36eb3cd
imagekit: Added ImageKit backend
ahnv Nov 24, 2023
a10abf9
Add 你知道未来吗 to contributors
ncw Nov 24, 2023
9e62a74
Add Abhinav Dhiman to contributors
ncw Nov 24, 2023
e7c002a
test_all: make integration test for serve s3
ncw Nov 24, 2023
aaa8973
serve s3: fix error handling for listing non-existent prefix - fixes …
ncw Nov 24, 2023
0244caf
serve s3: fix overwrite of files with 0 length file
ncw Nov 24, 2023
edb5ccd
smb: fix about size wrong by switching to github.com/cloudsoda/go-smb…
ncw Nov 20, 2023
b5857f0
smb: fix modtime of multithread uploads by setting PartialUploads
ncw Nov 25, 2023
74d5477
onedrive: add --onedrive-delta flag to enable ListR
ncw Nov 26, 2023
82b963e
Version v1.65.0
ncw Nov 26, 2023
85f142a
Start v1.66.0-DEV development
ncw Nov 26, 2023
96f8b7c
install.sh: fix harmless error message on install
ncw Nov 28, 2023
9bfbf2a
mount: fix macOS not noticing errors with --daemon
ncw Nov 28, 2023
4d4f3de
s3: add --s3-version-deleted to show delete markers in listings when …
ncw Nov 8, 2023
5833984
smb: fix shares not listed by updating go-smb2
halms Nov 29, 2023
9061e81
multipart copy create bucket if it doesn't exist.
msays2000 Nov 28, 2023
97d7945
Add halms to contributors
ncw Dec 1, 2023
caf5dd9
mount: notice daemon dying much quicker
ncw Nov 29, 2023
e3d0bff
docs: fix typo in docs.md
ben-ba Dec 1, 2023
08c460d
Add ben-ba to contributors
ncw Dec 2, 2023
f0c7741
onedrive: fix error listing: unknown object type <nil>
ncw Nov 28, 2023
298c13e
systemd: Fix detection and switch to the coreos package everywhere
darthShadow Nov 29, 2023
aee787d
serve nfs: Mark as experimental
ncw Nov 27, 2023
1ebbc74
nfsmount: compile for all unix oses, add --sudo and fix error/option …
ncw Nov 27, 2023
c3117d9
Doc change: Add the CreateBucket permission requirement for AWS S3
e-orz Dec 7, 2023
2f5685b
build(deps): bump actions/setup-go from 4 to 5
dependabot[bot] Dec 6, 2023
9fe343b
s3: S3 IPv6 support with option "use_dual_stack" (bool)
tonymet Dec 3, 2023
4751980
docs: update contributor email
emyarod Dec 7, 2023
8e21c77
Add Eli Orzitzer to contributors
ncw Dec 8, 2023
57ab4d2
Add Anthony Metzidis to contributors
ncw Dec 8, 2023
113b2b6
Add emyarod to contributors
ncw Dec 8, 2023
ef0f302
vfs: note that --vfs-refresh runs in the background #6830
ncw Dec 6, 2023
f45cee8
dropbox: fix used space on dropbox team accounts
ncw Dec 8, 2023
110d075
docs: fix broken link
keongalvin Dec 7, 2023
6c58e99
oauthutil: add Shutdown method
rkonfj Nov 30, 2023
3f159ba
backend: fs implements the `Shutdowner` interface
rkonfj Dec 8, 2023
242fe96
Add keongalvin to contributors
ncw Dec 10, 2023
f98e672
selfupdate: fix crash in tests if beta not found
ncw Dec 9, 2023
c69eb84
chunker,compress,crypt,hasher,union: fix rclone move a file over itse…
ncw Dec 8, 2023
743ea6a
oracle object storage: fix object storage endpoint for custom endpoints
msays2000 Dec 15, 2023
8503282
azure-files: fix storage base url
oks-maytech Dec 18, 2023
fba2d4c
docs: fix broken link in serve webdav
rarspace01 Dec 30, 2023
3bf8c87
docs/librclone: the newer and recommended ucrt64 subsystem of msys2 c…
albertony Jan 1, 2024
3ca766b
hasher: fix invalid memory address error when MaxAge == 0
nielash Dec 21, 2023
394195c
Add rarspace01 to contributors
ncw Dec 31, 2023
a3d1994
googlephotos: fix nil pointer exception when batch failed
ncw Dec 27, 2023
bb679a9
backend: add description field for all backends
Paul-Stern Dec 2, 2023
d977fa2
ftp: fix multi-thread copy
WeidiDeng Jan 3, 2024
451d7ba
oauthutil: avoid panic when `*token` and `*ts.token` are the same
rkonfj Jan 3, 2024
64df4cf
build(deps): bump golang.org/x/crypto to fix ssh terrapin CVE-2023-48795
dependabot[bot] Jan 3, 2024
7aa066c
Add Paul Stern to contributors
ncw Jan 3, 2024
208e49c
fs: update use of math/rand to modern practice
ncw Dec 1, 2023
578b9df
build: fix docker build on arm/v6
ncw Jan 3, 2024
0e746f2
amazonclouddrive: remove Amazon Drive backend code and docs #7539
ncw Jan 3, 2024
5fa13e3
protondrive: fix CVE-2023-45286 / GHSA-xwh9-gc39-5298
ncw Jan 3, 2024
486a10b
serve s3: fix listing oddities
ncw Dec 5, 2023
c16c22d
s3: fix crash if no UploadId in multipart upload
ncw Dec 16, 2023
1f6271f
s3: copy parts in parallel when doing chunked server side copy
ncw Dec 1, 2023
dedad9f
onedrive: fix "unauthenticated: Unauthenticated" errors when uploading
ncw Jan 5, 2024
d392f9f
accounting: fix stats to show server side transfers
ncw Jan 4, 2024
fbdf71a
operations: fix files moved by rclone move not being counted as trans…
ncw Jan 4, 2024
41b8935
docs: Fix broken test_proxy.py link again
vdm Jan 8, 2024
e20f2ee
Changelog updates from Version v1.65.1
ncw Jan 8, 2024
0563cc6
build(deps): bump github.com/cloudflare/circl from 1.3.6 to 1.3.7
dependabot[bot] Jan 8, 2024
1045f54
oracleobjectstorage: Support "backend restore" command - fixes #7371
nikhilahuja Dec 19, 2023
3df6518
Add Nikhil Ahuja to contributors
ncw Jan 13, 2024
519fe98
azureblob: implement --azureblob-delete-snapshots
ncw Jan 12, 2024
184459b
vfs: fix stale data when using --vfs-cache-mode full
ncw Jan 14, 2024
a5972fe
docs: update website footer
ncw Jan 11, 2024
f8c5695
docs: add warp.dev as a sponsor
ncw Jan 11, 2024
cacfc10
docs: add warp.dev sponsorship to github home page
ncw Jan 11, 2024
1b1e430
build: update direct dependencies and fix serve nfs
ncw Jan 13, 2024
43cc243
build: update indirect dependencies where possible
ncw Jan 13, 2024
13fb2fb
build: update to go1.22rc1 and make go1.20 the minimum required version
ncw Jan 13, 2024
938b43c
build: remove random.Seed since random generator is seeded automatica…
ncw Jan 13, 2024
da244a3
ssh: shorten wait delay for external ssh binaries now that we are usi…
ncw Jan 13, 2024
dd0e5b9
operations: use built in io.OffsetWriter for go1.20
ncw Jan 13, 2024
223d8c5
serve dlna: now only supported on go1.21 or later
ncw Jan 13, 2024
42cac4c
build: use API when fetching golangci-lint as it is more reliable
ncw Jan 15, 2024
6521394
imagekit: Updated docs and web content
hrhv Jan 16, 2024
d20f647
Add Harshit Budhraja to contributors
ncw Jan 17, 2024
ae3c73f
stats: fix race between ResetCounters and stopAverageLoop called from…
ncw Jan 16, 2024
78176d3
imagekit: updated overview - supported operations
hrhv Jan 17, 2024
17fea90
docs: add rclone OS requirements
kapitainsky Jan 17, 2024
c482624
config: add config/paths to the rc as rclone config paths equivalent
ncw Jan 17, 2024
806f6ab
add missing backtick
teraa Jan 18, 2024
b06935a
build(deps): bump actions/cache from 3 to 4
dependabot[bot] Jan 17, 2024
6692941
lsf: add --time-format flag
nielash Dec 8, 2023
9933d6c
check: respect --no-unicode-normalization and --ignore-case-sync for …
nielash Sep 21, 2023
5c7ba0b
bisync: fix tests on macOS
nielash Oct 1, 2023
0e5f121
bisync: merge copies and deletes, support --track-renames and --backu…
nielash Oct 1, 2023
932f9ec
bisync: document support for atomic uploads
nielash Nov 8, 2023
c0968a0
operations: add logger to log list of sync results -- fixes #7282
nielash Oct 1, 2023
3a50f35
sync: report list of synced paths to file -- see #7282
nielash Nov 6, 2023
978cbf9
bisync: generate final listing from sync results, not relisting -- fi…
nielash Oct 1, 2023
079763f
bisync: isDir check for deltas
nielash Oct 6, 2023
6d6dc00
bisync: rollback listing on error
nielash Oct 6, 2023
0cac5d6
bisync: introduce terminal colors
nielash Nov 6, 2023
fd95511
bisync: generate listings concurrently with march -- fixes #7332
nielash Oct 7, 2023
88e516a
moveOrCopyFile: avoid panic on --dry-run
nielash Oct 10, 2023
11afc3d
sync: --fix-case flag to rename case insensitive dest - fixes #4854
nielash Oct 9, 2023
f7f4651
bisync: handle unicode and case normalization consistently - mostly-f…
nielash Oct 9, 2023
9c96c13
bisync: optimize --resync performance -- partially addresses #5681
nielash Nov 6, 2023
58fd6d7
docs: add bisync to index
nielash Nov 8, 2023
98f539d
bisync: refactor normalization code, fix deltas - fixes #7270
nielash Nov 9, 2023
44637dc
bisync: high-level retries if --resilient
nielash Nov 11, 2023
4d5d6ee
bisync: provide more info in critical error msgs
nielash Nov 11, 2023
9cf7836
bisync: support files with unknown length, including Google Docs - fi…
nielash Aug 24, 2023
bbf9b1b
bisync: support two --backup-dir paths on different remotes
nielash Nov 12, 2023
7f854ac
local: fix cleanRootPath on Windows after go1.21.4 stdlib update
nielash Nov 15, 2023
422b037
bisync: fallback to cryptcheck or --download when can't check hash
nielash Nov 21, 2023
7c6f0cc
operations: fix renaming a file on macOS
nielash Nov 20, 2023
5762462
bisync: account for differences in backend features on integration te…
nielash Nov 17, 2023
16f7591
bisync: update version number in docs
nielash Dec 6, 2023
de27759
bisync: normalize session name to non-canonical - fixes #7423
nielash Dec 1, 2023
7b78acf
bisync: document beta status more clearly - fixes #6082
nielash Dec 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
36 changes: 18 additions & 18 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,12 @@ jobs:
strategy:
fail-fast: false
matrix:
job_name: ['linux', 'linux_386', 'mac_amd64', 'mac_arm64', 'windows', 'other_os', 'go1.19', 'go1.20']
job_name: ['linux', 'linux_386', 'mac_amd64', 'mac_arm64', 'windows', 'other_os', 'go1.20', 'go1.21']

include:
- job_name: linux
os: ubuntu-latest
go: '1.21'
go: '>=1.22.0-rc.1'
gotags: cmount
build_flags: '-include "^linux/"'
check: true
Expand All @@ -43,14 +43,14 @@ jobs:

- job_name: linux_386
os: ubuntu-latest
go: '1.21'
go: '>=1.22.0-rc.1'
goarch: 386
gotags: cmount
quicktest: true

- job_name: mac_amd64
os: macos-11
go: '1.21'
go: '>=1.22.0-rc.1'
gotags: 'cmount'
build_flags: '-include "^darwin/amd64" -cgo'
quicktest: true
Expand All @@ -59,14 +59,14 @@ jobs:

- job_name: mac_arm64
os: macos-11
go: '1.21'
go: '>=1.22.0-rc.1'
gotags: 'cmount'
build_flags: '-include "^darwin/arm64" -cgo -macos-arch arm64 -cgo-cflags=-I/usr/local/include -cgo-ldflags=-L/usr/local/lib'
deploy: true

- job_name: windows
os: windows-latest
go: '1.21'
go: '>=1.22.0-rc.1'
gotags: cmount
cgo: '0'
build_flags: '-include "^windows/"'
Expand All @@ -76,20 +76,20 @@ jobs:

- job_name: other_os
os: ubuntu-latest
go: '1.21'
go: '>=1.22.0-rc.1'
build_flags: '-exclude "^(windows/|darwin/|linux/)"'
compile_all: true
deploy: true

- job_name: go1.19
- job_name: go1.20
os: ubuntu-latest
go: '1.19'
go: '1.20'
quicktest: true
racequicktest: true

- job_name: go1.20
- job_name: go1.21
os: ubuntu-latest
go: '1.20'
go: '1.21'
quicktest: true
racequicktest: true

Expand All @@ -104,7 +104,7 @@ jobs:
fetch-depth: 0

- name: Install Go
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: ${{ matrix.go }}
check-latest: true
Expand Down Expand Up @@ -168,7 +168,7 @@ jobs:
env

- name: Go module cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
Expand Down Expand Up @@ -241,9 +241,9 @@ jobs:

# Run govulncheck on the latest go version, the one we build binaries with
- name: Install Go
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: '1.21'
go-version: '>=1.22.0-rc.1'
check-latest: true

- name: Install govulncheck
Expand All @@ -266,12 +266,12 @@ jobs:

# Upgrade together with NDK version
- name: Set up Go
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: '1.21'
go-version: '>=1.22.0-rc.1'

- name: Go module cache
uses: actions/cache@v3
uses: actions/cache@v4
with:
path: ~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
Expand Down
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
FROM golang AS builder
FROM golang:alpine AS builder

COPY . /go/src/github.com/rclone/rclone/
WORKDIR /go/src/github.com/rclone/rclone/

RUN apk add --no-cache make bash gawk git
RUN \
CGO_ENABLED=0 \
make
Expand Down
24,614 changes: 12,477 additions & 12,137 deletions MANUAL.html

Large diffs are not rendered by default.

43,905 changes: 22,884 additions & 21,021 deletions MANUAL.md

Large diffs are not rendered by default.

41,988 changes: 21,909 additions & 20,079 deletions MANUAL.txt

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ check: rclone

# Get the build dependencies
build_dep:
go run bin/get-github-release.go -extract golangci-lint golangci/golangci-lint 'golangci-lint-.*\.tar\.gz'
go run bin/get-github-release.go -use-api -extract golangci-lint golangci/golangci-lint 'golangci-lint-.*\.tar\.gz'

# Get the release dependencies we only install on linux
release_dep_linux:
Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
[<img src="https://rclone.org/img/logo_on_light__horizontal_color.svg" width="50%" alt="rclone logo">](https://rclone.org/#gh-light-mode-only)
[<img src="https://rclone.org/img/logo_on_dark__horizontal_color.svg" width="50%" alt="rclone logo">](https://rclone.org/#gh-dark-mode-only)
[<img src="https://rclone.org/img/logos/warp-github-light.svg" title="Visit warp.dev to learn more." align="right">](https://www.warp.dev/?utm_source=github&utm_medium=referral&utm_campaign=rclone_20231103#gh-light-mode-only)
[<img src="https://rclone.org/img/logos/warp-github-dark.svg" title="Visit warp.dev to learn more." align="right">](https://www.warp.dev/?utm_source=github&utm_medium=referral&utm_campaign=rclone_20231103#gh-dark-mode-only)

[Website](https://rclone.org) |
[Documentation](https://rclone.org/docs/) |
Expand All @@ -23,7 +25,6 @@ Rclone *("rsync for cloud storage")* is a command-line program to sync files and
* 1Fichier [:page_facing_up:](https://rclone.org/fichier/)
* Akamai Netstorage [:page_facing_up:](https://rclone.org/netstorage/)
* Alibaba Cloud (Aliyun) Object Storage System (OSS) [:page_facing_up:](https://rclone.org/s3/#alibaba-oss)
* Amazon Drive [:page_facing_up:](https://rclone.org/amazonclouddrive/) ([See note](https://rclone.org/amazonclouddrive/#status))
* Amazon S3 [:page_facing_up:](https://rclone.org/s3/)
* ArvanCloud Object Storage (AOS) [:page_facing_up:](https://rclone.org/s3/#arvan-cloud-object-storage-aos)
* Backblaze B2 [:page_facing_up:](https://rclone.org/b2/)
Expand All @@ -46,6 +47,7 @@ Rclone *("rsync for cloud storage")* is a command-line program to sync files and
* HiDrive [:page_facing_up:](https://rclone.org/hidrive/)
* HTTP [:page_facing_up:](https://rclone.org/http/)
* Huawei Cloud Object Storage Service(OBS) [:page_facing_up:](https://rclone.org/s3/#huawei-obs)
* ImageKit [:page_facing_up:](https://rclone.org/imagekit/)
* Internet Archive [:page_facing_up:](https://rclone.org/internetarchive/)
* Jottacloud [:page_facing_up:](https://rclone.org/jottacloud/)
* IBM COS S3 [:page_facing_up:](https://rclone.org/s3/#ibm-cos-s3)
Expand Down Expand Up @@ -120,6 +122,7 @@ These backends adapt or modify other storage providers
* Partial syncs supported on a whole file basis
* [Copy](https://rclone.org/commands/rclone_copy/) mode to just copy new/changed files
* [Sync](https://rclone.org/commands/rclone_sync/) (one way) mode to make a directory identical
* [Bisync](https://rclone.org/bisync/) (two way) to keep two directories in sync bidirectionally
* [Check](https://rclone.org/commands/rclone_check/) mode to check for file hash equality
* Can sync to and from network, e.g. two different cloud accounts
* Optional large file chunking ([Chunker](https://rclone.org/chunker/))
Expand Down
31 changes: 10 additions & 21 deletions RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,32 +124,21 @@ Cherry pick any changes back to master and the stable branch if it is active.

## Making a manual build of docker

The rclone docker image should autobuild on via GitHub actions. If it doesn't
or needs to be updated then rebuild like this.
To do a basic build of rclone's docker image to debug builds locally:

See: https://github.com/ilteoood/docker_buildx/issues/19
See: https://github.com/ilteoood/docker_buildx/blob/master/scripts/install_buildx.sh
```
docker buildx build --load -t rclone/rclone:testing --progress=plain .
docker run --rm rclone/rclone:testing version
```

To test the multipatform build

```
git co v1.54.1
docker pull golang
export DOCKER_CLI_EXPERIMENTAL=enabled
docker buildx create --name actions_builder --use
docker run --rm --privileged docker/binfmt:820fdd95a9972a5308930a2bdfb8573dd4447ad3
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
SUPPORTED_PLATFORMS=$(docker buildx inspect --bootstrap | grep 'Platforms:*.*' | cut -d : -f2,3)
echo "Supported platforms: $SUPPORTED_PLATFORMS"
docker buildx build --platform linux/amd64,linux/386,linux/arm64,linux/arm/v7 -t rclone/rclone:1.54.1 -t rclone/rclone:1.54 -t rclone/rclone:1 -t rclone/rclone:latest --push .
docker buildx stop actions_builder
docker buildx build -t rclone/rclone:testing --progress=plain --platform linux/amd64,linux/386,linux/arm64,linux/arm/v7,linux/arm/v6 .
```

### Old build for linux/amd64 only
To make a full build then set the tags correctly and add `--push`

```
docker pull golang
docker build --rm --ulimit memlock=67108864 -t rclone/rclone:1.52.0 -t rclone/rclone:1.52 -t rclone/rclone:1 -t rclone/rclone:latest .
docker push rclone/rclone:1.52.0
docker push rclone/rclone:1.52
docker push rclone/rclone:1
docker push rclone/rclone:latest
docker buildx build --platform linux/amd64,linux/386,linux/arm64,linux/arm/v7 -t rclone/rclone:1.54.1 -t rclone/rclone:1.54 -t rclone/rclone:1 -t rclone/rclone:latest --push .
```
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
v1.65.0
v1.66.0
2 changes: 1 addition & 1 deletion backend/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ package all
import (
// Active file systems
_ "github.com/rclone/rclone/backend/alias"
_ "github.com/rclone/rclone/backend/amazonclouddrive"
_ "github.com/rclone/rclone/backend/azureblob"
_ "github.com/rclone/rclone/backend/azurefiles"
_ "github.com/rclone/rclone/backend/b2"
Expand All @@ -25,6 +24,7 @@ import (
_ "github.com/rclone/rclone/backend/hdfs"
_ "github.com/rclone/rclone/backend/hidrive"
_ "github.com/rclone/rclone/backend/http"
_ "github.com/rclone/rclone/backend/imagekit"
_ "github.com/rclone/rclone/backend/internetarchive"
_ "github.com/rclone/rclone/backend/jottacloud"
_ "github.com/rclone/rclone/backend/koofr"
Expand Down
Loading