---
title: "Prevent Cross-Cloud Sync Data Loss with rclone on Linux"
description: "Use rclone to sync and migrate files between S3, Azure Blob, Google Drive, and local Linux storage with checksum verification and bandwidth limits."
canonical: "https://adamtheautomator.com/rclone-cross-cloud-sync/"
---

# Prevent Cross-Cloud Sync Data Loss with rclone on Linux

> Use rclone to sync and migrate files between S3, Azure Blob, Google Drive, and local Linux storage with checksum verification and bandwidth limits.

Source: https://adamtheautomator.com/rclone-cross-cloud-sync/

---

ATA Learning

Tap to hide

[

ATA Learning

](/)

*   [Home](/)
*   [Tutorials](/tutorials/)
*   [Instructors](/author/)
*   [Advertising](/advertising/)
*   [Recommended Resources](/resources/)
*   [About Adam](/about-adam/)

Search for:  

*   [](https://twitter.com/adbertram)
*   [](https://github.com/Adam-the-Automator)
*   [](https://www.linkedin.com/company/adam-the-automator-llc)
*   [](/feed/)

![Prevent Cross-Cloud Sync Data Loss with rclone on Linux](https://adamtheautomator.com/wp-content/uploads/publisher/3ec5d9c85b2b81f4bcbffee60ec8b04e/ec89f78e08ea2765d48f70a4cc5b4fc574b134b99d4b4c5e5a259cee6123bd84.jpg)

# Prevent Cross-Cloud Sync Data Loss with rclone on Linux

[![](https://secure.gravatar.com/avatar/d0b9d42e21e5622713f8b693aa5c0f9244d5f7dd200ed29b8398f52dee5de337?s=192&d=mm&r=g)Adam Bertram](https://adamtheautomator.com/author/adam-bertram/)11 November 202617 min. read

Categories: [Cloud](/category/cloud/)

Tags:[Linux](/tag/linux/)[AWS S3](/tag/aws-s3/)[Azure Storage Accounts](/tag/azure-storage-accounts/)

Table of Contents

*   [What rclone Does Between Two Clouds](#what-rclone-does-between-two-clouds)
*   [Where the Bytes Actually Travel](#where-the-bytes-actually-travel)
*   [Prerequisites and a Practice Harness You Can Delete](#prerequisites-and-a-practice-harness-you-can-delete)
*   [Stand Up a Local S3 Endpoint](#stand-up-a-local-s3-endpoint)
*   [Configure rclone for S3, Azure Blob, Drive, and Local](#configure-rclone-for-s3-azure-blob-drive-and-local)
*   [Credentials That Survive a Cron Job](#credentials-that-survive-a-cron-job)
*   [rclone copy, sync, and bisync: Which One Deletes](#rclone-copy-sync-and-bisync-which-one-deletes)
*   [Watch the Delete Happen](#watch-the-delete-happen)
*   [Run the First rclone Sync to Azure Blob, Dry First](#run-the-first-rclone-sync-to-azure-blob-dry-first)
*   [Read the Log Lines That Matter](#read-the-log-lines-that-matter)
*   [Prove the Transfer With rclone check](#prove-the-transfer-with-rclone-check)
*   [When the Hashes Disagree but the Files Match](#when-the-hashes-disagree-but-the-files-match)
*   [Tune rclone Sync Transfers for Your Object Sizes](#tune-rclone-sync-transfers-for-your-object-sizes)
*   [Small Files and Big Objects Need Different Knobs](#small-files-and-big-objects-need-different-knobs)
*   [Put It on a systemd Timer Without Overlaps](#put-it-on-a-systemd-timer-without-overlaps)
*   [Exit Codes Your Wrapper Has to Understand](#exit-codes-your-wrapper-has-to-understand)
*   [What the Move Actually Costs](#what-the-move-actually-costs)
*   [Recover When a Sync Dies Halfway](#recover-when-a-sync-dies-halfway)
*   [Deletes, Versioning, and the Trash](#deletes-versioning-and-the-trash)
*   [Troubleshoot the Errors You Will See](#troubleshoot-the-errors-you-will-see)
*   [Two Errors Worth Knowing](#two-errors-worth-knowing)
*   [What Prevents the Next Outage](#what-prevents-the-next-outage)

You pointed `rclone sync` at two buckets, went to lunch, and came back to find 1,800 destination files gone that had never existed on the source, so which part of that command decided they should disappear?

Whether those 1,800 destination-only files survive decides whether the migration finishes cleanly or empties a share. rclone gives you one binary, one config file, and one object model that treats [Amazon S3](https://aws.amazon.com/s3/), [Azure Blob Storage](https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blobs-introduction), [Google Drive](https://workspace.google.com/products/drive/), and a local directory as equal peers. Three of its verbs look almost identical on the command line and do very different things to the files you already have. This tutorial builds the four-remote setup on a localhost harness, then points the same commands at real remotes and verifies the transfer with checksums before it hands the job to a timer that cannot overlap itself.

## What rclone Does Between Two Clouds

`rclone sync` decided, because making the destination match the source is what that verb is for. rclone replaces the provider SDKs with a small, uniform set of names. A remote is a named backend plus a path, written `remote:path`, and every subcommand takes remotes rather than provider-specific clients. The [storage system overview](https://rclone.org/overview/) tabulates which backend supports which feature, and that table matters more than any flag list you will read.

Four commands cover the whole migration, once the remotes they name exist:

```bash
rclone lsd s3prod:                              # list buckets on a configured remote
rclone copy /srv/data azblob:backups/2026-11    # local disk to an Azure container
rclone copy :s3,provider=AWS,region=eu-west-1:archive /srv/restore   # no config file at all
rclone lsf localwork:/var/log                   # named local remote, created by the harness below
```

The `s3prod`, `azblob`, and `gdrive` blocks appear in the config file two sections down, and the harness in the next section creates the two local remotes. The `:backend:path` form builds a backend on the fly from connection-string parameters, which is how rclone fits inside a container image with no config volume. The [usage documentation](https://rclone.org/docs/) covers that syntax alongside the ordinary `remote:path` form.

### Where the Bytes Actually Travel

Where the bytes travel decides your throughput and your bill. rclone attempts a [server-side copy](https://rclone.org/docs/#server-side-copy) only when both paths belong to the same remote. An S3 bucket to an Azure container is a different remote, so every byte is downloaded by the machine running rclone and uploaded again. That host is now in the critical path for bandwidth, disk cache, and time.

![rclone sync architecture](https://adamtheautomator.com/wp-content/uploads/publisher/af06aff79d9d134f599e4397f28f017b3b532a1f02525754a39e1faecea777cf.jpg)

Plan the host accordingly. A cross-provider move needs roughly the dataset size in free scratch space for staging, a network path that can sustain both legs at once, and enough CPU to hash every object. Same-provider moves (bucket to bucket on one remote) skip all of that.

## Prerequisites and a Practice Harness You Can Delete

If you want to follow along hands-on, you’ll need:

*   A Linux host with rclone v1.60 or newer, because `bisync` behavior and conflict detection changed across releases. Install from the [official download page](https://rclone.org/downloads/) or your distribution package, then check `rclone version`.
    
*   Scratch space on that host at least as large as the largest single object, and ideally the full dataset, since cross-provider copies stream through the machine.
    
*   Cloud credentials only for the sections that touch real providers. The harness below uses the local S3 server built into rclone, so you can complete every verification step with no account.
    
*   `systemd` on the host if you want the timer section to apply directly.
    

The commands below were exercised against rclone v1.75.1 on macOS (Darwin 27.0, arm64), and every flag is documented in the [rclone command reference](https://rclone.org/docs/). The `systemd` unit and the `flock` wrapper are Linux-only, and the rclone flags inside them are the ones exercised here.

Build the harness in a throwaway directory. The first command creates a source tree with a few small objects and one file you will later use to prove that `sync` deletes:

```bash
mkdir -p /tmp/rdemo/{src,dst,s3root,cfg}
cd /tmp/rdemo
mkdir -p src/2026/nov src/archive
for i in 1 2 3 4 5; do dd if=/dev/urandom of=src/2026/nov/file$i.bin bs=1k count=64 status=none; done
echo "hello rclone" > src/notes.txt
echo "old" > src/archive/old.txt
```

`status=none` suppresses the per-file summary that `dd` prints by default, so the loop leaves you with five 64 KB files and a quiet terminal.

### Stand Up a Local S3 Endpoint

[`rclone serve s3`](https://rclone.org/commands/rclone_serve_s3/) speaks the S3 protocol against a local directory, which makes it a faithful stand-in for a real bucket. Start it in the background, pinned to localhost so nothing is exposed off the machine:

```bash
rclone serve s3 --auth-key ATAKEY,ATASECRET --addr 127.0.0.1:19000 /tmp/rdemo/s3root &
```

Then register it as a remote, exactly as you would with an AWS account, using `provider = Other` and the access key pair you just generated:

```bash
export RCLONE_CONFIG=/tmp/rdemo/cfg/rclone.conf
rclone config create s3local s3 provider Other \
  access_key_id ATAKEY secret_access_key ATASECRET \
  endpoint http://127.0.0.1:19000 force_path_style true
```

`rclone config create` writes that block into the config file named by `RCLONE_CONFIG`, so every later command in this shell resolves `s3local:` without another wizard run. Create the bucket and list it as a separate step, because creation and verification read differently when something goes wrong:

```bash
rclone mkdir s3local:archive
rclone lsd s3local:
```

`rclone mkdir` creates the bucket; `rclone lsd` lists it so you can confirm the name. `force_path_style` is required for most non-AWS S3 implementations, and the [S3 backend documentation](https://rclone.org/s3/) explains when to set it for the provider you actually use.

The delete demo and the verification step later name their paths as remotes, so add the two local remotes now, into the same config file:

```bash
rclone config create localwork local
rclone config create dst local
```

| Remote | Type | Where it points |
| --- | --- | --- |
| `s3local:` | S3 against the local endpoint | `/tmp/rdemo/s3root` |
| `localwork:` | local | any path, written `localwork:/tmp/rdemo/src` |
| `dst:` | local | any path, written `dst:/tmp/rdemo/dst` |

Every remote the rest of this post uses is defined in that table or in the config section that follows, so the harness is complete at this point. Everything from here on works unchanged against a real bucket.

## Configure rclone for S3, Azure Blob, Drive, and Local

`rclone config` is an interactive wizard that writes a single INI-style file, and `rclone config file` prints the path in use. On Linux that is `~/.config/rclone/rclone.conf`. The [config command documentation](https://rclone.org/commands/rclone_config/) covers the wizard, and the file it produces is plain enough to write by hand or generate from a secret store.

A four-remote config for the real providers looks like this:

```text
[s3prod]
type = s3
provider = AWS
region = eu-west-1
access_key_id = AKIA...
secret_access_key = ...

[azblob]
type = azureblob
account = ataarchive
sas_url = https://ataarchive.blob.core.windows.net/?sv=2024-11-04&ss=b&srt=sco&sp=rwdlac&se=2027-01-01T00:00:00Z&sig=...

[gdrive]
type = drive
scope = drive
service_account_file = /etc/rclone/gdrive-sa.json
team_drive = 0AJ...

[localwork]
type = local
```

Each block maps to one backend’s documentation: [S3](https://rclone.org/s3/), [the Blob backend](https://rclone.org/azureblob/), [the Drive backend](https://rclone.org/drive/), and the [local filesystem](https://rclone.org/local/). The `[localwork]` entry is the one the harness already wrote, and it stays in the file because a named local remote gives scripts an endpoint instead of a raw path they have to repeat.

Encrypt the config before it holds long-lived cloud credentials. `rclone config encryption set` wraps the file with a password, and the [configuration encryption section](https://rclone.org/docs/#configuration-encryption) is blunt about the tradeoff: the rclone docs state “There is no way to recover the configuration if you lose your password.” Store that password in your password manager and in your automation secret store, in two places, before you enable encryption on a production host.

* * *

_**Warning: Config encryption protects the file at rest. It does not protect a credential that leaks through `--dump headers`, a debug log, or a process listing. Keep `--log-level` at INFO for scheduled jobs and never print a config block into CI output.**_

* * *

### Credentials That Survive a Cron Job

Two authentication shapes cause most of the silent failures in unattended jobs.

*   [A SAS token](https://adamtheautomator.com/azure-sas-token/) in `sas_url` has an expiry baked into the URL. When it lapses, every command fails with an authorization error and nothing in the job’s output explains why. Track the `se` parameter and rotate the token before it expires.
    
*   An interactive Drive OAuth token depends on a refresh token that can be revoked or hit a testing-mode expiry. A service account with the folder shared to its address avoids the browser entirely, which is why the config above uses `service_account_file`.
    

The `scope = drive` line asks for access to everything the account can see, and `team_drive` then pins the remote to one shared drive by its ID. Drop `team_drive` and rclone reads the service account’s own drive, which is empty and looks like a permissions failure until you remember the field. The SAS token and the Drive OAuth token are each documented per backend, and neither one produces a helpful error at 3 a.m. by itself.

## rclone copy, sync, and bisync: Which One Deletes

The copy, sync, and bisync deletion table is the section to bookmark. Read the third column before you type the command.

| Command | What it writes | What it deletes | First-run requirement |
| --- | --- | --- | --- |
| `rclone copy` | New and changed files on the destination | Nothing | None |
| `rclone sync` | Makes the destination match the source | Files on the destination that are absent from the source | None, but always test with `--dry-run` |
| `rclone bisync` | Changes on both sides | Files on either side, based on the last stored listing pair | `--resync` on the first run and after any gap |

The [sync documentation](https://rclone.org/commands/rclone_sync/) states the contract plainly: “Sync the source to the destination, changing the destination only. Doesn’t transfer files that are identical on source and destination, testing by size and modification time or MD5SUM. Destination is updated to match source, including deleting files if necessary (except duplicate objects, see below). If you don’t want to delete files from destination, use the copy command instead.” The [copy documentation](https://rclone.org/commands/rclone_copy/) describes the same transfer without the deletion, and [bisync](https://rclone.org/bisync/) adds a bidirectional mode that keeps a pair of listings under `~/.cache/rclone/bisync/` and reconciles both sides against them.

### Watch the Delete Happen

Nothing teaches this faster than seeing one file vanish. The harness defines `localwork:` and `dst:`, so this demo names them directly. Copy the source into a destination, then add a file by hand and sync:

```bash
rclone copy localwork:/tmp/rdemo/src dst:/tmp/rdemo/dst
echo "handwritten" > /tmp/rdemo/dst/keepme.txt
rclone sync localwork:/tmp/rdemo/src dst:/tmp/rdemo/dst --dry-run -v
```

The dry run prints a line for each file it would delete, with `Skipped delete as --dry-run is set` next to the name. Run the same command without `--dry-run` and `keepme.txt` is gone, because it exists on the destination and not on the source. That is what happened to those 1,800 files, and it is correct by design once you have chosen `sync` on purpose.

## Run the First rclone Sync to Azure Blob, Dry First

Start every new pair with a rehearsal. The `--dry-run` flag performs the listing, the size and modtime comparison, and the transfer plan, then prints what it would do without writing anything. Add `-v` so the log shows individual file decisions instead of a summary.

```bash
rclone copy /srv/data azblob:backups/2026-11 \
  --dry-run -v --stats 10s
```

Read the tail of that output before you drop the flag. It reports the number of files listed, the number that differ, and the total size queued. A plan that lists three times more files than you expected usually means the source path is wrong, and the copy is about to reproduce a directory tree you did not intend.

When the plan looks right, run it for real with the parallelism turned up for object storage and a bandwidth ceiling that leaves room for the rest of the host’s traffic:

```bash
rclone copy /srv/data azblob:backups/2026-11 \
  --transfers 8 --checkers 16 \
  --bwlimit "08:00,512k 19:00,off" \
  --stats 60s --log-level INFO -P
```

`-P` draws a live progress bar with per-file counters, and `--stats 60s` sets how often the byte and file totals refresh in the log. `--bwlimit` accepts a timetable, so the job can crawl during business hours and run at full speed overnight. The [`--bwlimit` timetable specification](https://rclone.org/docs/#bwlimit-bwtimetable) documents the grammar in full, and it is worth reading once rather than guessing at the syntax.

### Read the Log Lines That Matter

The final stats block answers three questions. `Transferred` tells you the bytes that actually moved, which is zero on a second run over unchanged data. `Checks` tells you how many files rclone compared and decided to skip. `Elapsed time` gives you the number to divide into bytes when you want a real throughput figure. If `Transferred` shows a small number and `Checks` shows a large one, the job is doing its job.

## Prove the Transfer With rclone check

An `rclone sync` that exits 0 has moved what it planned, having compared only size and modification time; whether the bytes match needs `rclone check`. That command reads what both sides expose and reports the differences:

```bash
rclone check /srv/data azblob:backups/2026-11
```

The [check documentation](https://rclone.org/commands/rclone_check/) describes the behavior as comparing sizes and hashes and logging a report, without altering either side. On the harness, the same check runs against the two local remotes and finishes in under a second:

```bash
rclone check localwork:/tmp/rdemo/src dst:/tmp/rdemo/dst
```

A clean run says `0 differences found` and exits 0. Corrupt one destination file and the command exits 1 with `1 differences found`, which your wrapper script must treat as a hard failure.

![Hash comparison by backend](https://adamtheautomator.com/wp-content/uploads/publisher/14d45784013e8bd42b943178bf4a7ded676e92f5d3b9f98a08ea9236619e6a15.jpg)

### When the Hashes Disagree but the Files Match

Cross-provider comparisons fail for reasons that have nothing to do with corruption, and the [rclone S3 page](https://rclone.org/s3/) spells out the main one: for objects uploaded in multiple parts, the ETag header is no longer the MD5 sum of the data, so rclone stores its own digest in `X-Amz-Meta-Md5chksum` metadata (base64-encoded MD5) on upload. An object uploaded by another tool has no such metadata, and the comparison has nothing reliable to read.

Three practical responses, in order of preference:

*   Use `--one-way` when the source is authoritative and the destination legitimately holds extra files. It checks that every source file exists and matches, and stops complaining about destination-only objects.
    
*   Use `rclone check --download` to compare by reading the content back instead of trusting a hash header. It costs a full download of both sides and settles the question.
    
*   For encrypted remotes, use [`rclone cryptcheck`](https://rclone.org/commands/rclone_cryptcheck/) instead, since the stored object hashes do not correspond to the plaintext.
    

Google Docs files are the other common false alarm. They are not binary objects and have no MD5, so a Drive folder full of documents will report differences against an S3 copy that exported them.

## Tune rclone Sync Transfers for Your Object Sizes

The defaults are conservative because rclone cannot know what it is talking to. `--transfers` defaults to 4 and `--checkers` to 8, and both are per-process. Storage systems with high per-request latency reward more parallelism; a small VM or a rate-limited backend punishes it with throttling errors.

The rclone docs give a concrete starting point: “`--transfers` sets the number of files transferred in parallel. The default is 4. Double this until the transfer rate stops increasing or the storage system starts returning errors. Values of 32 to 64 are common and values in the hundreds work well with the large object storage systems (e.g. s3, azureblob, googlecloudstorage, b2) when transferring lots of small files”. That guidance is about small objects. A single 200 GB file will not go faster with 64 transfers, because there is only one of it.

### Small Files and Big Objects Need Different Knobs

The right settings depend on which side of the size distribution your dataset sits on:

*   Millions of small objects: raise `--transfers` and `--checkers` together, and add `--fast-list` if the listing fits in memory. The [flag documentation](https://rclone.org/flags/) warns that `--fast-list` uses more memory in exchange for fewer transactions, which is a good trade on a paid-request backend and a bad one on a small host.
    
*   A few huge objects: raise `--multi-thread-streams` (default 4) to split each download into parallel ranges, and leave `--transfers` low. The [S3 backend](https://rclone.org/s3/) exposes chunk size options that matter more than transfer count at this scale.
    
*   Mixed content: run two jobs with filters rather than one job with compromised settings. `--min-size 100M` on the large-object job and `--max-size 100M` on the small-object job is easier to reason about than a single tuning pass.
    

Measure before you keep a setting. Run the same 2,000-file test set at 4, 16 and 64 transfers and keep the number where throughput flattens. On the harness, the entire dataset is about 320 KB, so the run finishes before tuning has any effect and says nothing about production throughput.

## Put It on a systemd Timer Without Overlaps

A sync that runs every hour needs two guarantees: it starts on schedule, and a second copy never starts while the first is still running. `systemd` gives you the first, and `flock` gives you the second.

Write the worker as a small script so the unit file stays readable:

```bash
#!/usr/bin/env bash
set -euo pipefail
exec rclone sync /srv/data azblob:backups/2026-11 \
  --transfers 8 --checkers 16 \
  --max-delete 500 \
  --log-file /var/log/rclone/sync.log --log-level INFO \
  --stats 60s
```

`--max-delete 500` is the guard rail that matters most in production. The [bisync safety measures](https://rclone.org/bisync/) state the idea in one line: “Abort on excessive deletes - protects against a failed listing being interpreted as all the files were deleted.” The [sync command reference](https://rclone.org/commands/rclone_sync/) makes its own related promise: “Files in the destination won’t be deleted if there were any errors at any point.” If a botched filter or a partial listing suddenly plans 40,000 deletions, the job stops instead of executing them.

Wrap that script in a oneshot service so systemd owns the schedule and the lock:

```text
[Unit]
Description=Cross-cloud rclone sync
After=network-online.target
Wants=network-online.target

[Service]
Type=oneshot
ExecStart=/usr/bin/flock -n /run/rclone-sync.lock /usr/local/bin/rclone-sync.sh
TimeoutStartSec=6h
```

The `-n` flag makes `flock` exit immediately when the lock is held, so a slow run skips the next tick instead of stacking a second process on the same destination. `TimeoutStartSec=6h` is the kill switch on the service: a run that outlives it is terminated mid-transfer, so set it above your slowest expected run rather than to a round number.

Then the timer that fires the service, in its own unit file next to it:

```text
[Unit]
Description=Hourly cross-cloud rclone sync

[Timer]
OnCalendar=hourly
Persistent=true
Unit=rclone-sync.service

[Install]
WantedBy=timers.target
```

`OnCalendar=hourly` sets the schedule, `Unit=` names the service each tick starts, and `Persistent=true` makes a window missed during downtime run once on the next boot. Enable the pair with `systemctl enable --now rclone-sync.timer`, and check the next run time with `systemctl list-timers rclone-sync.timer`.

### Exit Codes Your Wrapper Has to Understand

The [rclone documentation](https://rclone.org/docs/#exit-code) defines a small, useful set of exit codes, and treating every non-zero value as “retry” will hide real problems:

*   `0` means the operation succeeded.
    
*   `1` means an error that fits no other category.
    
*   `2` means a syntax or usage error, which a retry will reproduce.
    
*   `5` means a temporary error, the kind more retries might fix, so it is worth one more run.
    
*   `9` means the operation succeeded but no files were transferred, and it only appears when you ask for it with `--error-on-no-transfer`.
    
*   `10` means the job hit `--max-duration` and stopped early.
    

Code 9 is the one people miss. Run `rclone copy` with `--error-on-no-transfer` on a source that is already in sync and the command exits 9, which lets a monitoring check treat “nothing changed” as a distinct, non-alarming outcome.

## What the Move Actually Costs

Every cross-provider transfer through a host pays for two legs: egress from the source provider and ingress to the destination. Ingress is free on S3 and Azure Blob in most regions; egress is billed per gigabyte. Check the current rate cards for [S3 data transfer](https://aws.amazon.com/s3/pricing/) and [Azure bandwidth](https://azure.microsoft.com/pricing/details/bandwidth/) before committing to a dataset size; the numbers change and the tiers are not linear.

Multiply the dataset size in gigabytes by the source’s per-gigabyte egress rate. A 2 TB move at an illustrative 0.09 USD per GB:

| Line item | Illustrative figure |
| --- | --- |
| Dataset size | 2 TB, or 2,048 GB |
| Source egress rate | 0.09 USD per GB |
| Egress subtotal | about 184 USD |
| Request charges | separate, billed per 1,000 requests on Amazon S3 and per 10,000 transactions on Azure Blob Storage |

Substitute the current rate for your region and tier; the calculation is unchanged.

Request charges land on the destination side for writes and on the source side for listings. Those charges are billed per batch of operations (per 1,000 requests on Amazon S3 and per 10,000 transactions on Azure Blob Storage), and small files generate far more requests per gigabyte than large ones. That is the hidden cost in the small-file tuning above, and why `--fast-list` pays for itself on a backend that charges for listings. Google Drive adds a separate constraint: the [Drive API quotas](https://developers.google.com/workspace/drive/api/guides/limits) cap requests per minute per user, so aggressive `--transfers` values produce `403` errors long before the invoice reflects anything.

## Recover When a Sync Dies Halfway

A transfer that stops at 60 percent is a normal event on a dataset measured in terabytes. Recovery is straightforward if you understand what rclone already wrote.

On restart, rclone re-lists both sides and compares size and modification time. Files that arrived completely are skipped, and the job continues from where it stopped. That means the correct recovery step is the same command you ran before, preceded by a dry run to confirm the remaining plan:

```bash
rclone sync /srv/data azblob:backups/2026-11 --dry-run -v | tail -20
```

Interrupted uploads can leave partial objects on backends that support partial uploads. `--partial-suffix` (default `.partial`) names the temporary file so an aborted upload is visible and cannot be mistaken for a complete object. Raise `--retries` for flaky links, and keep `--low-level-retries` at its default unless you have a specific transport problem to work around.

### Deletes, Versioning, and the Trash

Three backend behaviors change what a `sync` delete means in practice:

*   `sync` deletes after transferring by default (`--delete-after`), so a failed transfer never precedes a delete. `--delete-before` and `--delete-during` change that ordering and exist mainly for very large trees where the delete phase would otherwise be slow.
    
*   Google Drive sends deletes to the trash by default. `--drive-use-trash=false` makes them permanent, which is the setting to avoid on a first migration.
    
*   Bucket versioning on S3 and soft delete on Azure Blob keep deleted objects recoverable for a configured retention window. Turn one on before the first `sync` against a destination that matters, and a mistaken first run costs you a restore instead of the files.
    

`rclone delete` removes files without transferring anything, which makes it the tool for reclaiming space on a destination after a filter change. The [delete documentation](https://rclone.org/commands/rclone_delete/) covers the filter interaction, and the [filtering guide](https://rclone.org/filtering/) explains why mixing `--include`, `--exclude`, and `--filter` in one command produces surprising results.

## Troubleshoot the Errors You Will See

The strings below are the ones that appear often enough to memorize, in the order you are likely to meet them:

| Error you see | Usual cause | Fix |
| --- | --- | --- |
| `AuthenticationFailed ... Signature not valid in the specified time frame` | SAS token expired or clock skew | Regenerate `sas_url` with a later `se` value |
| `403 rateLimitExceeded` or `userRateLimitExceeded` | Drive API quota, too much parallelism | Lower `--transfers`, add `--tpslimit 8` |
| `301 Moved Permanently` or `PermanentRedirect` on S3 | Wrong `region` in the remote block | Set the bucket’s real region |
| `directory not found` with exit code 3 | Path typo or missing bucket | `rclone lsd remote:` to list what exists |
| `corrupted on transfer` | Byte mismatch during transfer | Re-run with `--retries 3`, then `rclone check --download` |

### Two Errors Worth Knowing

The Drive rate limit shows up under sustained load, so a job that ran fine for an hour can fail on minute 61. The [Drive backend documentation](https://rclone.org/drive/) covers the quota behavior and the pacing flags that keep a large transfer inside it. The S3 redirect error means the request reached the right service and the wrong region, so fix the `region` value in the remote block before you go looking at permissions; the [same S3 page](https://rclone.org/s3/) lists the value each provider expects.

### What Prevents the Next Outage

Log every scheduled run to a file and keep the last week of it. The [logging and exit code reference](https://rclone.org/docs/#exit-code) defines what each status means, and a wrapper that distinguishes “nothing changed” from “temporary error” from “syntax error” will page you for the right reasons. Route exit 1 and exit 2 to an alert, retry once on exit 5, and treat exit 9 as a normal no-change run. Send that verdict to a healthchecks.io ping, which also notices a run that never started, or to a `systemd` `OnFailure=` unit that fires a notifier.

* * *

_**Pro Tip: Keep a `--combined` report from your first production sync. It writes one line per file with a symbol for matched, missing, different, or errored, which turns an argument about what moved into a file you can diff.**_

* * *

Moving a dataset across providers this way requires no managed migration product. A single config file, three verbs you now know the difference between, a check that reads real hashes, and a timer with a lock will move the data and tell you, in the log, exactly what it did. Run the plan with `--dry-run`, read the deletes, then let the job write.

Share this article

[Share on X](https://twitter.com/intent/tweet?url=https%3A%2F%2Fadamtheautomator.com%2Frclone-cross-cloud-sync%2F&text=Prevent%20Cross-Cloud%20Sync%20Data%20Loss%20with%20rclone%20on%20Linux)[Share on Facebook](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Fadamtheautomator.com%2Frclone-cross-cloud-sync%2F)[Share on LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fadamtheautomator.com%2Frclone-cross-cloud-sync%2F)

## Related Posts

![](https://adamtheautomator.com/wp-content/uploads/2021/08/pankaj-patel-Ylk5n_nd9dA-unsplash-scaled.jpg)

### [How to Set Up An AWS S3 Static SSL Website](/aws-s3-static-ssl-website/)

Learn how to set up an AWS S3 static SSL website served securely using AWS Cloudfront in this step-by-step guide!

![](https://adamtheautomator.com/wp-content/uploads/2020/07/How-To-Upload-Local-Files-to-AWS-S3-with-the-AWS-CLI.png)

### [Upload Files to S3 with AWS CLI: A Comprehensive Guide](/upload-file-to-s3/)

Master the AWS CLI tool for uploading, downloading, and synchronizing files between local locations and AWS S3 buckets.

![](https://adamtheautomator.com/wp-content/uploads/2020/02/0.png)

### [Deploy AWS CLI and CloudFormation for Infrastructure](/aws-cli-and-cloudformation/)

Master AWS CLI and CloudFormation deployment for efficient infrastructure management. Learn the advantages over manual provisioning.

## Categories

*   [IT Ops](/category/it-ops/)
*   [Cloud](/category/cloud/)
*   [DevOps](/category/devops/)
*   [Home Ops](/category/home-ops/)
*   [Information Security](/category/infosec/)
*   [Software Development](/category/software-development/)

## Site

*   [Home](/)
*   [Tutorials](/tutorials/)
*   [Instructors](/author/)
*   [Advertising](/advertising/)
*   [Recommended Resources](/resources/)
*   [About Adam](/about-adam/)

Copyright 2026© ATA Learning | [Privacy Policy](/privacy/)