Back up a VPS you can actually restore

Ask anyone why they back up and the answer is usually “in case the disk dies”. On a modern VPS that is close to the least likely thing that will happen to you. Your volume sits on NVMe in a redundant array with a hypervisor that will migrate you off a failing host before you notice; drive failure is the provider's problem and it is a solved one. Building your entire backup strategy around it is like buying a fire extinguisher for a house that floods.

Here is what actually takes people's data, roughly in order of frequency. First, and by a wide margin: you. A rm -rf with a variable that expanded to nothing. A DROP DATABASE on the wrong terminal because two tabs looked identical. A migration script run twice. A docker compose down -v where the -v was muscle memory. These are not exotic; they are Tuesday.

Second, applications that delete their own data. An upgrade that runs a destructive migration and fails halfway. A plugin that clears a cache directory that turned out not to be a cache. A log rotation configured against the wrong path. Third, compromise — and note that this is now the case where an attacker wants the backups gone, which changes the design in ways we come back to below. Fourth, and easy to forget on a no-KYC host: losing the account. A balance that hit zero while you were away, a password manager entry you never wrote down, an email address you abandoned. We suspend within hours of a zero balance and destroy seven days later, and that is deliberately unforgiving — it is the arithmetic that lets us not ask who you are. There is no support agent who can look up your identity and make an exception, because there is no identity to look up.

That last category is the one that separates privacy-first hosting from the mainstream kind, and it should shape your plan. Every control that makes this server hard to attribute to you also makes it hard for anyone — including us — to give it back to you. The backup is what turns that trade from a risk into a choice.

Snapshots are wonderful and you should use them. Take one before every kernel upgrade, every database migration, every time you are about to do something you would rather be able to undo. They restore in seconds, they cost almost nothing, and they will save you an afternoon a dozen times a year.

They are still not backups, for three structural reasons that no amount of provider marketing changes. They live inside the same account: whoever can log into your panel can delete them, and if the account stops working the snapshots stop existing with it. They live in the same failure domain: same provider, same control plane, often the same storage cluster, which means a single bad event can plausibly take both the original and the copy. And they are opaque: a snapshot of a running machine captures a database mid-write exactly as faithfully as it captures everything else, so a snapshot restore of a busy MySQL server gives you a crash-recovery scenario rather than a clean database.

The old 3-2-1 rule — three copies, on two kinds of media, one of them off-site — is usually recited without noticing that the middle clause is meaningless on rented infrastructure. You do not have two kinds of media. You have a virtual block device, and another virtual block device, and both of them are somebody's SAN. What survives the translation to a VPS is the part that mattered all along: at least one copy must be somewhere a single bad event cannot reach. Different provider or at minimum a different region, different credentials, different account, and — if the reason you host offshore is legal rather than technical — a different jurisdiction, so that one court order does not land on both copies at once.

This is the section other backup tutorials do not have, and it is the one that matters most if you paid for this server in Monero.

The default advice everywhere on the internet is to push your backups to an object store: Backblaze B2, Amazon S3, Wasabi, a Google Drive via rclone. It is cheap, it is durable, it works. It also, in one command, undoes a great deal of what you were doing. Opening that account required a card and usually an identity document. From the moment the first snapshot uploads, that provider holds a timestamped record associating your name and payment details with your server's IP address, refreshed every hour, forever. You did not verify your identity with us; you verified it with them, and then you drew a line between the two.

The second half is worse, and it is the part people miss. The API key that authorises those uploads is a file on the production server. Anyone who gets root on that box — or anyone who lawfully acquires the disk — does not merely get your data. They get a credential that resolves, in one API call, to a billing identity. The anonymous server has become a signpost pointing at your bank.

None of this makes object storage wrong. It makes it a decision rather than a default. Three ways out, in descending order of how well they preserve what you started with:

• A second no-KYC VPS, ideally in a different jurisdiction from production, paid from the same crypto balance. The destination inherits the anonymity properties of the source instead of contradicting them. This is what we do and what the rest of this guide assumes.
• A storage provider that takes crypto without identity. They exist; they are smaller; check whether the payment path really is identity-free before you trust the marketing, and check the egress pricing before you find out during a restore.
• A machine you physically own, pulling over WireGuard from behind your own connection. Excellent for anonymity, poor for availability, and it means your restore speed is your home upload speed. Reasonable as the third copy, weak as the only one.

Whatever you choose, the account that holds the backups should not share credentials, email address or recovery path with the account that holds production. The whole point is that one compromised login must not reach both. This is the same discipline as keeping identities separate elsewhere, applied to the least glamorous corner of the system.

Almost every backup tutorial produces the same architecture: a cron job on the production server that authenticates to a remote repository and writes to it. It is simple, it works, and it has a property nobody mentions — the production server holds a credential that can delete the entire repository. Pruning old snapshots requires delete rights, so the key that runs your nightly job is also the key that empties the vault.

Think about what that means during a compromise. An attacker with root on the box does not need to find your backups; you have helpfully left them a working credential and a config file naming the repository. Enumerating and destroying backups before triggering anything visible is not a hypothetical refinement — it is standard practice, because it is what turns an incident into a negotiation. Your nightly job was the reconnaissance.

There are three designs, and the difference between them is entirely about which machine holds which key.

Plain push is the one above. Production has read-write-delete. Convenient, and it fails completely against exactly the scenario where you most need a backup. Use it only if the sole threat you actually care about is your own fingers.

Append-only push keeps the same shape but removes the dangerous verb. The repository is served by something that understands the protocol and refuses deletions: rest-server --append-only for restic, or borg serve --append-only forced from authorized_keys. Production can create new snapshots and cannot remove old ones. Pruning happens later, from somewhere else, using a different key. This is a large improvement for about twenty minutes of work, and for most people it is the right stopping point.

Pull inverts the connection. The backup host reaches into production over SSH, copies what it needs, and runs the backup tool locally on its own disk. Production holds no backup credential at all — there is nothing on that machine to find, because the machine does not know where its backups live. This is the strongest arrangement and it is what the step-by-step below builds.

Be honest about what pull costs, because it is not free. You have moved the key rather than eliminated it: the backup host now holds an SSH key into production, so a compromise of the backup host reaches forward into the live system. That is a better trade — the backup host runs nothing, exposes nothing but SSH, and is a far smaller target than a public web server — but it is a trade, and you close most of the remaining gap by forcing that key to a read-only command so it cannot be used for anything but reading files.

Three tools cover essentially every case, and the choice is less agonising than the forum threads suggest.

restic encrypts by default, deduplicates across snapshots, speaks SFTP, S3, REST and a dozen other backends, and ships as a single static binary you can drop onto anything. Its repository format is content-addressed, so a snapshot is cheap and identical data is stored once no matter how many machines send it. The costs are real but modest: it wants memory proportional to the repository index, and an interrupted run can leave a stale lock that the next run refuses to step over until you clear it with restic unlock. This is the default recommendation and what the steps below use.

Borg deduplicates better, compresses better, and is noticeably faster on repositories with millions of small files. Its trade-off is coupling: Borg must be installed on both ends and versions must match closely, one repository is really designed for one client, and it has no native object-store backend without a helper layer. If your source is a large mail spool or a filesystem full of small files, and you control both machines, Borg will use half the space. Its --append-only mode is also the cleanest implementation of the idea in either tool.

rsync is not a backup tool and pretending otherwise is how people end up with one faithfully mirrored copy of a corrupted directory. It has no versioning, no deduplication and no encryption at rest; rsync --delete propagates your mistake to the copy at wire speed. It is nevertheless the right tool for moving bytes between two machines you control, which is precisely the job it does in a pull design, with restic providing the versioning and encryption once the bytes have landed. Use each for what it is.

One thing not to do: do not roll your own with tar and a date-stamped filename. It works for about four months, until the day the disk fills because nothing ever expired, or the day you discover that a full copy every night of a 40 GB dataset is 1.2 TB a month of storage you are paying for to hold thirty nearly identical things.

The instinct is to back up the whole filesystem. Resist it. A root filesystem is mostly distribution packages you can reinstall in ninety seconds, and including them costs storage, transfer time and, worse, attention — a 40 GB backup nobody wants to test is less useful than a 900 MB one that gets restored quarterly.

What genuinely cannot be reconstructed is short: /etc (your entire configuration and the reason a rebuild takes an hour instead of a weekend), /home and /root, /srv and /var/www, application state under /var/lib/ and /opt/, named Docker volumes, cron and systemd units you wrote, and your database dumps. Skip /proc, /sys, /dev, /run, /tmp, /var/cache, swap files, sockets and /var/lib/docker/overlay2 — that last one is rebuildable from your compose file and is often the single largest directory on the disk.

Now the part that quietly ruins restores. Copying a database's data directory while the database is running produces a file set that is very likely to be unusable. The engine holds state in memory, writes across multiple files in an order that matters, and your copy walks that tree over several minutes catching different files at different instants. InnoDB will sometimes crash-recover from it and sometimes will not; the failure surfaces months later during a restore you are already having a bad day about.

Dump through the engine instead, then back up the dump:

# MariaDB / MySQL — InnoDB gets a consistent snapshot from one transaction
mariadb-dump --single-transaction --quick --routines --triggers --events \
  --all-databases | zstd -T0 > /var/backups/db/all-$(date -u +%F).sql.zst

# PostgreSQL — pg_dump is consistent by design; grab roles separately
pg_dumpall --globals-only > /var/backups/db/globals.sql
pg_dump -Fc -Z6 mydb    > /var/backups/db/mydb.dump

# SQLite — never cp a live database; WAL mode makes that a torn file
sqlite3 /var/lib/app/app.db ".backup /var/backups/db/app.db"

Two footnotes worth having. --single-transaction gives you consistency for InnoDB only — if any table is still MyISAM it is copied outside the transaction and can be inconsistent with the rest, so convert those tables or lock them. And once you are dumping, exclude the live data directory from the file pass. Backing up both means storing a large torn copy alongside the good one, and giving a future restore two candidates where one is quietly wrong.

Containers do not change the principle, only the path: run the dump with docker compose exec -T db mariadb-dump … and back up the resulting file, not the volume underneath it.

Client-side encryption is the entire reason it is acceptable to store your data on a machine that is not the one it came from. restic and Borg both encrypt before anything leaves the source, so the backup host holds ciphertext and can be treated as untrusted storage. That property is worth exactly as much as your handling of the key, and no more.

The failure mode is depressingly common. The repository password sits in /etc/restic/env on the production server, which is fine and necessary for automation. Then production is what you lose — destroyed, seized, or simply gone with the account — and you are looking at several hundred gigabytes of encrypted blocks and the dawning realisation that the only copy of the key was inside the thing you were protecting yourself against losing.

So: the password on the server is a working copy, never the record. The record lives somewhere that survives the server — a password manager whose vault is itself backed up elsewhere, or written on paper in a drawer, or both. Write down the repository location and the exact restore command next to it, because a passphrase with no context is a puzzle you will be solving under stress in eighteen months. And take five minutes to verify it: from a different machine, with only what is in the password manager, run restic snapshots against the repository. If that works, you have a backup. If it needs something that only exists on production, you have a very expensive folder.

Both tools support multiple keys on one repository (restic key add), which is the clean way to give the pruning job or a second admin access without sharing the original passphrase. And if the production disk is encrypted with LUKS, keep the two secrets genuinely separate — storing the restic passphrase inside the LUKS volume and the LUKS passphrase inside the restic repository is a loop that fails closed on both sides.

Retention looks like a storage-cost question and is really a detection-latency question. The number that matters is not how much disk you want to spend; it is how long a problem can go unnoticed in your system. Whatever that period is, your oldest backup must be older than it.

Seven daily snapshots feels generous and is thin in practice. A corrupted table nobody queries, a slow deletion by a misbehaving cron job, an intrusion that sat quietly for a month before doing anything visible — all of these routinely take longer than a week to surface, and if your history is seven days deep then every snapshot you hold is already contaminated. Dwell time in real intrusions is regularly measured in weeks. Cheap monthly snapshots are the specific defence against that, and deduplication makes them very cheap indeed: a monthly kept for a year adds a fraction of a full copy, because only the blocks that actually changed are stored twice.

restic forget \
  --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 2 \
  --prune

That ladder — a week of days, a month of weeks, a year of months, a couple of years of years — is around twenty-six snapshots and, on typical data, well under twice the size of a single full copy. It is the default we would argue for unless you have a specific reason to deviate.

Two operational notes. forget without --prune only removes the labels, so the space is not reclaimed until you prune — which surprises people watching a disk that will not shrink. And --prune requires delete rights on the repository, so in an append-only or pull design it does not run on the production server. It runs on the backup host, or from your laptop, with a key that production has never seen. That separation is the whole point; do not undo it for convenience at the last step.

The classic backup disaster is not a job that crashes. It is a job that stops running and tells nobody, discovered eleven months later by someone who needed it. Every element below exists to make that specific outcome impossible.

Use a systemd timer rather than cron. You get real logs in the journal with exit statuses attached, Persistent=true so a run missed while the machine was off happens at next boot instead of being skipped forever, and RandomizedDelaySec so a fleet does not stampede the backup host at 03:00 exactly. Cron's failure notification is an email to a local mailbox that, on a modern server, goes precisely nowhere.

# /etc/systemd/system/backup.timer
[Unit]
Description=Nightly encrypted backup

[Timer]
OnCalendar=*-*-* 03:17:00
RandomizedDelaySec=1800
Persistent=true

[Install]
WantedBy=timers.target

Then add a dead-man's switch, which is the single highest-value line in this entire guide. At the very end of the script — after the backup has succeeded, not before — make an HTTPS request to a monitor that expects to hear from you daily and alerts when it does not. This inverts the notification logic: instead of relying on a failure to generate a message, absence itself becomes the alarm. A job that has been dead for three days is then a message in your inbox rather than a discovery in a crisis. Self-host the monitor on the backup host if you would rather not create another third-party account; it is a few lines and a timer of its own.

Finally, the small operational hazards, all of which we have hit at least once. An interrupted run leaves a lock and every subsequent run fails with a message you stop reading after the fourth night — handle restic unlock deliberately, not reflexively. A full destination disk fails every job until someone looks; alert on free space, not just on job status. An SSH key with an expiry, a rotated host key, an nftables rule added during unrelated work — each will silently sever the pull. And systemctl enable --now, not just start: a timer that was never enabled works beautifully until the first reboot and never runs again.

Everything above is preparation. This is the part that converts it into a backup, and it is the part almost everyone skips.

Once a quarter, deploy a fresh VPS — the $5 Starter is ample and billing is hourly, prorated to the second, so the whole exercise costs a couple of cents. Restore into it using only what you would have in a real disaster: the repository address, the passphrase from your password manager, and the written procedure. Deliberately do not use anything from production, because in the scenario you are rehearsing production does not exist. Bring the application up, point a hosts-file entry at the new IP, click around. Then destroy it.

# on a throwaway machine, with nothing but the passphrase
export RESTIC_REPOSITORY=sftp:[email protected]:/srv/restic/prod
restic snapshots                      # can you even see the history?
restic restore latest --target /restore
restic check --read-data-subset=10%   # verify stored blocks, not just metadata

What this catches is never what you expect. It is the config file that was in a directory the include list did not cover. The database dump that has been zero bytes for five weeks because a password changed and the script did not check its exit status. The application that will not start because a secret lives in an environment variable held by the orchestrator and was never in the filesystem at all. The undocumented DNS record. The certificate that has to be reissued before anything answers on 443. Every one of those is a ten-minute fix on a quiet afternoon and an ugly two hours at three in the morning.

Write down how long the drill took, end to end. That number — not the backup frequency — is your real recovery time, and it is the only honest answer when someone asks how long you would be down. Add restic check --read-data-subset=5% to a monthly timer as well: it reads and verifies a rotating sample of the actual stored blocks rather than only the index, which is how you find silent corruption while there is still a good copy to fall back on. If you are ever moving the whole server to a new host, a rehearsed restore is also most of the migration already done.