Everybody says "test your restores". Almost nobody says what that means. Here is a ladder. Each step is a stronger claim than the one before it.
- The job ran. Weakest claim. It says nothing about the data.
- The backup has the expected size and file count. Compare to the last run. A sudden drop is an alarm.
- The tool's own integrity check passes (
restic check,borg check, or the equivalent: it verifies the repository structure and, if you ask, the data). - A sample restore matches. Restore to a scratch directory and compare checksums of a random handful of files against the originals.
- A full restore, timed. Now you know how long recovery takes, which is the number the business actually cares about.
- A restore on a clean machine using only what is documented, including the encryption key and passphrase. This catches the classic: the key was on the machine that just died.
Most setups sit at step 1 or 2 and call it a day. Put step 4 in a cron job and step 6 in the calendar, once a quarter.
A backup is what you can restore. Everything else is optimism with extra disk space.