from Hacker News

You're backing up for the wrong disaster

by jitbit on 3/26/22, 6:50 PM with 6 comments

• by karmakaze on 3/27/22, 2:18 AM

  At one place I remember that we had the usual scheduled backups for our MySQL databases. We also wanted a quicker way to bring systems back with less data loss, so we also had a lagged replica that purposely applied changes delayed by I think it was about 4 hours. The other handy thing was that swapping a replica for the write leader was a common operation that we did whenever we upgraded hardware or software versions so it was a well exercised procedure.

• by bradknowles on 3/27/22, 6:22 AM

  W. Curtis Preston, author of the O'Reilly book "Backup and Recovery" has said:

  > Backups are useless. Restores are priceless.

  I also like:

  > The state of a backup is unknown and unknowable, until such time as it has been tested with a restore.

• by geoduck14 on 3/26/22, 8:10 PM

  Sounds legit. This goes double on the cloud. In my early days of using AWS, it was so easy to create and destroy EC2s, I created some EC2s for testing - then terminated them... then realized I terminated the wrong ones.

  Whoops

• by hamiltonians on 3/26/22, 9:01 PM

  human error is here. creating a backup and forgetting the password for example or mistyping it

• by Avlin67 on 3/27/22, 10:05 AM

  so true, had a zfs nas in 2009 with raidz, snaphots, everything, yet I did delete personnal stuff by mistake.