Full System Backup - Linux

Support
#1

Hi, everyone. First post here.

I’m on Linux and I read this post. But there isn’t a lot of information on there.

If I have a hard drive failure, will I be able to restore my entire OS with Duplicacy ? I’m backing up my / folder and excluding some unnecesary folders like /sys,/tmp,/media,/mnt, lost+found, etc. but I can already foresee some issues. For example /etc/fstab because of the difference in UUIDs probably wont mount my drives?

Is what I’m trying to do possible with Duplicacy ?

#2

You probably need a hybrid approach: disk imaging software to create infrequent local periodic (perhaps even manual, after significant configuration change) bare metal snapshots and Duplicacy to create deep versioned user data only backup.

When disaster strikes you would restore an image and then restore user data from duplicacy backup on top.

It would be counterproductive to backup system data with duplicacy both in terms of performance, maintenance, reliability, cost and preventing user data from competing with system junk for the backup bandwidth.

2 Likes
#3

I use Acronis to create scheduled backups and i then backup those via Duplicacy. (For Bootable OS) everything else just gets uploaded via Duplicacy.

#4

Why do you backup backup? Your Acronis backup is only useful locally; if you replicate it to remote destination and your box dies it will be way faster to reinstall than to download that multi-terabyte archive.

I don’t know what’s the state of affairs is on Linux today but both macOS and Windows have already reached the state where reinstall is unattended and takes about half an hour and can be done routinely as it’s not a such a big ordeal as it once was. I know because I happen to migrate between machines all the tine and it’s completely painless. Maintaining Acronis or Macrium of whatever your tool of choice bare metal image is therefore more work than than any benefits it provides even in the lan; much less remotely. Consider OS reinstall an ultimate “restore”.

If you keep your data in iCloud (or OneDrive) this is even less of an issue: install, login, done. Keeping large installer images of the tools you need handy may be a good idea however (stuff like daVinci, matlab, capture One, steam cache, etc, anything else you might want avoid downloading).

1 Like
#5

I back the files up to a different machine in the house and then that uploads my Acronis Files. So they are not stored on the same PC in the house.

#6

Agreed. But depending on the package manager @Morafla can simply create a blacklist for the files which haven’t been changed outside the package manager.

I did (try) this in my restic setup, but restic was too slow to process the large list of blacklisted files. Will try this with Duplicacy in the future. :slight_smile:

With pacman (for Arch/Manjaro/EndeavoursOS etc) this would look like this:

echo "Generating exclude lists..."

# fetch all files currently supplied by packages
rm -f /tmp/restic-backup.pkg_files
while IFS= read -r -d $'\n' filepath; do
	[ -f "$filepath" ] && echo "$filepath" >> /tmp/restic-backup.pkg_files
done < <(sudo pacman -Ql | cut -f 1 -d ' ' --complement)

# check all files supplied by packages for changes, and write the changed files to a list
sudo paccheck --md5sum --quiet --db-files --noupgrade --backup | awk '{ print $2 }' | sed "s/'//g" > /tmp/restic-backup.changed_files

# backup the changed files (remove them from the blacklist)
grep -v -x -f /tmp/restic-backup.changed_files /tmp/restic-backup.pkg_files | sed 's/\[/\\[/g' > /tmp/restic-backup.blacklist

# add the global exclude list to the black list
cat "$GLOBAL_EXCLUDE" >> /tmp/restic-backup.blacklist
#7

If you’re curious about the performance, listing in pacman takes around ~4 seconds, doing a md5 check of the system with paccheck takes ~8 seconds. Grepping both results together takes ~40 ms.

That’s on a 10 core server with a SSD-RAID10-NAS.

So as long as the backup program can process the large blacklist better than restic, there shouldn’t be any performance concerns. :slight_smile:

I also create a list of the installed packages and their versions, so I can just fetch all packages again and apply the backup on top.

#8

So I tried this on duplicacy and it works fine. Takes a while (like 2-3 minutes) to process the large filter file, but continues afterwards without any issue:

Repository set to /
Storage set to sftp://123@456/backups/duplicacy
Erasure coding is enabled with 10 data shards and 1 parity shards
RSA encryption is enabled
No previous backup found
Indexing /home/ruben/root
Parsing filter file /home/ruben/.duplicacy/filters
Loaded 493738 include/exclude pattern(s)

I published the code here, if someone needs it:

1 Like