Could y’all opine if my backup disk should be formatted XFS or Btrfs in Unraid? I’ve recently upgraded my file storage
before: Windows directories to a separate windows machine with a disk in a hotswap cage using FreeFileSync.
now: an Unraid NAS, with the backup disk in a hotswap cage on the same machine. I am setting up Duplicacy. I will move the hotswap cage to a seprate machine later…I still need to build it.
For the current setup, I have a disk mounted in Unraid which is not a part of my array. It is currently NTFS. I assume it would be better to convert it to XFS or Btrfs.
I am backing up
my personal files, photos, wedding videos, ebooks, music. Approximately 6TB of data.
I am not backing up my media movie and TV files (for which I own hard copies and don’t feel the need to backup…except my copy of 1985 Transformers the Movie
And if all of this goes well, I’ll upgrade to backing up to separate machines and cloud locations.
Backup to the disk in the same system makes little sense - if your power supply dies entire system is destroyed. Single disk is less reliable than an array; and yet your backup shall be more reliable than whatever source data resides on. Don’t backup to a single disk.
Use zfs on your main array where the data lives. With zfs you can enable periodic snapshots — that would be your local backup history. Coupled with default filesystem compression It would be more efficient that any backup solution, and 100% cheap resource wise.
Btrfs is ok too, but zfs is better (more mature and stable, with features you can benefit from in the future if you choose so, like metadata device, l2 arc and much better caching policy). It does require more ram for better performance so if your sever has less than 8GB of ram then btrfs will be better option.
Don’t use xfs as it does not offer data consistency guarantees. Duplicacy can mitigate it somewhat with erasure coding but it’s not free and not as robust as filesystem native support for self healing
With zfs (and btrfs), you could directly replicate filesystem snapshots. Also no third party tools required. zfs send / zfs receive replicates a resumable snapshots to the remote system straight from the filesystem so no seek latency, or any additional computations needed: it’s a filesystem data stream, not file by file replication.
Then (and in addition) backup the mounted snapshots with duplicacy to offsite storage. I would however suggest copying media as is: it does not benefit from deduplication or compression or versioning; any changes in media files are corruptions, so copying them to cloud storage as is with object lock enabled is more prudent approach.
Personal anecdote — I started with backup to cloud storage, but now all my data resides on the servers I control (at home and friend and family locations) that replicate to each other via ZeroTier. There is one additional backup to amazon glacier deep archive (in case all servers fail at once) but duplicacy does not support backup to glacier, so this is out of scope of this forum. Perhaps storj or wasabi would be next best thing — but that’s at least 4x more expensive.
These are great notes. Thank you for taking the time to share them.
You mentioned “Use zfs on your main array where the data lives”…when I set up my Unraid array with the parity drives, somehow I missed that Unraid supported snapshots of the array, and I did not notice the option for ZFS…I’ll have to revisit that…
