I am having a bit of trouble finding the root cause of this issue and how to better configure my setup to avoid this. I first noticed this because my offsite storage was growing much faster than local storage when I expected it to be the same. I ran a prune -exhaustive -exclusive -dry-run -all on the b2 storage and there were lots of unreferenced chunks. Running without dry-run cleaned up the b2 storage and the local and nas storage were back inline size-wise.
I’ll start with an overview of my setup. I have two storages: one local on my nas and offsite in b2. I have multiple machines backing up to the nas storage, each machine has its own snapshot id. I run duplicacy-web in Docker on my nas. Other machines in the house run duplicacy cli via cron. There are backups that run on the nas to the nas storage via duplicacy-web schedules. My desire is to have the local storage copied offsite and no clients ever back up to the b2 storage; it is copy only.
My schedules in duplicacy-web:
- A check job runs on both storages, in parallel, daily starting at midnight. The intent is to keep the graphs in duplicacy-web up todate.
- The nas’s own backup jobs run, in parallel, hourly starting at midnight. There are two snapshot IDs from this based on logical grouping of files on the nas.
- Pruning happens daily at 02:30 am. The prune jobs have identical arguments (
-keep 0:1800 -keep 7:30 -keep 1:7 -a -threads 10). They run in parallel. The intent is to minimize the amount of chunks we copy to b2 and also to keep b2 storage closely mirrored to the local storage. - Offsite copy runs every 6 hours, starting at 04:00. The intent is to simply copy the local storage to b2 for offsite backup purposes.
I’m having a hard time figuring out where I go wrong in my setup that causes this. The prune jobs finish in time for the next copy to start. Backups do often run during the prune, but the prune is non-exclusive, so I believe that shouldn’t matter (?). Any help in improving this or diagnosing where I’ve introduced the problem would be helpful.