Merging chunks older than xx days

Support
#1

I’m not sure if I am going to make sense, but, here goes…

I’ve been running Duplicacy since about August and it is working great. However, with it running every 15 min, I have thousands of chunks (or whatever they are called). Is there a job I can run that will merge, say, all of the chunks that are older than 30 days so that I am not keeping them all and so that it makes doing a restore easier?
I know that if I merge the chunks older than xx days, that I can’t restore to a version before that number of days.

I hope this makes sense?

#2

Hey rbeatse,

that’s what the prune command is for. You specify how many version you like to keep and it will remove every version (and the storage requirements) of the versions between them.

You can specify multiple filters, and they read as the following:

Keep n day intervals after x days.

So for -keep 1:2 it would keep daily backups if they are at least 2 days old. So if you do hourly backup those will be thin out until there’s only one left for each day. Prune will do a second pass automatically and remove all unneeded chunks after the first pass has scanned the whole backup storage.

I use this:

-keep 1:7 -keep 7:62 -keep 30:730 -keep 365:1460 -keep 0:3650

Which keeps:

  • daily backups after 7 days
  • weekly backups after 2 month
  • monthly backups after 2 years
  • yearly backups after 4 years
  • removes the backups altogether after 10 years.
#3

Thank you so much for this information. Is this something that is only in the CLI? I have set everything up, so far, using the web UI but I can figure out the CLI if I need to, I guess. I don’t suppose there is a way to export the job the Web UI is running in the background so I can be sure that I keep what I have (I’m not the most techy guy).

Thanks again for the explanation!,

#4

No. You can schedule a prune job in the web ui.

Technically I have to say that prune doesn’t merge chunks.
It just deletes unneeded ones according to prune rules.

Pruning won’t affect restore speed significantly. Just check speed.

#5

Ah, you’re using the GUI? That’s even simpler.

Here’s what @bkeeper means in pictures:

Prune_Job_01
Prune_Job_01504×929 61.4 KB

(go to SCHEDULE and click on the green button down low to create a new one)

Prune_Job_02
Prune_Job_02708×647 98 KB

(Give the Schedule a name and put it to something like daily)

Prune_Job_03
Prune_Job_031911×912 186 KB

(The new Schedule is now at the bottom of the list, click on the plus on the bottom left)

Prune_Job_04
Prune_Job_04735×590 96.1 KB

(Select type Prune, and select the retention policy you like to keep)

#6

If there are too little fields for your retention policy, just click safe and modify the command line options afterwards by clicking on it:

-keep 1:7 -keep 7:62 -keep 30:730 -keep 365:1460 -keep 0:3650 -id my_backup_one -threads 4

This would delete the snapshots in the same manner like mentioned before, but only with the given id of your backup. You can omit -id my_backup_one and add an -all if you like to have it apply the retention policy on “all” backups. :slight_smile:

Here’s the full help:

$ duplicacy prune --help
NAME:
   duplicacy prune - Prune snapshots by revision, tag, or retention policy

USAGE:
   duplicacy prune [command options]  

OPTIONS:
   -id <snapshot id>            delete snapshots with the specified id instead of the default one
   -all, -a                     match against all snapshot IDs
   -r <revision> [+]            delete snapshots with the specified revisions
   -t <tag> [+]                 delete snapshots with the specified tags
   -keep <n:m> [+]              keep 1 snapshot every n days for snapshots older than m days
   -exhaustive                  remove all unreferenced chunks (not just those referenced by deleted snapshots)
   -exclusive                   assume exclusive access to the storage (disable two-step fossil collection)
   -dry-run, -d                 show what would have been deleted
   -delete-only                 delete fossils previously collected (if deletable) and don't collect fossils
   -collect-only                identify and collect fossils, but don't delete fossils previously collected
   -ignore <id> [+]             ignore snapshots with the specified id when deciding if fossils can be deleted
   -storage <storage name>      prune snapshots from the specified storage
   -threads <n>                 number of threads used to prune unreferenced chunks