Pruning with hourly backups

Support
#1

Hi,
I’m currently configuring Duplicacy to backup my unRAID server. I’m planning on running an Appdata backup every 12 hours and a Userdata backup every 2 hours. Ideally, I’d like to keep all backups/revisions for the first 60 days, then move to a 1 per day format for the next 6 months after. After those 6 months lapse, I’d like to keep 1 backup per week until 1 year where everything after is deleted. I’d like to follow this structure for both backup jobs.

I’ve configured separate backup storage for each job to make pruning easier in the scheduling. -keep only handles 1 revision every day. I’m struggling to find the best combination to achieve my retention policy, as for the first 60 days, I’ll be creating a total of 14 revisions a day. (12 on Userdata and 2 on appdata).

Any help and advice would be greatly appreciated!

EDIT: I think I’ve been looking at this wrong. I believe Duplicacy should keep all revisions UNLESS explicitly told not too? I’ve ended up with -keep 0:365 -keep 7:240 -keep 1:60 -a

Which I assume means the following
Since everything is pointed at backups older than 60 days, anything less should auto be kept.
Keep 1 revision each day for backups older than 60 days. (Day 61-240 | 6 months)
Keep 1 revision every 7 days for backups older than 240 days (Day 241-365 | 4 months)

Hopefully this is correct!

#2

Yep.

I find the logic slightly confusing, but I believe this is correct:
duplicacy prune -a -keep 0:365 -keep 7:240 -keep 1:60

#3

I agree, long supporter of Duplicacy, but the prune methodology needs some work in my opinion, very quickly gets complicated and hard to understand. Hopefully one day they’ll be a nice GUI update for it :slight_smile:

1 Like
#4

When using -keep arguments duplicacy deletes everything, except what it is instructed to keep, (but it will keep at least one revision. I.e. if your last backup was 10 days ago and you tell it to delete all revisions older than 5 days (-keep 0:5) it will still keep one.)

Each keep m:n argument results in_all_ revisions older than n days to be deleted, except preserving one every m days.

You can also delete specific revisions, so if you need more fine-tuning you can always do that with scripts.

But since duplicacy makes effectively incremental and deduplicated backups prune does not save as much space as people tend to assume. So the specific prune schedule does not matter much, the default that WebUI configures is usually good enough.

#5

So essentially what you’re saying is my intended retention policy is not possible with duplicacy?

Is there any better way to handle revisions when you’re backing up more than once per day? Not sure if im just a one off edge case but it feels like that’s a massive flaw within Duplicacy if that’s the case :frowning:

#6

Why not? The command line you proposed will do just that:

What will be kept:

  • no revisions: applies to those older than 365 days
  • weekly revisions: applies to those older than 240 days, and thus those that did not match the previous rule.
  • daily revisions: applies to those older than 60 days, and did not match previous rules.
  • everything younger than 60 days (because there is no rule affecting those revisions)

Duplicacy considers every -keep argument in order for every revision, and uses the first that is matching. That is why you have to sort them - if you had -keep 10:200 -keep 2:300 – the second argument would have no effect – anything that it would match would the previous one match too, and be used instead.

#7

Oh okay awesome. Sorry just this line caught me off-guard thinking Duplicacy wasn’t going to keep the backups less than 60 days old because it didn’t have a relevant -keep argument

So in theory the current set of arguments I have will achieve what I’m looking to do? :slight_smile:

#8

In practice too. You can always add -dry-run argument and see what would happen without it actually happening.

1 Like