Specify from which snapshot to delete those revisions.
Thank you. That wasn’t obvious to me when using the docker GUI for prune.
I agree. Perhaps duplicacy (at least CLI, but also GUI – otherwise what’s the point of it existing?) shall provide better diagnostic in the logs: if the combination of command line arguments user specified results in some arguments having no effect – there shall be some warning, because it’s definitely not what user wanted.
A lot of people, including myself, have wasted a lot of time figuring out why prune refuses to act in similar scenarios, only to discover that ether -a
or -id
has to be provided, otherwise it will happily and successfully do nothing… Which is technically not incorrect (-- “Prune these revisions from no snapshots”; --(Does nothing) “Done, exactly as requested!”); just poor user experience.
Hey. Wondering of someone could just do a sanity check on my policy? TIA!
-keep 7:7 -keep 30:60 -keep 0:270 -a -exhaustive
-keep 7:7: Keep one backup every week for anything older than seven days
-keep 30:60: Keep one backup every month for anything older than two months
-keep 0:270: Keep nothing older than nine months
Days 1 - 7: Daily (7 snapshots) Not specified in command, but assumed?
Days 8 - 60: Weekly (7 snapshots)
Days 60 - 270: Monthly x (7 snapshots)
Day 270+: 0
Storage: 21 total snapshots
-a Scan all snapshots
-exhaustive Remove all un-referenced for any reason
- Remove -exhaustive flag.
- reverse order of -keep flags.
Thank you. After running prune I see “Missing Chunks” under the check process status. Alos, the storage size was not updated on the graph. What can I run to reconcile?
Run check -fossils
- see if it’s the same.
That worked, thank you. But my storage size should have been reduce by 50% based on my revised prune policy above. Now that the check went through I see only a small decrease in size. What am i missing?
Pruning happens in a two-step process - marking then deleting files. But you need to run backups for all IDs between these steps. So chunks will eventually be deleted after you backup then run another prune.
You can speed this up but using prune -exclusive
but you need to make sure no other operations are running on your storage.
Also, if that -fossils
worked, you may want to run it again with -fossils -resurrect
before you do any clean-up.
Thank you. What would be a boilerplate daily job then?
- Backup
- Prune
- Check
Is that more or less the routine?? Special commands?
Thx for your help.
Yep that’s usual. No special commands. You don’t always have to run prunes or checks daily though. Weekly prunes are fine too.
Hi everyone,
I only have space for 1 backup on my google drive.
So I want tonight to have a backup. Tomorrow it should delete that backup and take a new one.
I’m using: -keep 1:1 -keep 1:2 -a
But it’s not deleting anything. Can anyone help?
I ran this policy and what i see in my restore files are:
Daily for the last month
Weekly for 6 months after that (until mid april)
Monthly for two months after that (mid march, feb)
Very different from the stated goal. Any ideas?
-
Backup are incremental and deduplicated. If you take backup today and it transferred 10GB of data, and then you add 1MB of new unique data and take backup tomorrow — your storage will grow by 1MB, give or take. There is no need to keep aggressively deleting old backups.
This needs to be sorted by time:
: -keep 1:2 -keep 1:1 -a
This would mean: for snapshots older than 2 days keep one snapshot every dat. For snapshots older than 1 day keep snapshot every day. Quote a pointless redundant policy.
Also prune will not delete the last remaining backup.
My suggestion: do -keep 0:10 -a
: this will keep rolling history of 10 days worth of backups.
Again, remember, backups are incremental.
I’m still wondering why a policy of…
-keep 0:270 -keep 30:60 -keep 7:7
Produces:
Daily for the last month
Weekly for 6 months after that (until mid april)
Monthly for two months after that (mid march, feb)
?
The only part that seems consistent is the 0:270
See:
It does not.
Your policy as written will yield
no revisions older than 270 days,
one revision every month between two months and 270 days
one weekly revision starting from week two until two months
every revision for a week.
you can use -dry-run parameter to see what prune will be doing without actually doing it.
That’s also what I expected. But after running that exact policy, as you can see, it did not produce those results. These are all my restore files, which reflect a different policy of:
Daily for days 1-30
Weekly for days 30 - 180
Monthly for days 180 - 300
Hi @saspus Just wondering what you make of these screenshots, as compared to the retention policy. I also keep getting Missing Chunks after running checks, and also tried -fossils -resurrect at @Droolio 's suggestion, but still produces Missing Chunks. Not sure what’s going on. Should I be posting this somewhere else?
Did you have that different policy before? Maybe you have another instance pruning with that policy from somewhere else?
The missing chunks can happen if any prune is ever interrupted. It’s a pure cosmetic issue, no data is lost: ironically, exact opposite happens: the snapshot files that were supposed to be deleted from the store, remain, because prune was interrupted before it could delete them. But the chunks that were not used by any other snapshot has been deleted, so these ghost snapshots will now always fail check. Therefore the solution is not to resurrect them from the dead, but to finish deleting them. Have a look here: Single missing chunk after first backup, every time - #2 by saspus
The actual fix would be to implement two-step snapshot deletion approach, the same one is used for snapshots, this issue has been open for a while now. Zero size chunks: how to solve it once and for all? - #2 by saspus
@gchen, can fixing this be prioritized? This appears to cause unnecessary concerns about data integrity for many users. Including myself.
I did have a different policy in the past but my understanding is that once I ran the new policy (several times at this point) the restore files would reflect the new policy as files were pruned according to the new rule set.
I will try this, and the re-prune. Hopefully my policy will then be reflected in the restore files (?).
I concur