Prune B2 copy SNAPSHOT_INACTIVE and FOSSIL_POSTPONE messages

Support
#1

I have a local storage from which selected snapshots are copied to a B2 storage. Prune commands log messages such as these:

Running prune command from /share/CACHEDEV1_DATA/.qpkg/Duplicacy/.duplicacy-web/repositories/localhost/all
Options: [-log prune -storage NAS0 -id Duplicacy-config -keep 0:365 -keep 7:90 -keep 1:7]
2020-12-16 14:34:28.589 INFO STORAGE_SET Storage set to /share/TR-004/Duplicacy/storage
2020-12-16 14:34:28.594 INFO RETENTION_POLICY Keep no snapshots older than 365 days
2020-12-16 14:34:28.594 INFO RETENTION_POLICY Keep 1 snapshot every 7 day(s) if older than 90 day(s)
2020-12-16 14:34:28.594 INFO RETENTION_POLICY Keep 1 snapshot every 1 day(s) if older than 7 day(s)
2020-12-16 14:35:19.045 INFO SNAPSHOT_NONE No snapshot to delete

Running prune command from /share/CACHEDEV1_DATA/.qpkg/Duplicacy/.duplicacy-web/repositories/localhost/all
Options: [-log prune -storage Backblaze-B2 -id Duplicacy-config -keep 0:365 -keep 7:90 -keep 1:7]
2020-12-16 14:38:17.041 INFO STORAGE_SET Storage set to b2://kgorlen-NAS0
2020-12-16 14:38:17.447 INFO BACKBLAZE_URL download URL is: https://f000.backblazeb2.com
2020-12-16 14:38:18.159 INFO RETENTION_POLICY Keep no snapshots older than 365 days
2020-12-16 14:38:18.159 INFO RETENTION_POLICY Keep 1 snapshot every 7 day(s) if older than 90 day(s)
2020-12-16 14:38:18.159 INFO RETENTION_POLICY Keep 1 snapshot every 1 day(s) if older than 7 day(s)
2020-12-16 14:38:38.724 INFO FOSSIL_COLLECT Fossil collection 1 found
2020-12-16 14:38:38.724 INFO SNAPSHOT_INACTIVE Ignore snapshot Keith-Archive whose last revision was created 7 days ago
2020-12-16 14:38:38.724 INFO SNAPSHOT_INACTIVE Ignore snapshot Household-Archive whose last revision was created 7 days ago
2020-12-16 14:38:38.724 INFO SNAPSHOT_INACTIVE Ignore snapshot Joint-Archive whose last revision was created 7 days ago
2020-12-16 14:38:38.724 INFO SNAPSHOT_INACTIVE Ignore snapshot Carol-Archive whose last revision was created 7 days ago
2020-12-16 14:38:38.725 INFO FOSSIL_POSTPONE Fossils from collection 1 can't be deleted because deletion criteria aren't met
2020-12-16 14:38:38.725 INFO SNAPSHOT_NONE No snapshot to delete

First,the SNAPSHOT_INACTIVE messages are incorrect because all the listed snapshots have last revisions created over one month ago.

Second, what is causing the FOSSIL_POSTPONE message and how do I fix it?

#2

According to the description of how the algorithm works here: Lock free deduplication algorithm - How-to - Duplicacy Forum

Chunks that are set as “fossils” will only be deleted once a new snapshot (backup) is taken for a given snapshot id.

This is a safety mechanism as the prune command has no idea if a backup is running so it checks if a snapshot that is newer than the fossil collection exists before finishing the chunk removal process (the snapshot file is uploaded at the end of the backup process).

You do have the option to pass the -exclusive option to the prune command which skips this two-step process…with the important caveat that this must only be done while no other operation is running against the storage.

1 Like
#3

Sorry, I don’t understand. For which of the two messages is this an explanation?

I ran a prune with the -exclusive option on the B2 storage yesterday (12/15) to remove the chunks from an incomplete initial backup of a new repository. The -Archive snapshots have not been backed up in over a month – the latest revisions are over 7 days old and should not be interfering with fossil collection.

#4

A new snapshot needs to occur after the fossil collection step so the subsequent prune can know its safe to remove fossils.

So its not about no backups having occurred, its about a new backup having been taken after the first prune so the second step in the two-step fossil collection algorithm can safely remove the unreferenced chunks.

The -exclusive option skips this process as you are telling prune that no other backup operation is running against the storage at the time.

To remove unreferenced chunks from an incomplete backup you will need to use the -exhaustive option to the prune command as the snapshot file (which lists the chunks referenced by that backup) is only uploaded at the end of a successful backup.

A prune operation (without the -exhaustive option) only takes into account chunks that are referenced by backups to be removed based on your chosen retention policy (ie the-keep options provided).

1 Like
#5

I think I’ve found the problem. It’s related to https://forum.duplicacy.com/t/copy-prune-operations-on-non-existent-storage-ids-do-not-fail/4644?u=gorlen. One of the copy commands to the B2 storage had a misspelled snapshot -id, this went unnoticed because the copy command wasn’t failing, and so new revisions weren’t being copied to one of the snapshots, thus causing the FOSSIL_POSTPONE messages.

It appears that a SNAPSHOT_INACTIVE message means that a snapshot is being ignored by pruning because the latest revision is older than 7 days and is not preventing fossil deletion, correct?

#6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.