Prune tried to delete snapshot that doesn't exist - caused subsequent check failure

Bug
,
#1

Not touched Duplicacy in quite a while on my server and suddenly it started to fail. I can only imagine it’s because of a bug.

As usual, on sunday at 00:00 my weekly prune job runs, but this time it failed after 23 mins

ERROR SNAPSHOT_DELETE Failed to delete the snapshot at revision 11648: URL request ‘https://api002.backblazeb2.com/b2api/v1/b2_delete_file_version’ returned 400 File not present: snapshots//11648 4_z6ddea1936da5ce3a69ba0f1f_f10204e90a2629f15_d20210101_m100021_c002_v0001143_t002

Subsequently the scheduled daily check was run (01:00), which also failed, this time with
ERROR SNAPSHOT_CHECK Some chunks referenced by some snapshots do not exist in the storage
(in fact there’s 534 chunks listed in the log - presumably these didn’t get cleaned up by the prune command since it failed prematurely)

Since then there’s been hourly backups which completed without error, and i’ve just come to check b2 this morning, and the snapshot that Duplicacy tried to delete does indeed not exist.

I can only assume there’s a bug somewhere, where duplicacy had deleted a chunk, then tried to prune it again.
Backups are done hourly, so Revision 11648 was created 01/01/2021 10:00 am, (11649) is still there, and has 11:00 as the time. Out of all the revisions this year (again hourly), this happens to be the first one left after pruning.
image

For completeness, my prune job is set to keep one revision per hour up to a day, then one per week up to a month, then one per month up to a year
[-keep 0:365 -keep 30:30 -keep 7:7 -a -threads 30

#2

Ok this is looking pretty bad… :frowning: . I ran a subsequent prune to hopefully clean up the unreferenced chunk and it just deleted EVERY SNAPSHOT for all my backups…

The prune command logged the following message for every revision

INFO FOSSIL_GHOSTSNAPSHOT Snapshot revision 4902 should have been deleted already

I haven’t touched any configuration recently, and these settings/schedules have worked for almost a year and a half without any major issue. Something has gone really wrong with duplcaicy here…

#3

OK false alarm… mostly… The revisions that matter are fine. I ran a check and the ones that matter still exist. It was just surprising to see so many old snapshots older than a month suddenly get deleted by a prune when it’s only expected to see one (1 year old) get deleted each month. Turns out that these were much older snapshots that had been/should have been deleted in the past. No idea why they are suddenly getting cleaned up now though… Maybe suggests there is an underlying issue.

#4

Does the check job still fail?

#5

The check job passed after the last prune so it seems that whatever the problem was, it self-corrected. the strange part was that the issue appeared in the first place, and the other strange part is that the latest prune removed revisions that should have been long gone.

#6

I had something very similar, if not the same, happen to me when using B2 as well. It happened a few weeks ago and hasn’t come back since, so I had written it off as a one off fluke.