Many missing chunks, even after new initial backup

duplicacy check is complaining about many missing chunks in almost all of my snapshots (and, I suspect, has been for a while).

I recently tried the solution of changing my backup ID in preferences and running a new initial backup, which completed successfully. However, if I change back to my old backup ID, run a backup, and then run a check, a number of missing chunks are reported for that fresh backup. When I search for one of the missing chunks in the prune logs, I get the following:

logs/prune-log-20190729-040123
22130:Marked fossil d350cb071ecd9b3017793710b19d55fac040bf1c3982174b3827d468394372ee

logs/prune-log-20190819-040348
61353:Deleted fossil d350cb071ecd9b3017793710b19d55fac040bf1c3982174b3827d468394372ee (collection 3)

logs/prune-log-20190805-040419
162824:Deleted fossil d350cb071ecd9b3017793710b19d55fac040bf1c3982174b3827d468394372ee (collection 1)

logs/prune-log-20190812-040415
300300:Marked fossil d350cb071ecd9b3017793710b19d55fac040bf1c3982174b3827d468394372ee

…this looks pretty similar for most of the chunks. Any idea what happened, or, more interestingly, why a fresh backup under the old ID references chunks which the new initial backup didn’t recreate?

For what it’s worth, I have tried deleting my cache and looking for the chunks directly on the storage (I don’t see them).

A new backup ID is equivalent to a -hash option which cause all files to be packed and split. A new backup with the old backup ID will only pack new files since last backup and assume all other chunks referenced by the last backup exist.

The prune code had a bug that was fixed in CLI 2.1.1 2 years ago. Any chance you were still running an earlier version last year?

It’s possible, but I’m not sure. Hmm, okay, so it sounds like the best option at this point might be to to start a new backup ID and abandon the old one. Enough chunks are missing from older backups that I may not be losing much history that isn’t already lost, and having a known-good current backup is much more important. Does that sound like the best option to you, too?

Right, there isn’t too much you can do when there were missing chunks in an old backup which have propagated to the most recent one (and can’t be recovered). For new backups you should run a check job after each backup to catch such errors early.

Understood. That sounds like a plan!