Robustness with respect to missing chunks

Missing chunks is a popular topic, and I figured I’d try to simulate that and see what can be done if that is to happen. So I manually deleted some subfolders under /chunks in storage (so these chunks would be, indeed, missing).

First of all, recommended approach (creating a new snapshot name / backup id with the same parameters and running it) worked. This did fill in all the holes in the original snapshots (obviously not guaranteed). But why couldn’t I simply run the original backup with -hash? I tried, and it didn’t re-upload missing chunks even after I dropped cache. It sounds like it should have accomplished the same?

OK, but what if there are still missing chunks after that? At that point, I have no idea what these relate to, only that some snapshots reference them. But is there any way to find out which file(s) are impacted by these chunks? Short of attempting to restore everything, I suppose. There could be millions of files and hundreds of snapshots, so figuring out whether or not I should care about a single missing chunk might be important.

Also, I assume there is no way to suppress some known missing chunks from check reporting, or is there? You may not want to delete whole snapshots (multiple even) if one unimportant chunk is missing, so you may want to accept that on a going forward basis. You’d still want to see new missing chunks though.

2 Likes

Hello,

  • it should be possible to find which file/files is corrupted via check command with files parametr, but it it’s be slow. Or with test restore…

  • If you have missing chunk and cant fix it with backup with “-hash” and cant delete all affected snaphots, then you have probably two choices:
    a) live with it - check command will always find error and backup monitoring will be nightmare
    b) start backup from scratch

I wrote something on this topic recently Web UI Restore with RSA Key and Passphrase - #11 by Flibble
My opinion on Duplicacy is that if there is a risk of loss/damage of chunks in the storage, it is necessary to use copy with bit-identical