Which file is tied to this ERROR DOWNLOAD_CORRUPTED The chunk xxx?

Support
#1

I had this issue with Duplicacy and Wasabi’s us-west-1 almost a year ago. Now, the saga repeats 1 year later, except this time it affected my “personal files” repo instead of my “Macrium Reflect” repo. However, I will continue to restore with persist. My only question is how do I know which file was affected? Thank you.

I am using the Web GUI by the way.

2022-04-10 19:39:02.511 ERROR DOWNLOAD_CORRUPTED The chunk a0e0092f16e2d37877bf12f4ae53f158ff0269be02d3cd068f3686741644178e has a hash id of c6957553f981ae716ca75c97829790cbd4094ecf19d6574c26bc956176618822

#2

Well, it did report which was the broken file at the end of the RESTORE. Is there a way to determine that even earlier on?

2022-04-10 20:24:19.750 WARN DOWNLOAD_FAIL Failed to restore OneDrive/personal_files/Camera Roll/20191014_194814.mp4: chunk a0e0092f16e2d37877bf12f4ae53f158ff0269be02d3cd068f3686741644178e is corrupted

I have to say that I’m becoming a lot more comfortable with Duplicacy having done 3 full restores even with this one broken but thankfully useless file. I can’t say who is at fault, whether the software or the backend. Now, in order to isolate this. I’m willing to try another provider. Which backend provider should I try to iron out these issues?

After reading up on Zero size chunks: how to solve it once and for all?, I see we should only use S3/B2 for server side checksum verification. This means Amazon S3 is the only preferred backend then? Am I right? Everything else is as @saspus would say is flaky?

#3

Use a good object storage provider, not a file storage (like OneDrive, Google Drive, etc). The latter are not designed to be efficient with tens of thousands of files in the same “folder”.

Backblaze B2 is a good option. For the S3 standard, in addition to Amazon itself (which is good but expensive), there are options like Wasabi and others.

And enable erasure coding, it’s worth it.

#4

Unfortunately, I was using Wasabi and it’s S3 interface. I have to go look for some trial account with Amazon in order to test Duplicacy.

#5

For the last few years I’ve been switching between B2 and Wasabi, always for cost reasons (some use cases requiring egress work better on Wasabi, which doesn’t charge for it), but technically I’ve had no problems with either. At the very beginning, I had some difficulties with operations between buckets in Wasabi, right when it came out, but never a loss of file in either one.

I’m currently in a “Wasabi phase”…

1 Like