Azure error: Failed to find the path for chunk ... 500 Operation could not be completed...

#1

Hello,

I was trying to create my first backup, probably around 3TB of data to Azure blob
storage, but it failed after a couple of hours with:

22:29:45.032 Failed to find the path for the chunk xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:storage: service returned without a response body (500 Operation could not be completed within the specified time.)

This is using Duplicacy 2.0.6 on Windows Server 2012 R2, with no rate limit and two threads. Outbound bandwidth is around 50-80mbps (not necessarily to Azure, just to the internet as a whole).

I know it might be difficult to detect if it’s Azure at fault, a momentary network interruption, or a bug in Duplicacy, but I wonder what the retry policy is. Under these circumstances, I wish that instead of aborting the backup as it did, Duplicacy could have continued with the next file.

Jeff

#2

Currently the Azure backend doesn’t support retrying. It is based on the Azure sdk, which added retry logic recently. So I’ll update my fork and this will get into version 2.0.7.

#3

Thanks – I’ll give 2.0.7 a shot when it’s released.

I’d prefer to use Azure as I have a good-sized service credit (via their non-profit program) to use with Azure services. But free storage doesn’t mean anything if I can’t use it reliably. Once you incorporate the latest release of the Azure SDK you linked to, do you feel that it will be ready for production? The SDK seems to be in perpetual beta, which doesn’t give me a lot of reassurance. If not Azure, which backend (hosted storage) would you recommend, given that I’d be using Duplicacy for mission-critical backups?

#4

The most popular backend I guess is Backblaze B2, likely due to its low cost. It is also the most stable one – it was one of the first to be supported and in the past we worked closely with their people to make sure we got it right. It may not be as fast as Amazon S3, so if the performance matters more to you then S3 would be the top choice.

Both B2 and S3 backends have a built-in retry mechanism, so they are more robust than the Azure one for sure.

The new S3 compatible storage Wasabi may also be worth trying. It’s even cheaper than B2 in terms of storage cost and API fees, although outgoing traffic is more expensive. It is faster than S3 in my test.