Restore from downloaded cloud storage / what happens if chunks are missing/corrupt

Support
#1

Hi there,

new user here. As many others, I used to use CrashPlan, and while searching fro alternatives I found Duplicati which I use and I also like but I just think better safe than sorry and am looking for a second way to backup my documents :slight_smile:

So, I ended up evaluating Duplicacy and so far I quite like it and, given the limited GUI version forced me to use the CLI version I even think about changing to CLI mode with Duplicati ®. I’ll probably have a few questions so I’m glad this issues section exists and it seems to be very active whenever a question appears - so this is great too. I’ll for sure still get a license if I chose to use Duplicacy as a regular backup tool … the price seems very fair and I just think that

First questions …

  1. in case of a disaster and the need of a full restore would it be possible to download all files from the storage (say OneDrive) to a local HD/SSD and be able to a fast restore using these local copy? I.e. would it be possible to go to someone with a much better internet connection, just download everything from the storage onto a portable drive and restore from that one at home?

  2. what happens in case of some typical issues while a backup is made, i.e. the PC crashes or there’s a problem with the internet connection. Will this result in a corrupt backup? Will everything be ok after the next run that’s not interrupted?
    2.1) what happens if some chunk is corrupted/missing - will I still be able to restore everything else and will this limit the damage to just the files in the currupted/missing chunks?

Thanks a lot,
Nelvin

#2

in case of a disaster and the need of a full restore would it be possible to download all files from the storage (say OneDrive) to a local HD/SSD and be able to a fast restore using these local copy? I.e. would it be possible to go to someone with a much better internet connection, just download everything from the storage onto a portable drive and restore from that one at home?

This is fully supported in version 2.0.10. In the past different storages have different chunk nesting levels so you may need to flatten/unflatten the chunk directory if you move chunks from one storage to another. But starting from 2.0.10 the chunk directory always has a nesting level of 1 for all storages, so it becomes rsync/rclone compatible.

However, there isn’t a 2.0.10 release for the GUI version, so if you want this feature you may need to wait for the 2.1.0 release that will come out very soon.

what happens in case of some typical issues while a backup is made, i.e. the PC crashes or there’s a problem with the internet connection. Will this result in a corrupt backup? Will everything be ok after the next run that’s not interrupted

Unlikely. Duplicacy features a database-less design so when a backup is interrupted it may leave some unreferenced chunks in the storage in the worst case. And these unreferenced chunks can be reclaimed by the prune command with the -exhaustive option.

what happens if some chunk is corrupted/missing - will I still be able to restore everything else and will this limit the damage to just the files in the currupted/missing chunks?

Currently a corrupted/missing chunk will abort the restore process. I think it is a simple change to make it skip affected files and continue for the rest.

#3

Thanks for the infos. I already wondered why there are subfolders in my storage tests on local drives but not in my OneDrive folder.
With regards to the version I’m fine to use the 2.0.9 as the next full/GUI release does not seem to be that far away and I can restart my backups then easily, as long as I do have working backups right now already.

With regards to the corrupted/missing chunk - does this just mean a full restore will be aborted or is there no way to get to the other files even in a more manual way???
I.e. if a corrupted/missing chunk may make a backup fully useless (if it’s one of the first chunks used during restore) this would make for a horrible way to handle backups - I really hope it would just mean a bit more work during restore?
If not, i.e. it means the backup will really become useless, is this a high prio list on the roadmap? I mean, of course it’s always bad if some files are missing even when backupped, but seriously what would be a catastrophe is, to need the backup where, in theory 99.9% of the files are there but you can’t restore them because the first chunk is somehow missing.

If it just means we have to use the CLI then everything is absolutely fine.

#4

It depends on the type of the corrupted/missing chunk. If it is a snapshot chunk (i.e., a chunk that is needed to reconstruct the snapshot file), then there is really no way to skip this chunk. And you maybe to rebuild a corrupted/missing chunk if the files in the repository have been changed.

I think if you’re really worried about this issue, you should back up the same repository to multiple storages and run the check command after each backup. One design decision we made is that we assume the underlying storage will behave reasonably and not do crazy things (like removing or altering files for no reason), so we didn’t implement fail-safe techniques like error correction.

#5