Restore fails. The chunk... has a hash id of... retrying

Support
#1

Hello. I started a backup a few days back with…

/root/go/bin/duplicacy restore -r 1

and it was humming along nicely and then I got the below error. I tried to switch to -hash but it stopped at the same chunk.

I am trying (duplicacy check -a -stats) now and will post it at the bottom. But below is a cut of the last few lines of from terminal with a bit of commentary for your review. I am transferring from one freenas box to an other. Both boxes are i7 with 24gb of ram. If needed I can post machine specks. The volumes are ZFS and permissions are read/write with access to the entire directory structure.

Downloaded chunk 348797 size 1682649, 12.38MB/s 26 days 02:01:57 4.5%
Downloaded chunk 348798 size 3826781, 12.38MB/s 26 days 02:01:50 4.5%

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2

Then I tried…

root@duplicacy:/mnt/backup # /root/go/bin/duplicacy restore -stats -hash -overwrite -r 1

Storage set to sftp://root@10.0.1.64//mnt/Raidz2/duplicacy/

Enter SSH password:

Enter storage password:

Discarding file attributes

Restoring /mnt/backup to revision 1

Downloaded chunk 348182 size 15749527, 1KB/s 226905 days 13:57:56 0.0%

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2

I thought it may be a problem with the snapshot I was trying to restore so I tried another one…

root@duplicacy:/mnt/backup # /root/go/bin/duplicacy restore -stats -hash -overwrite -r 2

Storage set to sftp://root@10.0.1.64//mnt/Raidz2/duplicacy/

Enter SSH password:

Enter storage password:

Discarding file attributes

Restoring /mnt/backup to revision 2

Downloaded chunk 341681 size 15749527, 1KB/s 222153 days 09:37:47 0.0%

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2

Ok that did not work. I am going a bit mad now. Lets see if I doing the same thing again will yield a different result.

root@duplicacy:/mnt/backup # /root/go/bin/duplicacy restore -stats -overwrite -r 1

Storage set to sftp://root@10.0.1.64//mnt/Raidz2/duplicacy/

Enter SSH password:

Enter storage password:

Discarding file attributes

Restoring /mnt/backup to revision 1

Downloaded chunk 1 size 15749527, 854KB/s 386 days 22:02:28 0.0%

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2; retrying

The chunk 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 has a hash id of a2e54b5fd73de0b96ea3702ae4ac61d90885bbbdc5d5fe302d4f6c9269774cc2

So you can see the version number.

root@duplicacy:/mnt/backup # /root/go/bin/duplicacy

NAME:

duplicacy - A new generation cloud backup tool based on lock-free deduplication

USAGE:

duplicacy [global options] command [command options] [arguments...]

VERSION:

2.1.1 (e8b892)

root@duplicacy:/mnt/backup # /root/go/bin/duplicacy check -a -stats

Storage set to sftp://root@10.0.1.64//mnt/Raidz2/duplicacy/

Enter SSH password:

Enter storage password:

Listing all chunks

All chunks referenced by snapshot backup at revision 1 exist

All chunks referenced by snapshot backup at revision 2 exist

All chunks referenced by snapshot backup at revision 3 exist

Snapshot mywork all revisions: 0 total chunk bytes, 0 unique chunk bytes

Snapshot backup at revision 1: 4149709 files (28914G bytes), 19546G total chunk bytes, 4,074M unique chunk bytes

Snapshot backup at revision 2: 4149829 files (28938G bytes), 19568G total chunk bytes, 438,685K unique chunk bytes

Snapshot backup at revision 3: 4172961 files (29295G bytes), 19920G total chunk bytes, 361,353M unique chunk bytes

Snapshot backup all revisions: 19925G total chunk bytes, 19925G unique chunk bytes
#2

I see that the check command sais that all your revisions are okay, no chunk’s missing. That’s good!

Try and run the restore commands to a new folder, clean, without anything inside (just initialize a new repository in an empty folder) with the following command: duplicacy -v -log restore -stats -r 1. Mind the arguments passed to duplicacy itself, and not to the restore option: -v -log: they print much more debugging info which may help in figuring out what the problem may be.

Lastly,

I don’t exactly understand what you’re saying in that post. Please try to put the code portions in between triple backticks (this is a backtick: ` ) like so:

```
this is a code portion
which spans multiple lines


and is much easier to read/distinguish from normal text!
```
#3

That error means that the chunk file 7dd091656d11accb1aea718d882e0b8950b6eb7f2cc52fb7d7c98b2a6aae2256 on the storage sftp://root@10.0.1.64//mnt/Raidz2/duplicacy/ is corrupted.

You might be able to regenerate this chunk if the corresponding file on the source doesn’t change. To do this, first move that chunk file out of the storage directory. Then edit the .duplicacy/preferences file under the source directory and change the id. Now run a backup and since it is a new id it will be an initial backup. Once the backup is done, check if that particular chunk file reappear.

It would be interesting to compare the regenerated chunk file with the original one to see the differences.

Invalid hash while checking chunks
Invalid hash while checking chunks
#4

Gchen, That’s interesting. Ok, first I will run a scrub on the ZFS volume containing my backup. This may flip the bit back that is corrupted in the chunk.

For now, I do not have access to the original volume. I will need to put in a request to have the tapes brought out of storage. This will take a few days. My project does not have priority access. Its why I made a backup. In the meantime, I will wipe the destination and restart with the commands form Cristian. The scrub should take less time then the tapes.

Thank you both for your help. I will keep you posted.

1 Like