Failed to list the directory chunks/: invalid character '\t' in string literal

Support
#1

It started when a backup was ready, it was running with -stats and just reported 100%.
Then i got this error:
“The chunk id should be xxx instead of yyy, length: 7520321”
I tried to start the backup again, and got:
“Failed to load files specified in the snapshot x as revision 9: invalid character ‘t’ after object key:value pair”
I then deleted the ‘9’ file in the snapshots directory, and tries to run again:
“The chunk xxx load from the snapshot cache has a hash if of yyy. Failed to download the chunk zzz: storage bad CRC on read”
i deleted the xxx chunk and tried to run prune -exhaustive:
“Failed to list the directory chunks/: invalid character ‘\x17’ in string literal”
and that \x17 seems to be different every time.

I am using google cloud storage as backend. Is this google having a bad day and reporting an error that is parsed as something else ?

Thanks!

#2

This is likely caused by corrupt files in the local cache (under .duplicacy/cache). You can try to remove this directory entirely. However, if the problem comes back again you may need to check the disk for errors.

#3

Hi, Thanks for your reply.
I have deleted the cache directory, and checked the disk: no errors, but the issue remains.

It is a virtual machine, and the host is fine too. i have encryption + RSA enabled too.

duplicacy check returns the same error with a different number/character every time, but sometimes it looks like it returns quicker then others.
duplicacy backup sometimes works, other times it fails with errors like “Entry X appears before Y” (Sorry, forgot to copy&paste the full error). but other times it does and completes.

and then another backup fails with “Failed to load files specified in the snapshot X at revision 6: invalid character ‘,’ after object key”. this time its always ‘,’ even when i delete the cache.

Duplicacy was version 2.7.1 , an upgrade to 2.7.2 doesnt change the issue.

Anything else i can test ?

#4

“The chunk id should be xxx instead of yyy, length: 7520321” is reported by the VerifyID() here:

This error means the chunk was somehow modified before the upload. This can be caused by a memory corruption.

The “storage bad CRC on read” error seems to come from here:

This again points to a memory corruption problem. So maybe you should run a memory test on the VM?

#5

This morning i was greeted by a message that there was an update for Virtualbox. i installed it and the issue seems to have gone.

A ran a check -files for a few minutes, and did reveal a number of corrupted chunks so im just gonna start a new backup, just to be sure.

It might be usefull to have an option with check to delete all corrupt files. Then you can run a new backup, which would upload all missing chunks and possibly fix old backups

#6

You seems to be ignoring the words of gchen. Virtualbox update can’t fix memory errors.

If your RAM is faulty, then there is no backup that can save you from data corruption. It is highly recommended that you do a memory test on your computer (the host, not on the VM). Please take your time and read some articles, how serious damage can failing RAM cause.

To test RAM for failures, let the computer execute memtest86 for at least a whole night long. If it doesn’t find any errors, then your hard drive/SSD can also be faulty. For example you can use Hard Disk Sentinel (I’m not affiliated with it) to view the S.M.A.R.T. data for your drives.

1 Like
#7

I ran both, and both did not report any issue.