Cipher: message authentication failed

[Christoph]

Any news? I’m seeing Cipher: message authentication failed again…

[gchen]

This hasn’t been done. I would suggest creating a new pcloud storage with Erasure Coding enabled and then copy over everything. This way you won’t lose any old backups.

[Christoph]

My backend is 4TB of which 3plus TB are used, so I can’t copy the entirety storage into the same backend without deleting the source (which I don’t think the copy command does). So I would have to download the entire storage then do this

@gchen

You can create a new local storage, let’s call it LOCAL2, that is copy-compatible with LOCAL and with erasure coding enabled. Then run a copy job to copy all backups from LOCAL to LOCAL2. After that, if you don’t want to change your existing schedules, you can just delete LOCAL from the storage page, and then create a new storage named LOCAL that points to the same storage url as LOCAL2. Everything should work the same way as before.

And then reupload it all. Is that correct? It would only take a couple of months, I guess.

[Christoph]

I was going to use my previous solution to fix another one of these error messages but I’m failing at step 3:

@Christoph

I was able to fix the problem by

1. manually deleting the corrupted chunk directly on the storage
2. running check
3. searching the check-log for “missing” and noting down which revisions in which snapshots had missing chunks
4. manually deleting those snapshots on the storage.
5. running check again to confirm that it worked

The problem is that there there is no instance of “missing” in the logs of the check -v
command. All i have at the end of the log is:

2024-02-02 00:44:16.582 ERROR DOWNLOAD_CHUNK Chunk 7136a6d409c040396ad421c40f3121e7f12122b1abe14e0f28d3176721235e04 can't be found
Chunk 7136a6d409c040396ad421c40f3121e7f12122b1abe14e0f28d3176721235e04 can't be found

@gchen Has something changed or what am I missing?

Edit: This post gave me the idea that I may have to delete the local cache. So I deleted the 71directory in ~/.duplicacy-web/repositories/localhost/all/.duplicacy/cache/pcloud_sftp/chunksand started a new check. Let’s see if that brings me back on track.

Edit2: Unfortunately, this didn’t change anything. So I’m still wondering how I can find out which snapshots I need to delete in order to get things up and running again.

[Droolio]

Unfortunately, this is still currently a labourious process - event with check -persist - as I found out recently.

You have to use the verbose -v flag (or maybe even debug -d) to get details on what snapshot references the missing chunk. But this may only show one snapshot at a time.

-persist needs to be reworked to cater for all errors - including when it encounters missing metadata chunks, when prior metadata chunks still exist. i.e. when you get an error such as:

Failed to load chunks for snapshot backup at revision 123: unexpected end of JSON input

[gchen]

@Christoph

The problem is that there there is no instance of “missing” in the logs of the check -v
command. All i have at the end of the log is:

2024-02-02 00:44:16.582 ERROR DOWNLOAD_CHUNK Chunk 7136a6d409c040396ad421c40f3121e7f12122b1abe14e0f28d3176721235e04 can't be found
Chunk 7136a6d409c040396ad421c40f3121e7f12122b1abe14e0f28d3176721235e04 can't be found

@gchen Has something changed or what am I missing?

That means the corrupted chunk is a metadata chunk and Duplicacy can’t construct the list of referenced chunks because of that. It doesn’t show the revision number, but usually it is the next revision right after the one that has been checked.

[Droolio]

Can we get check -persist to persist in these circumstances? As per:

Make `-persist` actually persist

Pinging @gchen for visibility. Have an issue with my check failing on my GCD: ... All chunks referenced by snapshot dolores-c_programdata at revision 3358 exist All chunks referenced by snapshot dolores-c_programdata at revision 3370 exist Chunk eaa25cb5e76c31c3218877c6e5b6067d94c01b3516c12bfe6e7e69024847dda3 can't be found Chunk ae84899894827746437a47f475e1400560af3e404acf6eca712b32821335ec84 can't be found Failed to load chunks for snapshot dolores-c_programdata at revision 33…