Error finding .fsl on B2

[brancaleone]

Since a few days i have errors during prune and check commands. It complains that some fossil cannot be found. Here is an example :

Failed to download the chunk 895d92a1528cf9a9253dbfbbb0321bfe1e33e207112662b304fddf64098d4739: URL request 'https://f003.backblazeb2.com/file/cdv-backup/chunks/89/5d92a1528cf9a9253dbfbbb0321bfe1e33e207112662b304fddf64098d4739.fsl' returned 404 File with such name does not exist.

I have check the B2 bucket and indeed the file with .fsl extension does not exist, but the chunk is there with a “hidden” version. From what i understand, the hidden version is the way to mark fossil files on B2 compared to .fsl extension on local drives :

So it seems normal that there is no .fsl files on the B2 bucket.

Previously this B2 bucket has been used as a copy destination from a primary local disk storage. Now it is only used as a direct storage target (for same repository).

I tried to run a prune with -exhaustive and it seemed to fix the issue for a couple of days, but now they are back with new chunks.

Do i get these errors because duplicacy expect the .fsl extension because the chunk was once on a local disk ? How to get rid of those errors ? I could delete oldest snapshots, but how to make sure which ones would cause the issue ?

[cooltony]

I’m having the same issue.

I started to see similar error log since March 26.
To my believe, it was caused by a backup or prune schedule on March 25, which created several 0 byte files.
I am aware the 0 byte file was intended by duplicacy for fossil collection…etc but only the 0 byte file created on Mar 25 (in my case) is causing the problem.
Because there are no program change on duplicacy side, I am thinking whether a server side change from b2 might caused the problem.

[gchen]

This commit should fix the issue: B2 backend should be able to download .fsl files · gilbertchen/duplicacy@1f9ad0e · GitHub

What happened was that the chunk in question was turned into a fossil by a prune command which failed to complete, so the revision file wasn’t deleted. But this chunk is a metadata chunk that is needed in order to gather information about the revision. The fix is to find the file id of the correct version and then download the file using the obtained file id (at the cost of one extra api call to b2_list_file_versions, which is a class C transaction).

[brancaleone]

Thanks for the feedback, currently the issue seems to have gone away after a few more prune -exhaustive runs.
About the failed prune command, that make me think that i am using duplicacy on a macbook and i have noticed that scheduled jobs still run even when the laptop is on sleep. Those jobs either run for a very long time or fail (b2 network call fail at some point). So maybe the failed prune job may be one that was running when the computer was sleeping.
Would it possible to stop all jobs schedule while the computer is on sleep ?

Best Regards

[andybao]

I believe the same fix should be applied to Google Drive too.

I’m getting the following error every time I run check on my backup that is stored on Google Drive:

2023-04-02 00:13:22.141 WARN DOWNLOAD_FOSSIL Chunk 258d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6 is a fossil
2023-04-02 00:13:22.335 ERROR DOWNLOAD_CHUNK Failed to download the chunk 258d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6: chunks/25/8d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6.fsl does not exist
Failed to download the chunk 258d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6: chunks/25/8d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6.fsl does not exist

See suspicious comment here… duplicacy/duplicacy_gcdstorage.go at e43e848d47176144049b93938f09820c61effc72 · gilbertchen/duplicacy · GitHub

Which seems to contradict the code here: duplicacy/duplicacy_chunkoperator.go at bc2d762e41c5ce11be01bffe32b39edc4d5b2eaa · gilbertchen/duplicacy · GitHub

[brancaleone]

Can a build be made available for this fix ? I still get those errors quite often and i’m not able to access a machine to build a macos version based on sources.

[noah.e.miller]

Thank you for committing a fix, @gchen. Just like @brancaleone I would love to see a new release including this fix, because I’m experiencing the same issue. Or in the meantime, can you suggest a workaround?