Error finding .fsl on B2

brancaleone · 27 March 2023 15:31

Since a few days i have errors during prune and check commands. It complains that some fossil cannot be found. Here is an example :

Failed to download the chunk 895d92a1528cf9a9253dbfbbb0321bfe1e33e207112662b304fddf64098d4739: URL request 'https://f003.backblazeb2.com/file/cdv-backup/chunks/89/5d92a1528cf9a9253dbfbbb0321bfe1e33e207112662b304fddf64098d4739.fsl' returned 404 File with such name does not exist.

I have check the B2 bucket and indeed the file with .fsl extension does not exist, but the chunk is there with a “hidden” version. From what i understand, the hidden version is the way to mark fossil files on B2 compared to .fsl extension on local drives :

So it seems normal that there is no .fsl files on the B2 bucket.

Previously this B2 bucket has been used as a copy destination from a primary local disk storage. Now it is only used as a direct storage target (for same repository).

I tried to run a prune with -exhaustive and it seemed to fix the issue for a couple of days, but now they are back with new chunks.

Do i get these errors because duplicacy expect the .fsl extension because the chunk was once on a local disk ? How to get rid of those errors ? I could delete oldest snapshots, but how to make sure which ones would cause the issue ?

cooltony · 29 March 2023 04:29

I’m having the same issue.

I started to see similar error log since March 26.
To my believe, it was caused by a backup or prune schedule on March 25, which created several 0 byte files.
I am aware the 0 byte file was intended by duplicacy for fossil collection…etc but only the 0 byte file created on Mar 25 (in my case) is causing the problem.
Because there are no program change on duplicacy side, I am thinking whether a server side change from b2 might caused the problem.

gchen · 30 March 2023 18:10

This commit should fix the issue: B2 backend should be able to download .fsl files · gilbertchen/duplicacy@1f9ad0e · GitHub

What happened was that the chunk in question was turned into a fossil by a prune command which failed to complete, so the revision file wasn’t deleted. But this chunk is a metadata chunk that is needed in order to gather information about the revision. The fix is to find the file id of the correct version and then download the file using the obtained file id (at the cost of one extra api call to b2_list_file_versions, which is a class C transaction).

brancaleone · 31 March 2023 11:50

Thanks for the feedback, currently the issue seems to have gone away after a few more prune -exhaustive runs.
About the failed prune command, that make me think that i am using duplicacy on a macbook and i have noticed that scheduled jobs still run even when the laptop is on sleep. Those jobs either run for a very long time or fail (b2 network call fail at some point). So maybe the failed prune job may be one that was running when the computer was sleeping.
Would it possible to stop all jobs schedule while the computer is on sleep ?

Best Regards

andybao · 2 April 2023 07:00

I believe the same fix should be applied to Google Drive too.

I’m getting the following error every time I run check on my backup that is stored on Google Drive:

2023-04-02 00:13:22.141 WARN DOWNLOAD_FOSSIL Chunk 258d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6 is a fossil
2023-04-02 00:13:22.335 ERROR DOWNLOAD_CHUNK Failed to download the chunk 258d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6: chunks/25/8d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6.fsl does not exist
Failed to download the chunk 258d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6: chunks/25/8d3ffbe7ec08f3a1d213b3404ca5dae0b7710f728dc0535848dc8acd74b8d6.fsl does not exist

See suspicious comment here… duplicacy/duplicacy_gcdstorage.go at e43e848d47176144049b93938f09820c61effc72 · gilbertchen/duplicacy · GitHub

Which seems to contradict the code here: duplicacy/duplicacy_chunkoperator.go at bc2d762e41c5ce11be01bffe32b39edc4d5b2eaa · gilbertchen/duplicacy · GitHub

brancaleone · 17 April 2023 10:02

Can a build be made available for this fix ? I still get those errors quite often and i’m not able to access a machine to build a macos version based on sources.

noah.e.miller · 10 May 2023 15:48

Thank you for committing a fix, @gchen. Just like @brancaleone I would love to see a new release including this fix, because I’m experiencing the same issue. Or in the meantime, can you suggest a workaround?

andybao · 31 July 2023 23:25

Hey @gchen,

Any updates on my Google Drive issue here? It’s been several months and both checks and prunes are still failing. I haven’t had a successful check/prune for nearly 6 months now…

gchen · 3 August 2023 19:49

I’ll run a test myself to confirm/fix the issue. In the meantime you can workaround it by manually moving the chunk from the fossils subdirectory to chunks. There should not be too many of such chunks (only metadata chunks are affected).

C11 · 15 September 2023 03:35

This is a real problem for me too… left a more detailed comment here:

Can we please get a mac release that solves this?
Carl

saspus · 15 September 2023 05:24

The fix seems to be already present in the mainline. You can build it yourself on macOS fairly easily:

Prerequisites:
- Install Xcode Command Line Tools:
```
xcode-select --install
```
- Install Homebrew, in the unlikely event you don’t already have it: https://brew.sh

In the terminal,

Install go:
```
brew install go
```

Fetch top of tree of duplicacy:

git clone http://github.com/gilbertchen/duplicacy /tmp/duplicacy

Build:

cd /tmp/duplicacy
go build -o /tmp/duplicacy_cli duplicacy/duplicacy_main.go

Verify:

% file /tmp/duplicacy_cli
/tmp/duplicacy_cli: Mach-O 64-bit executable x86_64

% /tmp/duplciacy_cli --help
NAME:
   duplicacy - A new generation cloud backup tool based on lock-free deduplication

USAGE:
   duplicacy [global options] command [command options] [arguments...]

VERSION:
   3.2.0 (unofficial)
...

Copy that /tmp/duplciacy_cli to where you need it, and if using WebUI - prevent webUI from overwriting it.

Note, it should build the AS version if building on AS Mac, but if not – you can specify arch explicitly to go build via GOOS and GOARCH env parameters. List of supported architectures you can get with go tool dist list

C11 · 15 September 2023 05:50

I might be able to figure this out… (I’m an old HW guy, I haven’t compiled anything in more than 10 years.)
How would you prevent the webUI from overwriting it?
Regardless… seems like I’m not the only one suffering this, seems like a good reason for a new mac release?

saspus · 15 September 2023 06:18

You definitely will. Go makes it obscenely easy. I’d say, uncomfortably so. No more fiddling with makefiles, c-compilers, linkers, wonky dependencies, and phases of the moon. Literally one command – go build ... … just mind-blowing.

I think there was an option in webUI, but I would just make it readonly. chmod +r /path/to/executable

Maybe gchen wants to test everything more extensively…

C11 · 15 September 2023 17:22

Thank you for your help!

asitdpyjjuurywdczx · 27 September 2023 01:36

This is happening to my spouse’s Windows installation. We do not have the technical expertise to build it ourselves. It took us just an hour+ to find our way here. B2 with a 0 byte .fsl file, though I don’t really understand what I’m looking at with their bucket structure. An update to the .exe with the fix would be super helpful to us regular humans. Thank you.