Prune: fossil folder vs .fsl extension

TheBestPessimist · 19 September 2018 07:44

In my google drive storage i have a folder named fossils which has 40+ GB of files w/o any .fsl extension. It has the same folder name format like chunks eg./fossils/00/chunkhash, /fossils/2e/chunkhash.

But i also have files in the chunks folder which have the extension .fsl.

The earliest file in fossils is from 31 dec 2017

While the latest is from 11 march 2018

And this is what google drive web shows when selecting the folder:

And these are some .fsl files in /chunks/

My questions are: why is there a need for 2 different fossil types, how are they different, and how were they created?

Christoph · 19 September 2018 08:23

You forgot to ask the question So: what’s the difference between the two types of fossils? Why do they exist?

Droolio · 19 September 2018 14:45

Found this comment in the relevant source code, which might begin to explain:

github.com

gilbertchen/duplicacy/blob/0af74616b72391958393dd65ad341e8cbd8f82f7/src/duplicacy_gcdstorage.go#L127-L129


// convertFilePath converts the path for a fossil in the form of 'chunks/id.fsl' to 'fossils/id'.  This is because
// GCD doesn't support file renaming. Instead, it only allows one file to be moved from one directory to another.
// By adding a layer of path conversion we're pretending that we can rename between 'chunks/id' and 'chunks/id.fsl'

Unsure if it uses both locations and just moves fossilised chunks between them for sake of being able to rename chunks into fossils - at least, it seems to be able to deal with both locations.

For why you still have a lot of fossils, are any of your repositories not being backed up? I gather it’ll only delete them if subsequent backups from all repositories are being made… so it can confidently delete the collected fossils (by excluding any chunks that feature in future backups from that collection).

You should have at least one fossil collection file in /fossils - with filename of say "1" - which lists the chunks in that collection. If you don’t, that file may have been purged and those excess fossils are now unreferenced, so you may need to run prune with -exchaustive. (I’d definitely be inclined to run a check -all first though )

gchen · 19 September 2018 14:59

The file name extension .fsl is for fossil files in a disk-based storage. In the Google Drive backend, all fossils are moved to the fossils directory without changing the names. I guess at one point you ran the prune command with the storage mounted as a local drive so it created those .fsl files there.