Cache folder is is extremely big! 😱

How-to
, ,
#1

When you prune any of the repositories, :d: still needs to download all existing snapshot files for all repositories (not the whole data, just the chunks which store each revision details – the meta chunks). The /cache/chunks/ folder is used to store only these meta chunks that make up the snapshots.

If there are very many revisions or if some repositories contain large number of files, then the /cache/chunks/ folder can grow to a surprisingly large size, reaching even more than 150GB.

This is recommended to run prune from only a single repository, and use the -all option:

duplicacy prune -all -keep 0:365 -keep 30:70 -keep: 7:14 -keep 1:1 -threads 30
1 Like
Cache directory clean up clarification
Newbie: Recommended usage of --repository and --pref-dir
Limiting disk usage during `duplicacy check -chunks`
Cache usage details
#3

As a palliative solution, it’s been a while since I’ve included a Rclone call in my scripts (rclone delete) with the pattern + */cache/**, pointed to the folders of Duplicacy files (preferences, filters, cache, etc).

Remembering that I have a bucket in B2 for each set of backups, so I have a more granular control over the executions, prunes, etc. This is one of the many advantages, IMHO, of having separate backups for each set of files, rather than large backups with all files.

#4

The /cache/chunks/ folder can be deleted any time. It will only slow down the next operation a little bit as some chunks will need to be downloaded again instead of being loaded from the cache. Other than this there is nothing else to worry about.

Should this how-to be combined with Cache usage details?

1 Like
#7

Could you clarify what “some” means here? My optimistic reading is: there may be a large number of chunks in the cache which are (almost) never needed in practice, so their deletion will not matter. But some very few chunks are needed and will have to be redownloaded. Is this correct?

#8

Does this mean we should run the prune command always with the -all flag?

#9

I was a bit confused when trying to run the example command from the original post since there is a bug with a colon after the 3rd keep, a fixed version can be found below:

duplicacy prune -all -keep 0:365 -keep 30:70 -keep 7:14 -keep 1:1 -threads 30
1 Like