Multi-tier chunks?

Support
#1

Probably a question for @gchen again :wink: So :d: supports multiple levels for chunk directory (though by default it is only 1). I can see it might be useful for storages that have some limits on number of entries in a single folder (are there any other use cases for that)? That’s not my question though. I was looking at prune implementation, and there is some non-trivial logic of handling scenarios with redundant chunks, e.g. the same chunk appearing at different levels (so D3/495A8D… and D3/49/5A8D…). The question is, how can it reasonably happen in the normal course of operation (so no one copies files manually etc)? I thought chunk levels are fixed at storage creation, and once they are, a chunk can only go to a single level determined by write level.

So how can something like that happen? Trying to find out why this functionality is needed in prune.

#2

You can change the nesting levels which are set in the file named nesting. So theoretically it is possible that different backup clients may choose different levels if not every client starts at the same time, although this is very unlikely to happen in real-world.

#3

Got it. Will see if there are notable gains to be had if assuming no changes to nesting levels (as would be the vast majority of cases), to simplify/avoid FindChunk in these scenarios.