I goofed on a filter and ate up my offsite storage. Expected ~3tb, currently taking ~5tb.
I removed the problem backup and am running prune -exhaustive, but the thing’s been sitting for 36h now with nothing to show on the progress bar. Clicking the bar opens a new browser tab, but that never loads anything.
Curious how long this process is expected to take? I back up to google drive, so I have 14 days to get this sorted before it gets upset at me.
Depends on a backend. Google drive at the sizes you listed could take a couple of weeks.
To prune duplicacy needs to enumerate all chunks. Google drive and other drive services are not designed for nor optimized for the usecase of storing millions of objects.
Right on, thank you!
The real problem here is that Duplicacy is missing a progress bar for too many things, I also need to prune something currently and I’ve been waiting a week so far and I still have no idea how long it will still take… Would it really be so hard to add some kind of “Pruned Revision 5/10000” progress bar?
At that stage it has not started pruning. It is enumerating existing chunks on the storage. It asks Google — give me list of files. Google says: here is first part of the list. There is more. Duplicacy says: give me more. Google says: here is next part. Etc. until the end of the list. It’s called pagination.
According to my understanding of Google drive api that’s why it’s progressless. Duplicacy simply does not know how many more pages it will get.
If duplicacy could ask Google — how many objects are in that folder first — then it could show progress bar. But as far as I know Google drive does not have that api.
Thai is yet another reason why drive type endpoints is a poor choice in this usecase.
I know you like to make everything about “drive type endpoints bad”, but I’m not talking about that stage of the process - the “pagination” is quick enough for me, no progress bar is really needed for that, it spends roughly an hour with “getting ready for the actual deletion”. What I have now spend a week on already is this step :
INFO CHUNK_DELETE The chunk xyz1 has been permanently removed
INFO CHUNK_DELETE The chunk xyz2 has been permanently removed
INFO CHUNK_DELETE The chunk xyz3 has been permanently removed
…
Millions of those messages. And I’m quite sure that while it’s doing that, Duplicacy internally has a list of all the chunks it still needs to remove that it could tell me about. It loops over the list and removes them one by one, but it never tells the user how many more chunks it still wants to remove.
Agreed.
But on the other hand, this taking weeks, progress bar or not, is a direct consequence of
Hence why I keep repeating it as a broken record.