Feature request: Add "show available storage space" option

Feature
#1

Hi there,

duplicacy is already able and authorized to connect to my storages, it is performing large backups and it is showing stats of the backup.

The only thing missing is, showing the remaining available space of the storage, to avoid a “full disk” on the storage side. Dealing with a “disk full behaviour” afterwards is always a mess. When I read the logs, I want to have all relevant information about the backup and storage situation there. I want duplicacy to be my only connector to the storage and I do not want to manually login and check on a regular basis.

Of course one could say this requirement is already resolved as there should be a warning e-mail of my storage provider, but this could be too late (as it happened to me!) Or one could say this requirement should be resolved by a different service or tool - but still, I am convinced it would be helping a lot, as duplicacy could implement and maintain it very easy and for a lot of storage providers and again - be the one cloud solution in advance of all others.
And relating to the –max-size Option discussed here I would say, showing available space only is the better way of helping to deal with this szenario, cause I need to check the duplicacy logs on a regular basis anyway.

Kind regards,
Manne

#2

Show it where? Would you also want duplicacy to use this information? If so, how?

Why do you need to check duplicacy logs regularly? At least for me, I almost never look at the logs after the first run or so unless there’s a warning, error, or unexpected increase in storage utilization.

Storage provider support

As for support, it’s unclear that it’s even possible to add this for all of the supported storage backends. For instance, from a quick skim of the API documentation I don’t think there’s a concept of available space in a bucket in B2, S3 (and Wasabi), and possibly other object storage providers – though it looks like it could be possible with OpenStack Swift. Support in sftp appears to be spotty.

OneDrive does seem to support showing remaining space determined by the quota. I wasn’t able to quickly find a way in Google Drive.

1 Like
#3

Thanks for your reply and thanks for considering it.

Show it where? Would you also want duplicacy to use this information? If so, how?

In my opinion it makes sense in the backup “stats”. I want duplicacy to retrieve and show it only. From there I could automatically monitor it along with other backup stats.

Backup for /volume1/backups/duplicacy-repo1 at revision 310 completed
Files: 181926 total, 431,204M bytes; 20 new, 27,428K bytes
File chunks: 87840 total, 431,505M bytes; 5 new, 27,428K bytes, 16,934K bytes uploaded
Metadata chunks: 14 total, 62,575K bytes; 6 new, 30,196K bytes, 12,811K bytes uploaded
All chunks: 87854 total, 431,566M bytes; 11 new, 57,625K bytes, 29,746K bytes uploaded
Total running time: 00:01
**Storage status:  state normal, available: 1024GB, used: 975GB**

The last line is my suggestion. The line is shown only, when the storage provider supports it.

Btw: I could also imagine a separated json file for the last backups details (last successfull performed backup timestamp, time-diff between now and last backup, and the stats above to be able to add this to a monitoring.

Why do you need to check duplicacy logs regularly?

I actually do it manually right now, as I have no automated handling for result codes, logs or stats yet.

unexpected increase in storage utilization

This is what I don’t want to check separately from all other issues that can happen.

But I agree, one could just monitor the consumed space of the backup stats instead…but in my case this still not helping if other services use the same storage for their purposes.

If not all storage providers support this by now, maybe OneDrive could be a reference implementation.

#4

I use Rclone about command to get this information about storages usage:

https://rclone.org/commands/rclone_about/

Of course it would be great if this statistic were incorporated at the end of the logs, as you suggested. :yum: