Progress stats for an active backup

#1

First up, been playing with this for a little bit, seems a brilliant tool, enough that I’m starting to work out how I could start using this to drive regular backups to b2.

Like everyone who was on a recently defunct service that rhymes with “PrashClan”, my initial upload is big and will take days if not weeks to complete.

I’m wondering about two things. If I trigger a backup job, is there any way to check in and see how close to complete it is?
1) While it’s running?
Looks like “-stats” does this - if I have to restart the job partway through, will it accurately reflect progress after the restart?

2) While it's not running?

This would be useful once I’m in steady state and doing incremental backups so I can write a monitor script to confirm that “most” of the data is backed up (and start to warn if I haven’t been able to back up for a while).
It looks like it would be relatively easy to cobble some sort of textual diff in a script that uses the “list -files” option, but it’d be dead handy if there was some sort of (fast) dry-run option for backup that could tell you how much data it would back up if run. Failing that, it would be dead handy if ‘list’ had a ‘most recent snapshot’ option, otherwise I’ll have to always make two calls - one to list all the snapshots, and then another one to list the files in the most recent snapshot.

#2

I don’t work for Acrosync but from what I have observed the “time remaining” calculation is naive. You can tell how close you are in space but not in time. The percentage will be accurate in other words but the time remaining probably won’t be.

It looks at the amount completed / the total size and uses that to calculate a rate. It then divides what’s left by that rate. It doesn’t take into consideration that there are different kinds of work to do and they each have their own natural rate. If a chunk has already been uploaded that proceeds much more quickly than actually uploading the chunk, at least for me. Scanning the filesystem goes at about 35 MB/s while uploading goes at only 1 MB/s.

It’s hard to see how they could completely fix this because they can’t predict which future chunks will or won’t be present in the destination in the general case. If they gave you both rates you could decide for yourself whether to be optimistic or pessmistic.

#3

If you’re a programmer the cat output might be more useful. If you give it no arguments it dumps the snapshot manifest in JSON. I believe it dumps the most recent snapshot but I can’t tell for sure because I currently only have a single snapshot.

#4

Here is a discussion about the progress stats: https://github.com/gilbertchen/duplicacy/issues/128