Very slow backup retry with only two files skipped after an I/O error

[grbaset]

Hello, I tried using Duplicacy CLI (version 2.7.2, also tried building from source but it failed) to back up my root folder on macOS 10.15.7 (approximately 1.5 TB, with a few million files), using the -vss option and 4:1 erasure coding, but unfortunately at 93.1% it got interrupted by the following error (with {username} being my username):

Failed to read 0 bytes: read /tmp/snp_831848032/Users/{username}/Library/SyncedPreferences/com.apple.security.cloudkeychainproxy3.plist: input/output error
Shadow copy unmounted and deleted at /tmp/snp_831848032 
Incomplete snapshot saved to /Users/{username}/.duplicacy/incomplete 

Then, when I tried again, after indexing it tells me:

Incomplete snapshot loaded from /Users/{username}/.duplicacy/incomplete 
Listing all chunks 
Skipped 2 files from previous incomplete backup 

Despite the fact that way more than two files were backed up. And indeed, the incomplete file only contained two files, the rest being chunk hashes (I’ve seen this issue, but it wasn’t resolved).

Furthermore, this second run was very slow, with only a few chunks being skipped per second, average speed at about 10 MB/s, compared to over 30 MB/s in the initial run at the same point. CPU was almost idle and memory and disk I/O seemed fine. I interrupted this second run at about chunk 3800, and I tried again. This time however, over 300000 files were skipped. However, the third run went as slow as the second, so I interrupted it at about chunk 9600 (incomplete file is now about 300 MB vs 24 MB originally).

[gchen]

You can delete the incomplete snapshot file and try the backup again. Without the incomplete snapshot file Duplicacy will need to read all files again, but it won’t upload chunks that have already been uploaded in previous runs, so the speed will still be much faster than a fresh new backup.