I’ve got a single computer with a bunch of backups, roughly 1.3TB of data. I then want to check that all the backups are ok.
Running duplicacy check -all -files seems very slow.
Listing all chunks
19 snapshots and 817 revisions
Total chunk size is 1387G in 311396 chunks
Then after 10 hours it’s only got through about 1 and 1/2 of the snapshots.
CPU looks idle:
Tasks: 939 total, 1 running, 938 sleep, 0 d-sleep, 0 stopped, 0 zombie
%Cpu(s): 3.0 us, 0.5 sy, 0.0 ni, 96.2 id, 0.1 wa, 0.2 hi, 0.0 si, 0.0 st
MiB Mem : 257598.0 total, 231542.0 free, 22012.5 used, 6507.7 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 235585.5 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
21258 root 20 0 5505376 350824 20256 S 156.8 0.1 16,10 duplicacy
iotop shows extremely low I/O:
21293 be/4 root 6.35 M/s 0.00 B/s ?unavailable? duplicacy check -all -files -stats
21297 be/4 root 9.46 M/s 0.00 B/s ?unavailable? duplicacy check -all -files -stats
21303 be/4 root 1667.52 K/s 0.00 B/s ?unavailable? duplicacy check -all -files -stats
21306 be/4 root 5.41 M/s 0.00 B/s ?unavailable? duplicacy check -all -files -stats
21307 be/4 root 4.84 M/s 0.00 B/s ?unavailable? duplicacy check -all -files -stats
21311 be/4 root 20.73 M/s 0.00 B/s ?unavailable? duplicacy check -all -files -stats
21315 be/4 root 13.41 M/s 0.00 B/s ?unavailable? duplicacy check -all -files -stats
There is probably another 300MB/sec available for reads.
So I’ve got low CPU, low read speeds. I guess I could manually parallelise - is that the best option?
But also is this efficient? Is it re-reading the same chunks multiple times?