Initial backup very slow

Support
,
#1

I’m trying to use Duplicacy to backup the entire file system on a fairly bare-bones Ubuntu 18.04 VM. I have initialized backup storage for repository “/” on a local disk mounted at “/backup” and excluded that directory from the backup. I have encryption enabled.

Running the backup says it will take about 9 days to backup the 5GB worth of files on this machine. Is that a normal time period? I see duplicity slamming the CPU with 100% usage during the backup but very little disk activity. After leaving it overnight for about 12 hours it has created about 1GB of chunks.

Another strange thing I noticed is that with -stats I can see that duplicacy is skipping every chunk during the backup. With -d and -v flags I see the hash for every chunk is the same. I have restarted the backup a few times playing around with -threads but it seems to make no impact on performance.

Skipped chunk 2f0c734ef3c69b4557991cb467d028a20f9c7d9574fced3cae269e6c987a246b in cache
Skipped chunk 455187 size 1048576, 158.27MB/s 9 days 18:46:08 0.3%
Skipped chunk 2f0c734ef3c69b4557991cb467d028a20f9c7d9574fced3cae269e6c987a246b in cache
Skipped chunk 455188 size 1048576, 158.27MB/s 9 days 18:46:06 0.3%
Skipped chunk 2f0c734ef3c69b4557991cb467d028a20f9c7d9574fced3cae269e6c987a246b in cache
Skipped chunk 455189 size 1048576, 158.27MB/s 9 days 18:46:05 0.3%
Skipped chunk 2f0c734ef3c69b4557991cb467d028a20f9c7d9574fced3cae269e6c987a246b in cache
Skipped chunk 455190 size 1048576, 158.27MB/s 9 days 18:46:03 0.3%
Skipped chunk 2f0c734ef3c69b4557991cb467d028a20f9c7d9574fced3cae269e6c987a246b in cache
Skipped chunk 455191 size 1048576, 158.27MB/s 9 days 18:46:01 0.3%

Is something going wrong here or am I just being impatient? Here’s the output of my benchmark:

Storage set to /backup
Generating 244.14M byte random data in memory
Writing random data to local disk
Wrote 244.14M bytes in 0.62s: 396.55M/s
Reading the random data from local disk
Read 244.14M bytes in 0.08s: 3009.07M/s
Split 244.14M bytes into 43 chunks without compression/encryption in 2.72s: 89.60M/s
Split 244.14M bytes into 43 chunks with compression but without encryption in 4.09s: 59.69M/s
Split 244.14M bytes into 43 chunks with compression and encryption in 4.07s: 59.98M/s
Generating 64 chunks
Uploaded 256.00M bytes in 0.86s: 298.81M/s
Downloaded 256.00M bytes in 2.07s: 123.70M/s
Deleted 64 temporary files from the storage
#2

Alright, apparently I just rubber-ducky’ed myself here. I found the problem immediately after posting this and asking for help.

Adding the following to my filters fixed the problem immediately:

-proc/
-dev/
-sys/

Now here are my results:

Backup for / at revision 1 completed
Removed incomplete snapshot /.duplicacy/incomplete
Files: 109040 total, 5,425M bytes; 94779 new, 2,434M bytes
File chunks: 2710 total, 5,444M bytes; 407 new, 2,230M bytes, 1,274M bytes uploaded
Metadata chunks: 8 total, 37,067K bytes; 8 new, 37,067K bytes, 11,057K bytes uploaded
All chunks: 2718 total, 5,481M bytes; 415 new, 2,266M bytes, 1,285M bytes uploaded
Total running time: 00:02:40
302 files were not included due to access errors