Backup to S3 with -threads leads to send request error

Support
#1

Hi,
I am using a S3 backup as storage. When I run the following command, i get this error:
$ duplicacy backup -threads 10

Packed Folder/folder/file … (some 100 lines)
Failed to upload the chunk 93dcd3971984116c818aa866e7aaca867dce8d492e6538d28dc03566f3ff6f52: RequestError: send request failed
caused by: Put https://NAME.s3.eu-central-1.amazonaws.com/chunks/93/dcd3971984116c818aa866e7aaca867dce8d492e6538d28dc03566f3ff6f52: dial tcp: lookup NAME.s3.eu-central-1.amazonaws.com on 192.168.178.1:53: server misbehaving

When i run the backup command without threads it works but quite slow.
I am testing currently directly on Synology NAS DS 412+.
2 CPU (INTEL Atom D2700) / 4GB RAM

#2

Depending on your upload bandwidth, 10 threads may be too many.

You can run the CLI benchmark command with different thread numbers to figure out the optimal value.

#3

Hi
thanks for that hint.

I tried again with only 2 threads.

duplicacy backup -threads 2 -stats
…
Uploaded chunk 1010 size 1584296, 3.23MB/s 00:34:04 42.4%
Uploaded chunk 1008 size 13378972, 3.22MB/s 00:34:03 42.6%
Uploaded chunk 1011 size 4183564, 3.22MB/s 00:34:05 42.6%
Failed to upload the chunk 08f595bcfc226c510f235e8e0c0e17e4ff8ea11cd5cd08e1dee62d636bb64df4: RequestError: send request
failed
caused by: Put https://BUCKET.s3.eu-central-1.amazonaws.com/chunks/08/f595bcfc226c510f235e8e0c0e17e4ff8ea11cd5cd08e1d
ee62d636bb64df4: dial tcp: lookup BUCKET.s3.eu-central-1.amazonaws.com on 192.168.178.1:53: server misbehaving

For me this looks like some random error while uploading to s3. Perhpas it could help to retry 2-5 times on error wihout canceling directly

I am not sure if this have something todo witht he error but I noticed a decrease of the uploadspeed…

Skipped chunk 3 size 13969581, 11.08MB/s 00:17:13 0.1%
Skipped chunk 76 size 5892025, 8.47MB/s 00:21:53 3.0%
Uploaded chunk 857 size 5565565, 8.75MB/s 00:13:58 36.1%
Uploaded chunk 890 size 1664384, 5.71MB/s 00:20:51 37.7%
Uploaded chunk 942 size 3195345, 4.14MB/s 00:27:50 39.6%
Uploaded chunk 1003 size 1060948, 3.29MB/s 00:33:40 42.0%