Slow B2 download/restore due to -threads not working correctly

Bug
,
#1

I’m restoring with -threads 12 but duplicacy is only downloading with 1 thread. The speed is under 3 MB/s (24 Mbit/s) despite my connection being capable of 200 Mbit/s. I have also checked my router and confirmed that there is only 1 connection/session to Backblaze instead of 12. So it doesn’t seem like -threads is working correctly for restores. Note that when I backup/upload with -threads 12, I can see 12 connections at the router.

VERSION:
3.2.3 (254953)

COMMAND:
duplicacy -d restore -r 1037 -threads 12 -stats -overwrite -delete

#2

According to this thread multithreaded Backblaze restore worked in may of 2023:

Maybe a regression? Can you try with older duplicacy version?

#3

I went back to 3.1.0, and it’s the same behavior. Here’s a snippet of the debug log in case it helps:

Downloading backup/.chunks/d14a/d14aeaab7c99f1baed86622cc5dc6e71eb657dba0464c215c612c54a637f2155
Prefetching backup/.chunks/d14a/d14aeaab7c99f1baed86622cc5dc6e71eb657dba0464c215c612c54a637f2155 chunk 5008b9cceba9e53af30a1787c5381
33973e4d303550b2972c975be6f9428ce38
Updating /aux/encrypted_data/b2_pbs/backup/.chunks/d14a/d14aeaab7c99f1baed86622cc5dc6e71eb657dba0464c215c612c54a637f2155 in place
Downloaded chunk 992 size 3980343, 1.97MB/s 16:59:03 2.6%
Downloading backup/.chunks/d14b/d14b1cb4e8a662d27b1b0f347ffa1ae35228b826f24e0e269a651f578f50c764
Prefetching backup/.chunks/d14b/d14b1cb4e8a662d27b1b0f347ffa1ae35228b826f24e0e269a651f578f50c764 chunk 7264318f312a3b07d39c70fe05a96
d1eaf6c2311e32fe6ef126a4ed84e260b40
Updating /aux/encrypted_data/b2_pbs/backup/.chunks/d14b/d14b1cb4e8a662d27b1b0f347ffa1ae35228b826f24e0e269a651f578f50c764 in place
Downloaded chunk 993 size 2494122, 1.98MB/s 16:58:18 2.6%
Downloading backup/.chunks/d14b/d14b7de2db6df2d62b7bd61b3a6f6538d458883681fd3f59bc5090b56ddc8f5a
Prefetching backup/.chunks/d14b/d14b7de2db6df2d62b7bd61b3a6f6538d458883681fd3f59bc5090b56ddc8f5a chunk a5c5aa7ceb006f6e516497c04e56f
b4a9f066d40be03dd067c778a002e00e27c
Updating /aux/encrypted_data/b2_pbs/backup/.chunks/d14b/d14b7de2db6df2d62b7bd61b3a6f6538d458883681fd3f59bc5090b56ddc8f5a in place
Downloaded chunk 994 size 2442994, 1.97MB/s 16:58:46 2.6%
#4

@saspus Any assistance on this? It’s taking me a long time to restore. Thanks.

#5

I’m out of ideas. Maybe @gchen can suggest something? Why would duplicacy ignore threads parameter on restore?

#6

Like backup, restore only works on one file at a time. The CLI does not ‘prefetch’ as many chunks as specified by the -threads option. Instead, it will only download chunks needed by the current file that is being restored. Therefore, there may not be enough parallelism to make all downloading threads busy.

The best way to speed up the restore option is to copy the b2 storage to a local storage and then restore from the local storage. This of course assumes that you have enough space on your local disk.

Restore single-threaded?
#7

This looks like unnecessary synchronization — why wait for the first file to download before scheduling the next, leaving thread pool idle…

This is of course will be prohibitively expensive; but I’m wondering if it’s worth trying to mount the remote storage with local cache, e.g. with rclone mount, instead. But perhaps it will hit the same limitation.

#8

This is of course will be prohibitively expensive; but I’m wondering if it’s worth trying to mount the remote storage with local cache, e.g. with rclone mount, instead. But perhaps it will hit the same limitation.

I just gave this a quick try with rclone mount, and it’s actually far slower than duplicacy accessing B2 directly.

Downloaded chunk 1 size 321, 222KB/s 48 days 11:56:05 0.0%
Downloaded chunk 2 size 3602553, 231KB/s 46 days 13:00:11 0.0%
Downloaded chunk 3 size 3767721, 245KB/s 43 days 22:58:59 0.0%
Downloaded chunk 4 size 1834820, 251KB/s 42 days 19:39:35 0.0%
Downloaded chunk 5 size 7337331, 261KB/s 41 days 02:59:52 0.0%
Downloaded chunk 6 size 2553437, 271KB/s 39 days 15:32:06 0.0%
Downloaded chunk 7 size 1980889, 277KB/s 38 days 20:57:44 0.0%
Downloaded chunk 8 size 7666199, 271KB/s 39 days 16:20:42 0.0%
Downloaded chunk 9 size 1128850, 265KB/s 40 days 14:42:39 0.0%
1 Like