Duplicacy new install 6.9TB backup showing 22 days to Complete with 4 threads into Backblaze b2

Support
#1

Greetings from Chicago:
I am trying to backup 6.9TB from a local drive to b2 bucket (backblaze) and it is showing 22 days. How do I optimize the backup speed ? Increase the number of threads?

Also, how do I schedule the backup to run only after business hours (from 7pm to 7am).
I am on a trial version now

https://imgur.com/a/6k5HCDq

I looked at the b2 bucket and it shows a lot of chunk files - does it wait to convert those chunks into proper file names after the backup? What happens if the backup fails after 15 days, would it not have done the saving of the files that has been in chunks ?

Thanks

#2

That’s one way to do it. For me - I managed to double my upload speed by using a VPN (not gonna give its name here so it doesn’t become an “advertisement”)
Another important thing is to make sure the region (of the B2 account) is as close to you as possible : i.e. US if you’re in the US, europe if you’re in europe (or close to it), etc.

No, that’s the way the de-duplicative backup process of Duplicacy works: it chops up files into “chunks” so that it can more efficiently prevent full re-upload of files that are too similar (or identical) to files that have already been backed up.

When you restore the backup - you will see the actual “file tree” hidden behind all these chunks and would be able to choose what to restore and where to restore to

If you want the files to exist “normally” in the destination you need a copy or a sync tool and not a deduplicative backup like Duplicacy.

It is my understanding that the backup process can be paused or even cut off in the middle and then resumed (but maybe more knowledgeable people can give out more accurate info).
I personally have stopped the backup many times in the middle and then resumed and all seemed to be OK.

#3

Correct. If backup is interrupted in the middle of the upload, when it resumes the chunks that were already uploaded will be used and backup will continue from that point. The revision (as the snapshot is called in the Duplicacy nomenclature) will only be “committed” at the end of the upload of all chunks;

#4

Thank You Klayhamn
I have increased the threads to 8. I will keep you posted on the progress

#5

Ok, I increased the thread to 8 and started the backup again. Then while it was showing as reindexing, I stopped the job to increase it to 16. Now the whole job is missing!
How did that happen? is there a way to bring it back to the original backup job/status - that has done 1.3TB already?

#6

I started another job with 16 threads - see below
https://imgur.com/a/Ua8M84c

What happens to the partial packup with 4 threads that disappeared?

Thanks!

#7

Good Morning Klayhamn
The 16 thread job is progressing fast - it shows about 1 day 15 hours remaining.
What would happen to the 4 thread job that i changed to 8 thread and it disappeared? would the chunks be left in the bucket?

Also, when this 16 thread job is finished, how would i set the daily backup going forward on scheduled between 7pm and 6am Monday-Sat and all day Sunday?

Thanks!

#8

I don’t know what caused the 4 thread job to disappear, but chunks had already been uploaded any interrupted job will be reused in new uploads, so you don’t need to worry about this. If you’re unsure you can run a prune job with -exhaustive after the initial backup is done to remove any unreferenced chunks.

You can create two schedules, one enabled on Monday-Sat and the other on Sunday only, and then add the backup job to each schedule.

#9

Thank You Gchen.

The backup was showing as remaining 1 day yesterday. Today when I checked it shows 3 days
How is that happening?

Also, how can I pause the backup during business hours? there is no pause button only STOP button.

Thanks
Shah

#10

how can I pause the backup during business hours? there is no pause button only STOP button.

Thanks
Shah

#11

There is no pause. However, if you stop and then restart later, it will effectively resume from where you stopped before.

#12

Thank You Sevimo,
How about a scheduler - where can i find the scheduler ? Does the backup stop/pause once it is outside the scheduled times?

#13

Thank You All, I will close this ticket and start a new one with appropriate subject line.