Best way to backup a repository piece by piece?

Support
#1

Hi there,
I’m interested in Duplicacy mostly because of how it’s customizable and easy to migrate backups.

Here is what I want to do and I would like to know the best way to do it safely:
Let’s say I have a 3To computer drive I want to backup to Gdrive. This computer is not connected to a very fast internet connexion, so I have decided to make the initial backup to a local Synology Nas and then connect this Nas to a faster internet provider and upload from there to Gdrive.

I don’t have enough room on the nas so I have decided to do the initial backup piece by piece. Is this the correct way to do it:

1 - I select the entire 3To repository but I exclude half the folders so I have enough room to backup a first batch to the nas.

2 - I backup the first half to the local nas.

3 - I upload this first batch to Gdrive.

4 - I delete the chunks on the nas to free up space (can I delete everything or do I have to keep some specific files?)

5 - I select the other half of the 3To repository and exclude the first part already backed up and send it to the nas.

6 - I upload this other half to Gdrive in the same folder.

7 - Once the initial backup is done, I change the backup destination on my computer from local nas to Gdrive folder, so it’s now linked to the cloud.

Is this a correct way to do it? Or is it impossible because of folder names and other stuff I didn’t think of? Any tips?
Thanks a lot for your help!

#2

That should work, but in step 5 you should use a new repository id (by manually editing the .duplicacy/preferences file), otherwise Duplicacy will attempt to load the last backup which is now broken since all chunks are gone. If you use a new repository id there aren’t previous backups so Duplicacy will start a new initial backup.

#3

Thanks.
So does it mean I will have 2 different repositories after that and I will always have to run 2 separate tasks in the future to save the entire 3To drive?
Also I have concerns about potential conflicts with the folders in the chunks directory: is running a brand new backup task going to create again new folders labeled 00, 0a, 0b… ? If so, do I just need to merge their contents?
I’m definitely going to try with a test repository first, I’m still a noob in Duplicacy :slight_smile:

#4

After step 7, you can remove the exclude patterns and run a full backup to Gdrive, and this backup will be much faster because most of the chunks already exist in the storage. You only need to run 1 task to backup the entire directory.

On the Gdrive storage there will be two subdirectories (named after the repository ids) under the snapshots directory. The first one was created by the first upload from the NAS to Gdrive, the second one is used by the second upload from the NAS to Gdrive as well as the full backup from your computer to Gdrive. You don’t need the first one, so you can remove the first subdirectory in the Gdrive web UI.

But then you may have some unreferenced chunks in the storage. You can clean it up by running:

duplicacy prune -exhaustive -exclusive -dry-run  # make sure it doesn't remove too many chunks
duplicacy prune -exhaustive -exclusive  # now actually remove all unreferenced chunks

Also I have concerns about potential conflicts with the folders in the chunks directory: is running a brand new backup task going to create again new folders labeled 00, 0a, 0b…

No, there is always only one chunks directory on the storage so there is no need for merging.

#5

This is precious, thank you for your help!
I’m appreciating your software more and more.

Just one last question because this isn’t crystal clear to me:

You say there is only one chunks directory. This I understand. But inside this directory there are dozens of 2 letters folders. If I create one backup task and then delete the local version and start the same task with different excluded directories, it will create identical folders inside “chunks”, right? So I would have to merge contents of the 2 versions. Or am I wrong?

#6

The first upload and the second upload will upload to the same chunks directory so if a folder already exists from the first upload, the second upload will reuse that folder.