Am I doing this right?

Support
, , ,
#3

Several strategies can be used:

  1. Set up a repository for each folder and run a backup job for each of them, as you described;

  2. Configure a unique repository in the “root” folder (/mnt/user/ in your case) and select subfolders to be backed up with filters and include patterns;

  3. Use symbolic links, as suggested by @TheBestPessimist;

To choose the best strategy must be considered aspects such as:

  • number and size of files and folders;
  • need (or not) of different backup configurations (size of chunks fixed or variable, same or diferentt storages, etc.);
  • need (or not) of different schedules;
  • Frequency of source files updating.

I particularly use the first option because it allows me a more granular control over what will be backed up and when it will be backed up. But that depends on each use case.

Take a look at the -repository option at the end of this page:

#4

First off I want to say thanks!!

I managed to get this working with

duplicacy init -e -repository /mnt/user/Downloads/Manual/Test/ -pref-dir /usr/local/duplicacy/TDS TDS gcd://Backup/TDS

Before I read your message, I would CD into the directory and then initiate the backup command. How do I do this now that I am not changing directories?

Edit: Nevermind, I just type duplicacy backup and it runs. I am going to try this on multiple directories

#5

Ok I am really confused now.

I ran the following two commands

duplicacy init -e -repository /mnt/user/Downloads/Manual/Test2/ -pref-dir /usr/local/duplicacy/TDS2 TDS2 gcd://Backup/TDS2

duplicacy init -e -repository /mnt/user/Downloads/Manual/Test/ -pref-dir /usr/local/duplicacy/TDS1 TDS1 gcd://Backup/TDS1

So now I have 2 directories that I want to backup “Test” and “Test2”. I cannot have them in the same -pref-dir? Also can I place them in the same location gcd://Backup/TDS or does that need to be unique too?

Finally, how do I initiate the backup for each of these repo’s without CD-ing into the directory? Thanks for your help, where do I donate for beer or coffee?

#6

Ah yes, this seems to be working. I really wish I could just specify the folders I wanted to backup without going through all this symlink information. Borg does a great job at this with a bash script. Is there a way to bash script this so I can just deploy the script to lets say 20 identical machines without having to symlink stuff?

#7

But using symlink is really simple: save all the ln -sf /source/folder /destination/folder/link in a duplicacy-setup-script.sh and just run it on all the machines.

Afterwards you just init a single repository on each of those machines, in the folder where you symlinked everything.

1 Like
#8

If you use the -repository option then -pref-dir isn’t needed and you can avoid symlinks by initializing a pseudo repository for each directory to be backed up:

cd /mnt/user/Duplicacy/repository1
duplicacy -init -repository /mnt/user/Downloads/Manual/Test1 repository_id storage_url

cd /mnt/user/Duplicacy/repository2
duplicacy -init -repository /mnt/user/Downloads/Manual/Test2 repository_id storage_url

Here /mnt/user/Duplicacy/repository* are the pseudo repositories and the real directories to be backed are /mnt/user/Downloads/Manual/Test1*. This strategy also has the advantage that all .duplicacy directories will be created under /mnt/user/Duplicacy.

3 Likes
How to initiate a backup for multiple repositories?
#9

Exactly, that is the strategy I use, with all Duplicacy settings centralized in a folder, outside the repositories.

#10

This is perfect!!! Yeah I really like your software, just tricky setting up on unRAID and the .duplicacy files were driving my OCD off the wall. Thank you for taking the time to reply!

#11

A post was split to a new topic: How to initiate a backup?

#12

For anyone: Feel free to use the :heart: button on the posts that you found useful.

For the OP of any support topic: you can mark the post that solved your issue by ticking the image box under the post. That of course may include your own post :slight_smile:

#13

Hey gchen, that -repository option is giving me “incorrect usage” Is this available on the CLI version for linux_64 2.1.0 ?

#14

Please upgrade to 2.1.2. Version 2.1.0 is way too old!

#16

Awesome, thanks! Did not see that before.

1 Like
#17

Hello @TheBestPessimist

Per @gchen 's solution for -repository. Can I back multiple directories to the same repository?

IE:

duplicacy -init -repository /mnt/user/Downloads/Manual/Test1 REPOID1 STORAGE1
duplicacy -init -repository /mnt/user/Downloads/Manual/Test2 REPOID1 STORAGE1

If not, what would be the best way to put everything in the same destination folder under the same repo?

Thanks again, I have my script generated and running. I am just getting into the intricate portions.

#18

Just to be clear here:

  • repository means the folder you are backing up right now (the one which has .duplicacy folder)
  • repository id is the name of a repository (should be unique for each of your folders)
  • the storage is the destination of your backup

You can backup multiple repositories to the same storage as long as they have different repository ids and this is indicated as you get the benefit of deduplication.

So for your example:

  • repository: /mnt/user/Downloads/Manual/Test1 and /mnt/user/Downloads/Manual/Test2
  • repository id: REPOID1
  • storage: STORAGE1

The problem with your example is that for 2 different folders to backup (which represent 2 different repositories) you have the same repository id.

What you want is actually to create 2 repositories:

duplicacy -init -repository /mnt/user/Downloads/Manual/Test1 REPOID1 STORAGE1
duplicacy -init -repository /mnt/user/Downloads/Manual/Test2 REPOID2 STORAGE1
                                                             ^^^^^^^

And make your script cd into both of those and back them up.

A note regarding this: i believe @gchen’s example has a single mistake (that you followed hence the confusion) because he uses the same repository id for both repositories, which is wrong because those repositories are different (there are 2 different folders):

@gchen: is your example above wrong because of the same repository id? Can i fix the post if that’s the case?

I still believe (and this is how i do all my backups using duplicacy-utils) that using a single repository where i symlink all the folders that i need backed up is good enough.

#21

Awesome,

I know I asked a million questions here and I just want to say thank you very much. I have done a complete write up for the unRAID community and I hope more people get on board with this software. The only other software I have seen like this is Duplicati. The problem is Duplicati has so many issues with the database stuff, the slow restoring/browsing etc. Completely insane. My borg backups take about 4 hours each week to check/upload. I ran Duplicacy and it took 1:10 seconds after the initial run.

With that being said, I believe your symlink solution is absolutely the best solution now that I have analyzed all possible options.

Will Duplicacy CLI remain free forever? I am willing to completely make the switch at this point but am not interested in a monthly fee as I already pay that to my cloud storage provider.

1 Like
#22

Please see Duplicacy Pricing Information .

#24

Yes, the CLI version will remain free for personal use and it wouldn’t make sense to change this.

5 Likes
Big change in CLI license
#25

For anyone interested, I’m using a slightly different approach to this that makes working with multiple directories/repositories super easy. I haven’t found it specifically mentioned or documented, so figured I’d post it here alongside the other strategies posted above.

Basically, after you init a main/central repository, you can then use the add command to add other directories you want to backup. The trick is using the -repository option with the same storage as the main repository. For example:

cd ~
duplicacy init home gcd://Backups
duplicacy add -repository /mnt/work work work gcd://Backups

Then to backup each directory/repository:

duplicacy backup # backup home repository
duplicacy backup -storage work # backup work repository

It’s also possible to list or prune all repositories at once:

duplicacy list -all # list all snapshots from all repositories
duplicacy prune -all -keep X:X # prune all repositories at once

The nice thing about this method is you only have one central .duplicacy folder in the main repository, with a single preferences file listing all of your repositories/storages.

I’m not sure if there are any unapparent downsides, but it’s been working great for me for over a month of daily backups, pruning and several restores.

2 Likes
#26

After using this method for a while, I wanted to post an update for completeness. Even though you have a single .duplicacy folder, you still basically need to manage each repo separately, or remember to use -all in other cases. If you don’t need differing backup times or retention policies, it’s not really worth the extra steps.

I’m currently using the symlink method with a dedicated repository root folder, and I couldn’t be happier. You have everything related to your backups in its own central location, with an immediate overview of the top-level folders being backed up, and every duplicacy operation is just one short command away.

I’d definitely recommend the symlink method if you’re someone who likes to keep things simple, and only look into the other methods if you need more granular control.