Am I doing this right?

init
wikify
filters
backup

#1

I am using Duplicacy to backup my unRAID box. Previously I was using Borg + RClone but because Duplicacy runs the backup and pushes it to Google I’m looking to switch.

With Borg I would just include the directories I wanted to backup. I understand with Duplicacy, I need to cd into the directory first and run the backup command. My question is, if I have 40 parent directories and I only want to backup let’s say 10, do I need to run the backup command on each directory? For example

cd /mnt/user/Folder1
duplicacy -v backup -threads 2

cd /mntuser/Folder 2
duplicacy -v backup -threads 2

Also do I need to init on each new directory? Or will an init of the root directory /mnt/ work fine for the above example? Can they all go to the same snapshot ID?

Finally…last question, I see the .duplicacy file in each directory I “init” with. I used the -pref-dir command but it still places a .duplicacy file in each directory that I use init with. I don’t like having misc files lingering around my file system. Is there anyway to not have that happen or is it a requirement?


#2

i think you want to use duplicacy in symlink mode: Move duplicacy folder/Use symlink repository.

Basically duplicacy follows first level symlinks in your repository as if they are normal folders.


#3

Several strategies can be used:

  1. Set up a repository for each folder and run a backup job for each of them, as you described;

  2. Configure a unique repository in the “root” folder (/mnt/user/ in your case) and select subfolders to be backed up with filters and include patterns;

  3. Use symbolic links, as suggested by @TheBestPessimist;

To choose the best strategy must be considered aspects such as:

  • number and size of files and folders;
  • need (or not) of different backup configurations (size of chunks fixed or variable, same or diferentt storages, etc.);
  • need (or not) of different schedules;
  • Frequency of source files updating.

I particularly use the first option because it allows me a more granular control over what will be backed up and when it will be backed up. But that depends on each use case.

Take a look at the -repository option at the end of this page:


#4

First off I want to say thanks!!

I managed to get this working with

duplicacy init -e -repository /mnt/user/Downloads/Manual/Test/ -pref-dir /usr/local/duplicacy/TDS TDS gcd://Backup/TDS

Before I read your message, I would CD into the directory and then initiate the backup command. How do I do this now that I am not changing directories?

Edit: Nevermind, I just type duplicacy backup and it runs. I am going to try this on multiple directories


#5

Ok I am really confused now.

I ran the following two commands

duplicacy init -e -repository /mnt/user/Downloads/Manual/Test2/ -pref-dir /usr/local/duplicacy/TDS2 TDS2 gcd://Backup/TDS2

duplicacy init -e -repository /mnt/user/Downloads/Manual/Test/ -pref-dir /usr/local/duplicacy/TDS1 TDS1 gcd://Backup/TDS1

So now I have 2 directories that I want to backup “Test” and “Test2”. I cannot have them in the same -pref-dir? Also can I place them in the same location gcd://Backup/TDS or does that need to be unique too?

Finally, how do I initiate the backup for each of these repo’s without CD-ing into the directory? Thanks for your help, where do I donate for beer or coffee?


#6

Ah yes, this seems to be working. I really wish I could just specify the folders I wanted to backup without going through all this symlink information. Borg does a great job at this with a bash script. Is there a way to bash script this so I can just deploy the script to lets say 20 identical machines without having to symlink stuff?


#7

But using symlink is really simple: save all the ln -sf /source/folder /destination/folder/link in a duplicacy-setup-script.sh and just run it on all the machines.

Afterwards you just init a single repository on each of those machines, in the folder where you symlinked everything.


#8

If you use the -repository option then -pref-dir isn’t needed and you can avoid symlinks by initializing a pseudo repository for each directory to be backed up:

cd /mnt/user/Duplicacy/repository1
duplicacy -init -repository /mnt/user/Downloads/Manual/Test1 repository_id storage_url

cd /mnt/user/Duplicacy/repository2
duplicacy -init -repository /mnt/user/Downloads/Manual/Test2 repository_id storage_url

Here /mnt/user/Duplicacy/repository* are the pseudo repositories and the real directories to be backed are /mnt/user/Downloads/Manual/Test1*. This strategy also has the advantage that all .duplicacy directories will be created under /mnt/user/Duplicacy.


How to initiate a backup for multiple repositories?
#9

Exactly, that is the strategy I use, with all Duplicacy settings centralized in a folder, outside the repositories.


#10

This is perfect!!! Yeah I really like your software, just tricky setting up on unRAID and the .duplicacy files were driving my OCD off the wall. Thank you for taking the time to reply!


split this topic #11

A post was split to a new topic: How to initiate a backup?


#12

For anyone: Feel free to use the :heart: button on the posts that you found useful.

For the OP of any support topic: you can mark the post that solved your issue by ticking the image box under the post. That of course may include your own post :slight_smile:


#13

Hey gchen, that -repository option is giving me “incorrect usage” Is this available on the CLI version for linux_64 2.1.0 ?


#14

Please upgrade to 2.1.2. Version 2.1.0 is way too old!


#16

Awesome, thanks! Did not see that before.


#17

Hello @TheBestPessimist

Per @gchen 's solution for -repository. Can I back multiple directories to the same repository?

IE:

duplicacy -init -repository /mnt/user/Downloads/Manual/Test1 REPOID1 STORAGE1
duplicacy -init -repository /mnt/user/Downloads/Manual/Test2 REPOID1 STORAGE1

If not, what would be the best way to put everything in the same destination folder under the same repo?

Thanks again, I have my script generated and running. I am just getting into the intricate portions.


#18

Just to be clear here:

  • repository means the folder you are backing up right now (the one which has .duplicacy folder)
  • repository id is the name of a repository (should be unique for each of your folders)
  • the storage is the destination of your backup

You can backup multiple repositories to the same storage as long as they have different repository ids and this is indicated as you get the benefit of deduplication.

So for your example:

  • repository: /mnt/user/Downloads/Manual/Test1 and /mnt/user/Downloads/Manual/Test2
  • repository id: REPOID1
  • storage: STORAGE1

The problem with your example is that for 2 different folders to backup (which represent 2 different repositories) you have the same repository id.

What you want is actually to create 2 repositories:

duplicacy -init -repository /mnt/user/Downloads/Manual/Test1 REPOID1 STORAGE1
duplicacy -init -repository /mnt/user/Downloads/Manual/Test2 REPOID2 STORAGE1
                                                             ^^^^^^^

And make your script cd into both of those and back them up.


A note regarding this: i believe @gchen’s example has a single mistake (that you followed hence the confusion) because he uses the same repository id for both repositories, which is wrong because those repositories are different (there are 2 different folders):

@gchen: is your example above wrong because of the same repository id? Can i fix the post if that’s the case?


I still believe (and this is how i do all my backups using duplicacy-utils) that using a single repository where i symlink all the folders that i need backed up is good enough.


#21

Awesome,

I know I asked a million questions here and I just want to say thank you very much. I have done a complete write up for the unRAID community and I hope more people get on board with this software. The only other software I have seen like this is Duplicati. The problem is Duplicati has so many issues with the database stuff, the slow restoring/browsing etc. Completely insane. My borg backups take about 4 hours each week to check/upload. I ran Duplicacy and it took 1:10 seconds after the initial run.

With that being said, I believe your symlink solution is absolutely the best solution now that I have analyzed all possible options.

Will Duplicacy CLI remain free forever? I am willing to completely make the switch at this point but am not interested in a monthly fee as I already pay that to my cloud storage provider.


#22

Please see Duplicacy Pricing Information .


#24

Yes, the CLI version will remain free for personal use and it wouldn’t make sense to change this.