How to run duplicacy as a cron job on linux?

preferences
scripting

#1

There are certainly many ways of doing this but here is how I managed to schedule duplicacy on a Linux server (Ubuntu 18.04).

(If you know a better/different way, please add it! Or if I made a mistake (I’m not a Linux expert!), please correct it.)

To start with: I’m running duplicacy as root so that I can backup files from different users in the same backup. If you are just backing up files from a single user, you can probably save yourself a lot of trouble by not using sudo.

1. Initialize your repository (mine is called NAS)

cd /srv/NAS/
sudo duplicacy -d init -e NAS webdav://<myusername>@webdav.pcloud.com/Backup/Duplicacy

Make sure you adjust the working directory and the init command to suit your needs.

You will probably get some errors like

Reading the environment variable DUPLICACY_WEBDAV_PASSWORD
Failed to store the value to the keyring: keyring/dbus: Error connecting to dbus session, not registering 
SecretService provider: dbus: DBUS_SESSION_BUS_ADDRESS not set

Just ignore them and enter the requested passwords (which will trigger another error, but we’ll tackle that in the next step)

2. Save your passwords into your preferences file

sudo duplicacy set -key webdav_password -value "this is my webdav passphrase"


sudo duplicacy set -key password -value "this is my storage passphrase"

This is probably not the most secure method, but I couldn’t figure out how to get environment variables to work with sudo… I guess security is okay, since only root has access to the preferences file)

3. Setup your filters file

See this topic on how to do this.

4.

There shall be no step 4 in this tutorial :laughing:

5. Run your first backup

Before running the backup automatically, I like to text it manually to see that things are working fine:

Doing something like

sudo duplicacy -log  backup -stats

you should not be asked for any password.

You can either let your initial backup run through this way (in that case, perhaps use sudo duplicacy -log backup -stats & to let it run in the background) or you can stop it and let it be triggered according to schedule (see below).

6. Create a backup script

I want to keep logs of all automated backups and since I did not manage to achieve this on a single line in crontab, I use a script (which also adds some flexibility):

#!/bin/sh
cd /srv/NAS/ 
echo "`date`" Backing up $PWD ...
/usr/local/bin/duplicacy -log backup -stats
echo "`date`" Stopped backing up $PWD ...

Save the script wherever it suits you and don’t forget to make it executable. Mine is /home/christoph/backup_NAS.sh.

7. Schedule the backup

sudo crontab -e

and add something like

30 3 * * * /home/christoph/backup_NAS.sh > /srv/NAS/christoph/duplicacy/logs/NAS_backup_`date "+\%Y-\%m-\%d-\%H-\%M"`.log 2>&1

Adjust paths as appropriate.


So how does the gnome keyring work with sudo?
#2

In order to prevent a backup from running before the previous one has finished, I use a script that looks something like this:

$ cat /tank/.duplicacy/scripts/backup_tank.sh:

#!/bin/bash

# https://stackoverflow.com/a/185473/1388019
lockfile="/tmp/duplicacy_tank.lock"

if [ -e ${lockfile} ] && kill -0 `cat ${lockfile}`; then
    echo "duplicacy backup already running"
    exit
fi

# make sure the lockfile is removed when we exit and then claim it
trap "rm -f ${lockfile}; exit" INT TERM EXIT
echo $$ > ${lockfile}

# run the backup
cd /tank
/usr/local/bin/duplicacy -log backup -threads 4

# clean up lockfile
rm -f ${lockfile}

#3

I suppose there are certain circumstances where the lock file will not be deleted (when the script doesn’t complete) and the consequence would be that no more backups will be made. Maybe I’m missing something but for my “set and forget” attitude, this is way too risky.

I’m not sure how exactly to do this, but how about checking running processes and pipe the list through grep to see if it’s running, if yes, wait etc.

If your setup is more complex, you could also use the comment option and grep for your comment.


#4

Well, it’s not in bash syntax, but I’ve already used tasklist, as in the script below, with two consecutive calls to Rclone:

tasklist /FI "rclone.exe" 2>NUL | find /I /N "rclone.exe">NUL
if "%ERRORLEVEL%"=="0" exit /B       (Rclone is running)
rclone  ..... (second exec)

Would have to evaluate what is the equivalent of tasklist in linux and “translate” to bash … :roll_eyes:


#5

I think the closest might be ps aux (possibly with different options).