TrueNAS Core Plugin

saspus · 20 September 2022 08:46

Hi All,

I’ve put together a simple iocage plugin to simplify deployment of Duplicacy Web on TrueNAS Core.

Repository: GitHub - arrogantrabbit/iocage-plugin-duplicacy: This is a simple duplicacy_web plugin for TrueNAS Core

To install (right from the TrueNAS shell in the UI):

curl -L https://raw.githubusercontent.com/arrogantrabbit/iocage-plugin-duplicacy/master/duplicacy.json -o /tmp/duplicacy.json
sudo iocage fetch -P /tmp/duplicacy.json

Sample output:

truenas% curl -L https://raw.githubusercontent.com/arrogantrabbit/iocage-plugin-duplicacy/master/duplicacy.json -o /tmp/duplicacy.json
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   595  100   595    0     0   9459      0 --:--:-- --:--:-- --:--:--  9596

truenas% sudo iocage fetch -P /tmp/duplicacy.json
Plugin: duplicacy
  Official Plugin: False
  Using RELEASE: 13.1-RELEASE
  Using Branch: 13.1-RELEASE
  Post-install Artifact: https://github.com/arrogantrabbit/iocage-plugin-duplicacy.git
  These pkgs will be installed:
    - curl
    - wget
    - jq

Testing Host DNS response to pkg.FreeBSD.org
Testing duplicacy's SRV response to pkg.FreeBSD.org
Testing duplicacy's DNSSEC response to pkg.FreeBSD.org

Installing plugin packages:
  - curl...
  - wget...
  - jq...

Fetching artifact...
Cloning git repository

Branch 13.1-RELEASE does not exist at https://github.com/arrogantrabbit/iocage-plugin-duplicacy.git!
Using "master" branch for plugin, this may not work with your RELEASE

Running post_install.sh
Configuration file does not exist.
Creating default one enabling listening on all interfaces
Configuring netwait to wait for 1.1.1.1
netwait_ip:  -> 1.1.1.1
Enabling netwait and newsyslog services
newsyslog enabled in /etc/rc.conf
netwait enabled in /etc/rc.conf
Starting duplicacy updater
duplicacyupd enabled in /etc/rc.conf
Starting duplicacyupd.
Waiting for duplicacy binary to download
... Still waiting for duplicacy to download
... Still waiting for duplicacy to download
... Still waiting for duplicacy to download
Starting duplicacy service
duplicacy enabled in /etc/rc.conf
Starting duplicacy.
Ready

Admin Portal:
http://10.0.17.10:3875

And looks like so:

Clicking the Manage button takes you to duplicacy web.

There are a few things that can be added if there is interest:

An application icon (I don’t think I can just “borrow” an official one)
Settings configuration (such as selecting update channel; it’s now hardcoded to “stable”)
Configurable update check interval (now hardcoded to every 15 minutes)
Older FreeBSD versions support (No idea if anybody wants that)
Configurable non-root user ID to run duplicacy (Maybe useful for some)
Submit for inclusion to official ixsystems community plugin hub. This is something that @gchen would need to do if this takes off and/or if he wants to.

gchen · 20 September 2022 13:11

Great! This can be very useful to some users. I’ll setup a TrueNAS to try it.

I really appreciate all your works!

DavidxH · 18 October 2022 17:06

Many thanks for making this. I strongly encourage having this included in the TrueNAS list of plugins.

I’ve installed it on TrueNAS CORE 13.0-U2. No installation issues; however, I do not see a “manage” button - all I see is the standard jail “> SHELL” button. Accessing the GUI directly from a browser (http://[ip]:3875) appears to work fine.

I do think being able to configure backups to run under a non-root user would be useful.

One question: Do backup jobs continue after the browser session is closed? I assume they do, as I’m still seeing traffic on the firewall. My (initial) backups are running so slowly (<2MB/s[16Mb/s]) that I can’t be sure.

Backup speed will be the topic of another thread …

Thanks

saspus · 18 October 2022 17:19

Look in Plugins section.

yep!

DavidxH · 18 October 2022 18:38

Urg. Guess looking in the right place would help …

Many thanks again for the help, and for making this.

saspus · 18 October 2022 20:38

I’ll try to add that. It’s not obvious how to do conveniently: pre-defined user with a fixed user ID like other plugins do (e.g Plex) is counterintuitive and error prone. I’m not sure if there is a better way.

No problem, glad it’s helpful

DavidxH · 19 October 2022 02:21

While you’re at it, the ability to specify the host name would also be nice to have.

saspus · 26 October 2022 11:08

Done. to switch duplicacy to using another user – create that user on your main instance, and then use (assuming user name is duplicacy):

sudo iocage set -P duplicacy_user=$(id -u duplicacy),$(id -g duplicacy) duplicacy

or

sudo iocage set -P duplicacy_user=0 duplicacy

or

sudo iocage set -P duplicacy_user=193,339 duplicacy

The First number is user id, second number – group id. If the second number is missing – user ID and group ID will be the same. If user ID is 0 – root will be used. The last “duplicacy” – is a plugin/jail name

To get current setting:

sudo iocage get -P duplciacy_user duplicacy

This is the standard mechanism TrueNAS configures plugins, it appears they did not get around to implementing UI for it, like it was in a very old version of FreeNAS.

Hostname next

leo · 24 August 2023 11:35

@saspus thanks for making this!

Sorry to revive an old thread, but I’m running into an issue where the keyring isn’t working so Duplicacy asks for a password after every restart. I tried installing gnome-keyring-daemon in the jail and setting DWE_PASSWORD as an env variable but neither have worked. Any ideas?

@gchen - do you know why setting DWE_PASSWORD as an environment variable isn’t working? I found here that it should. Thanks!

saspus · 24 August 2023 16:36

Interesting. That flag was made obsolete quite some time ago, because the ui saves the state (to the duplicacy.json file?) after the first login. I’ve removed that flag from my docker container in 2020: Bitbucket

I’m wondering why does it fail for you. Are you using latest Duplicacy web version? (The plugin allows to specify the version IIRC)

saspus · 24 August 2023 16:50

Actually, what do you mean by “asks for password”? The WebUi asks you to login to see the schedules, backups, etc?

Or are backups not running until you login in the UI?

If the former — it’s by design. You don’t want anyone who steals your nas and boots it to get full access to your data.

The latter — DWE_PASSWORD was used to decrypt the credentials to do backups. But in the more recent versions of duplicacy web it’s no longer required, schedules will still run even if you don’t login to web Ui

leo · 24 August 2023 17:32

Or are backups not running until you login in the UI?

I think it’s the latter. The former would be if the Administrator password was set, right?

I’m moving from Unraid to TrueNAS Core but didn’t experience this behavior when Duplicacy was running as a Docker container. I think I have the latest with 1.7.2 on both.

saspus · 24 August 2023 17:43

I’ll to reproduce on my truenas once I get home tonight.

saspus · 26 August 2023 07:02

Indeed, does not work.

At start, duplicacy says:

2023/08/25 23:54:14 Schedule test next run time: 2023-0826 00:00
2023/08/25 23:55:29 Failed to marshal the configuration: json: error calling MarshalJSON for type main.StorageCredentials: No master password provided
2023/08/25 23:55:29 Failed to get the value from the keyring: keyring: No suitable keyring provider found (check your build flags)
2023/08/25 23:55:29 Failed to decrypt the testing data using the password from KeyChain/Keyring: crypto/aes: invalid key size 0
2023/08/25 23:55:29 Temporary directory set to /root/.duplicacy-web/repositories
2023/08/25 23:55:29 Schedule test next run time: 2023-0826 00:00
2023/08/25 23:55:29 Duplicacy Web Edition 1.7.2 (1B2557)
2023/08/25 23:55:29 Duplicacy CLI 3.1.0

and then when it’s time to run the schedule it says:

2023/08/26 00:00:01 Skipping schedule test at time 2023-0826 00:00 as no encryption password has been provided
2023/08/26 00:00:01 Please open the web browser at 0.0.0.0:3875 to enter the encryption password
2023/08/26 00:00:02 Schedule test next run time: 2023-0826 00:15

@gchen, can you please explain what does duplicacy_web does not like here? It seems it finds some keyring, but not quite, and then fails to decrypt? There is no keyring in the jail, just duplicacy_web. Maybe it needs some additional flags/packages to be able to manage the credentials to do backups without needing users to log in? this is on freebsd

I found this:

Has this been implemented on freebsd? or is this irrelevant, because the storage credentials are supposed to be stored some other way?

saspus · 26 August 2023 07:47

As a workaround, you can bring back DWE_PASSWORD.

Get to the duplicacy jail console:

sudo iocage console duplicacy

Edit the /etc/rc.d/duplicacy, and add:

export DWE_PASSWORD="duplicacy-web-admin-pa$$w0rd"

near the export HOME=/root, e.g. around line 14.

Then restart the service, and exit from the jail console:

service duplicacy restart
exit

It shall be working now:

2023/08/26 00:30:22 Temporary directory set to /root/.duplicacy-web/repositories
2023/08/26 00:30:22 Schedule test next run time: 2023-0826 00:45
2023/08/26 00:30:22 Duplicacy Web Edition 1.7.2 (1B2557)
2023/08/26 00:30:22 Duplicacy CLI 3.1.0
2023/08/26 00:45:01 Starting schedule test at scheduled time 2023-0826 00:45
2023/08/26 00:45:01 Created log file /root/.duplicacy-web/logs/backup-20230826-004501.log
2023/08/26 00:45:01 Running /root/.duplicacy-web/bin/duplicacy_freebsd_x64_3.1.0 [-log backup -storage test -threads 1 -stats]
2023/08/26 00:45:01 Set current working directory to /root/.duplicacy-web/repositories/localhost/0
2023/08/26 00:45:04 Schedule test next run time: 2023-0826 01:00

Note, there is no editor in the jail, you can install one with:

pkg install -y vim

or

pkg install -y nano

or whatever else you prefer.

leo · 26 August 2023 21:13

This worked!! Thank you @saspus…I was setting the env variable in the wrong spot before.

gchen · 28 August 2023 14:47

After the log message, the web GUI should continue to check if there is a keyring file under the ~/.duplicacy-web directory. Maybe this file is not made persistent by the plugin setup?

saspus · 28 August 2023 14:51

No, everything is persistent. Plugin is just an automated way to create a jail. Once it’s created, it exists, including everything inside of it.

That file simply never got created. I assume it should have been generated when the web gui password was set the first time? It wasn’t… Here is the list of files there:

root@truenas[~]# ls -l /mnt/pool1/iocage/jails/duplicacy/root/root/.duplicacy-we
b
total 24
drwx------  2 root  wheel     3 Aug 25 23:50 bin
-rw-------  1 root  wheel  2647 Aug 26 00:45 duplicacy.json
-rw-------  1 root  wheel  1033 Aug 25 23:50 licenses.json
drwxr--r--  2 root  wheel     6 Aug 26 00:45 logs
drwx------  3 root  wheel     3 Aug 25 23:50 repositories
-rw-r--r--  1 root  wheel    49 Aug 25 23:50 settings.json
drwx------  4 root  wheel     4 Aug 25 23:50 stats
root@truenas[~]#