I believe you could just maintain a host copy of licenses.json as I do in my image. Thoughts?
No, that does not work (I actually keep entire ~/.duplicacy-web minus caches minus logs host side).
Because duplicacy detects that the host changed those licenses issues for previous host now become invalid and it downloads a new one.
I do preserve /etc/machine-id but that is not enough. Perhaps duplicacy uses Ethernet MAC address as well, I can try preserving that as well, but I don’t want to keep chasing it; it would be easier if we had supported solution.
On linux Duplicacy finds the license id in /var/lib/dbus/machine-id. You can symlink it to /etc/machine-id.
This does not seem to work.
If I don’t have /etc/machine-id the software complains that it cannot find that file. So, I’ve symlinks both, /etc/machine-id and /var/lib/dbus/machine-id to the same file, located outside, that is populated once from /var/lib/dbus/machine-id.
And yet, every time duplicacy_web downloads new license:
Here is what I do in a new container:
ln -s /config/ ~/.duplicacy-web
...
echo Machine-ID=$(cat /var/lib/dbus/machine-id)
echo Licenses=$(cat /config/licenses.json)
...
duplicacy_web &
wait
and this is what I get:
First invocation, when licensees.json does not exist:
Machine-ID=94eb0440e21288f27c81ea6b5c4bbc1f
cat: can't open '/config/licenses.json': No such file or directory
Licenses=
Logging tail of the log from this moment on
Starting duplicacy
Log directory set to /logs
2019/01/26 02:09:45 A new license has been downloaded for 9cc80824b422
Duplicacy CLI 2.1.2
Second invocation:
Machine-ID=94eb0440e21288f27c81ea6b5c4bbc1f
Licenses={ "9cc80824b422": { "license": "7b....0a", "signature": "1b46287434cb0b0a97eec729d410d1e354fa384294cf1d8498dff4c26fe58f894eb004593bf3ce10787261fa46990e9240a56a90af6d312ee96659fc345f23ec1e58cee2ab67f0b45677226264d5674fac281f619ca8457320406b64437851ee15c128d9d9b38dede074cc9442d17a9f000945bc7bdbdaa743fb819aa2ec950c39e6d20de7146f541ef32bc2272a423f787f738c99265f1267820ce0fbb5d72bc87432ebf411ba025e108adec80b34709d2b9f9da65ae2c37d90f745937a4b489d1a4f3992a7bf0e723a8c92b3a9e6e651edd11572723df4204a6c711b3d3d5be79388a34337898f0fb31713ae13c40893fc9a11c4411b63aecd649805e08343" } }
Logging tail of the log from this moment on
Starting duplicacy
Log directory set to /logs
Duplicacy CLI 2.1.2
Duplicacy Web Edition Beta 0.2.10 (5771BE)
Starting the web server at http://[::]:3875
2019/01/26 02:09:59 A new license has been downloaded for 130ff08991ba
Note, same machine-id and yet it downloads new license.
So, what else is being used to come up with the host id?
9cc80824b422 and 130ff08991ba are host names. When the host name changes Duplicacy will download a new license (both the machine id and the host name should match what is in the license).
The host name is read from /proc/sys/kernel/hostname, but I guess it should be the same as what the hostname command returns.
1 Like
Ah! That totally makes sense. Docker creates hostnames that look like ids, that confused me.
So, this is what I’ve learned needs to be done to stabilize the license when changing the container (which is based on alpine linux):
- install dbus:
apk add dbus. - link
ln -s /var/lib/dbus/machine-id /etc/machine-id. - pass
--hostnameargument when launching container.
This result in correct behavior, docker seems to ensure that machine-id is generated consistently on the same host, no need to save/restore it.
Thank you for guidance!
2 Likes
What does “Match the requested format” mean when choosing a Backup ID?
When backing up to Wasabi, it worked perfectly on the first try, I chose “Companyname” as the backup ID. When backing up to SFTP, it seems the Backup ID is more sensitive. I keep getting:
Running backup command from /var/services/homes/admin/.duplicacy-web/repositories/localhost/1 to back up /volume1/Storage
Options: [-log backup -storage Synology_DS413j -stats]
2019-02-07 06:57:09.736 INFO REPOSITORY_SET Repository set to /volume1/Storage
2019-02-07 06:57:09.736 INFO STORAGE_SET Storage set to sftp://Backup@192.168.2.17/Backup
2019-02-07 06:57:11.341 ERROR SNAPSHOT_LIST Failed to list the revisions of the snapshot CompanyName: failed to send packet header: EOF
But when I choose a short one-character name for the Backup ID, like “a” or “c”, it works immediately. I’ve tried 20 different combinations, and that’s the only factor that makes it work every time. Here’s one I cancelled after a few seconds once I saw it was working:
Running backup command from /var/services/homes/admin/.duplicacy-web/repositories/localhost/14 to back up /volume1/Storage
Options: [-log backup -storage ds413j -stats]
2019-02-07 08:32:21.573 INFO REPOSITORY_SET Repository set to /volume1/Storage
2019-02-07 08:32:21.573 INFO STORAGE_SET Storage set to sftp://Backup@192.168.2.17/Backup
2019-02-07 08:32:23.170 INFO BACKUP_START No previous backup found
2019-02-07 08:32:23.170 INFO BACKUP_INDEXING Indexing /volume1/Storage
The Backup ID seems to matter a lot for SFTP, but not for Wasabi. Can we have some guidance on this?
Thanks!
What is your SFTP server? Another Synology? If so, add another slash to the url after server name before the path. See Chunk uploading error with Synology
3 Likes
Indeed it is another Synology.
I added the leading slash, and now my storage looks like this:
sftp://Backup@192.168.2.17//Backup
That did it! Thanks!!
Ouch - this won’t work for users (like me) who want to run it on a QNAP or Synology device with a vanilla Docker installation without dbus.
Any other way around this? Specifically I run docker in Swarm mode, so I’m unable to set a hostname (it’s not yet supported). I was about to graduate from CLI to a licensed user of the Web GUI, but need to make sure the license will be consistent.
It appears to be impossible to add more than one Storage entity at this time. Can someone confirm whether or not they see the addition sign (+) after adding the first storage entry?
Ignorant question: Can’t you just add in .dockerfile
run echo "hostname" > /proc/sys/kernel/hostname ?
That was my misunderstanding of what was going on. I have already changed the way machine id is handled. It is now stored in the config directory along with the rest of duplicacy configuration so once generated it will never change for that virtual installation.
I will add override of a hostbame via environment variable or a file later today, good idea.
FWIW I run and test it on synology myself.
Bottom left corner, there should be a green +.
Been running the new web edition for just over a month and one thing has become clear - my graphs are looking a bit wonky:
Does everyone else’s graphs look like this?
Clearly, the New Revisions graph needs to be turned into a bar chart, coz those Bézier curves have got some serious lick!
However, my particular set of graphs seem to be broken, perhaps due to those localhost-0 et al, activities. Those were an initial run of each backup job, before they were assigned to proper schedule.
Plus the scheduled activities only seem to show the last week’s worth. If this is by design, I don’t mind that, but the graph extends more than a week. (Also, if it IS a design choice, a longer time span would be nice to have, unless there are performance reasons?)
1 Like
Scratch that. I forgot why I did not do it in the first place – hostname can be provided as --hostname argument when starting container, something like this:
docker run --name duplicacy-web-container \
--hostname duplicacy-web-docker \
--publish 3875:3875/tcp \
--env USR_ID=$(id -u) \
--env GRP_ID=$(id -g) \
--volume /tmp/config:/config \
--volume /tmp/logs:/logs \
--volume /tmp/cache:/cache \
--volume ~:/backuproot:ro \
--volume /tmp/storage:/storage \
saspus/duplicacy-web
Synology UI uses “Container Name” as a hostname. So nothing to be done there. It’s already working correctly. Let me know if your use case is different.
That likely won’t work for me because I run the image via Docker Swarm, which means technically a worker can run on any (single) node and have a different hostname depending on how hostname is accessed via Duplicacy. That single instance is monitored by the swarm and restarted if necessary or relocated if a node goes down. Then duplicacy backs up shared storage.
Tried it and it keeps getting “localhost” as the server name. Is there any way to accept hostname as an ENVIRONMENT variable like:
environment:
- hostname:duplicacy-web-ui
Here is evidence of the problem:
Yeah, I’ve read a bit docker swarm documentation and indeed this seems to be inherent problem.
I’ll add environment variable to override the hostname; at some point we shall consult with @gchen since all that pretending for the instance to be different host feels like stretching boundaries of the license.
At least there should be a way to ensure that only one instance is running at a time for each combination of of id-hostname-license.
I just installed the latest beta and I’m wondering if there is a way to import the configuration from the GUI client ?
Thanks.
-DM
1 Like