Run web-ui in a docker container?


#1

First of all - nice work! Soon you’ll be able to differentiate Duplicacy from other backup tools on the basis of performance AND functionality/ease of use.

Note: I am running duplicacy on a Linux x64 server.

Is there any way we can get this into a docker image? I’m fairly religious about running my web apps in docker images (reverse-proxied by Traefik). Benefits?

  1. Minimize the attack vector (if you don’t run docker as root).

  2. Allows you to mount just the storage you want backed up to the image.

  3. Eliminates the need to start the application before modifying the port it listens on (as has been mentioned, port 8080 will commonly be in use on an existing (shared) server. In a docker container, you can choose to:

    1. Leave it at 8080 (since it’s only on the container) and reverse proxy to that port, without needing to bind it to the host at all (very common with several web apps I run).

    2. Bind port 8080 internally to whatever port you want on the host without touching the code.

Here’s the Dockerfile I’ve created. The only error I’m getting is that it cannot find the CLI executable. I tried ensuring the CLI version of Duplicacy was available inside the image (at /bin, in the path) but no matter how I executed to GUI runtime, it gave the CLI missing error inside the container upon startup. That’s the one piece I need to figure out.

FROM alpine:latest
ADD duplicacy_web_linux_x64_0.1.0 /bin/duplicacy_web
ADD duplicacy_linux_x64_2.1.2 /bin/duplicacy
RUN chmod +x /bin/duplicacy_web /bin/duplicacy
EXPOSE 8080

I’ve tried each of the following as the last line in the Dockerfile:
ENTRYPOINT ["/bin/duplicacy_web"]
CMD ["/bin/duplicacy_web"] and other variations. I just need to get the CLI found inside the image. I’m not exactly a developer though. Ideas?

Oh - here’s how a very simple docker run would work:

docker run -d -p <any_port>:8080 -v /volume/to/backup:/backup/name_of_volume -v /second/volume/to/backup:/backup/name_of_second_volume <repo>/duplicacy_web

And a simple docker-compose.yml:

image: <repo>/duplicacy_web
container_name: duplicacy_web
volumes:
  - /volume/to/backup:/backup/name_of_volume
  - /second/volume/to/backup:/backup/name_of_second_volume
ports:
  - <any_port>:8080

Then all your backup repositories are nicely mounted inside the container. (By the way, I presume the web GUI doesn’t change the underlying architecture in which .duplicacy folder is present in each repo. If the repo information is kept elsewhere, then you could mount the backup repositories as “read only”).


Screenshots of the new web-based GUI
#2

The web GUI always looks for the CLI under ~/.duplicacy-web/bin. If it can’t find it there, it will download the latest release from github.com. So I think if you can copy the CLI under that directory it should work.


#3

I am sure I have the files in the right place, but I still get this error:

/root/.duplicacy-web/bin/duplicacy_web
Created a new configuration.
Failed to locate the CLI executable

This is despite the fact that upon container startup, I can do the following inside the container:

~ # cd /root/.duplicacy-web/bin
~/.duplicacy-web/bin # ls -lart
total 47624
-rwxrwxr--    1 root     root      26327201 Nov 15 13:54 duplicacy
-rwxrwxr--    1 root     root      22428329 Nov 15 13:54 duplicacy_web
drwxrwxrwx    1 root     root          4096 Nov 15 13:54 .
drwxr-xr-x    1 root     root          4096 Nov 15 13:56 ..
~/.duplicacy-web/bin #

Is it looking for a specific filename or format? If it looks in ~/.duplicacy-web/bin, does it not like my filename?Here is my dockerfile now (it’s gotten really ugly as I hack it together with Duct Tape :slight_smile:

FROM alpine:latest
RUN mkdir -p /app
RUN mkdir -p /root/.duplicacy-web/bin
RUN chmod 777 /root/.duplicacy-web/bin
COPY duplicacy /app/duplicacy
COPY duplicacy_web /app/duplicacy_web
RUN cp /app/duplicacy /root/.duplicacy-web/bin/duplicacy
RUN cp /app/duplicacy_web /root/.duplicacy-web/bin/duplicacy_web
RUN chmod 774 /root/.duplicacy-web/bin/duplicacy
RUN chmod 774 /root/.duplicacy-web/bin/duplicacy_web
EXPOSE 8080
#ENTRYPOINT ["/root/.duplicacy-web/bin/duplicacy_web"]

The last line is commented out so the completed image may be run interactively to get a prompt. Trying to run the coomand in ENTRYPOINT manually starts to create a config, but is still giving an error that it cannot find the CLI. Any ideas?


#4

The CLI has to be named duplicacy_linux_x64_2.1.2. The version number may change later when there are newer releases on github, but you can ‘freeze’ the version by adding the line to duplicacy.json:

    "cli_version": "2.1.2"

#5

Renaming the file allowed me to clean things up. This is now working as a draft we can refine if it’s of any value to you. One incredibly important thing that’s missing is: where is duplicacy_web storing its data? I want to put it in a docker named volume or (more likely) - bind that volume to a host volume. Right now, no settings are persistent beyond restarts of the container.

FROM alpine:latest
COPY duplicacy_linux_x64_2.1.2 /root/.duplicacy-web/bin/duplicacy_linux_x64_2.1.2
COPY duplicacy_web /root/.duplicacy-web/bin/duplicacy_web
RUN chmod 774 /root/.duplicacy-web/bin/duplicacy_linux_x64_2.1.2 /root/.duplicacy-web/bin/duplicacy_web
EXPOSE 8080
ENTRYPOINT ["/root/.duplicacy-web/bin/duplicacy_web"]

And my duplicacy.yml (docker-compose) stack:

version: '3'
services:

  duplicacy:
    image: gkoerk/duplicacy:latest
    volumes:
      # Expose backup folders to the container:
      - /share/appdata:/backup/appdata
      - /share/Photos:/backup/photos
      - /share/downloads:/backup/downloads
      - /share/media/movies:/backup/movies
      # Mount external storage as one backup destination.
      - /share/USBDisk1:/destination/external
    networks:
      - traefik_public
    deploy:
      labels:
        - traefik.port=8080
        - traefik.network=traefik_public
        - 'traefik.frontend.rule=Host:backup.domain.com'          

networks:
  traefik_public:
    external: true

#6

Where does duplicacy_web store persisted data?


#7

All files are under ~/.duplicacy-web. The most important one is duplicacy.json which stores almost all config information.

The folder filters has all include/exclude patterns so this folder should be backed up too.

The folders stats and logs are less important; if you don’t want to keep the history you don’t need to back up these two.

And definitely no need to back up the repositories folder, as all files there are temporary (most are cache files).


#8

Okay - here is a better docker-compose.yml. Note that it offers persistence, but you will need to copy out the duplicacy.json file. Also - I can’t get Traefik to reverse-proxy the web app. Any idea what might be the issue? Right now I just get a “waiting for backup..com” and eventually a gateway timeout.

Dockerfile:

FROM alpine:latest
COPY duplicacy_linux_x64_2.1.2 /root/.duplicacy-web/bin/duplicacy_linux_x64_2.1.2
COPY duplicacy_web /root/.duplicacy-web/bin/duplicacy_web
RUN chmod 774 /root/.duplicacy-web/bin/duplicacy_linux_x64_2.1.2 /root/.duplicacy-web/bin/duplicacy_web
EXPOSE 8080
ENTRYPOINT ["/root/.duplicacy-web/bin/duplicacy_web"]

To build your image you run the following from within the same folder as the Dockerfile, and reference it as the image name in your docker-compose.yml (yes, there are several other ways to do this, please feel free to improve upon this however you see fit):

docker build -t <imagename>:<tag> .

docker-compose.yml (works via docker-compose up -d or in Swarm mode via docker stack deploy:

version: '3'
services:

  duplicacy:
    image: <imagename>:<tag>
    ports:
      - 8090:8080
    volumes:
      - /share/appdata/duplicacy/duplicacy.json:/root/.duplicacy-web/duplicacy.json
      - /share/appdata/duplicacy/stats:/root/.duplicacy-web/stats
      - /share/appdata/duplicacy/log:/root/.duplicacy-web/logs
      - /share/appdata:/backup/appdata
      - /share/Photos:/backup/photos
      - /share/downloads:/backup/downloads
      - /share/media/movies:/backup/movies
      - /share/media/music:/backup/music
      - /share/media/tv:/backup/tv
      - /share/USBDisk1:/destination/external
    networks:
      - traefik_public
    deploy:
      labels:
        - traefik.port=8080
        - traefik.network=traefik_public
        - 'traefik.frontend.rule=Host:backup.gkoerk.com'          

networks:
  traefik_public:
    external: true

And the command to setup the traefik_public network:

  • docker-compose:

docker network create --driver=bridge --subnet=172.1.1.0/21 traefik_public

  • docker stack deploy:

docker network create --driver=overlay --subnet=172.1.1.0/21 --attachable traefik_public

`


Duplicacy Web Edition 0.1 Beta
#9

I’ve also been toying around with running the web UI in a Docker container and though I’d share my working Dockerfile:

FROM alpine:latest
    
RUN ARCHITECTURE=linux_x64                                                                             && \
    SHA256_DUPLICACY=034720abb90702cffc4f59ff8c29cda61f14d9065e6ca0e4017ba144372f95d7                  && \
    SHA256_DUPLICACY_WEB=b0a9fc35f249ee8fa5d7da0fe0f8487041661ed19b24c115fd01aee37049ccfe              
    VERSION_DUPLICACY=2.1.2                                                                            && \
    VERSION_DUPLICACY_WEB=0.2.8                                                                        && \
                                                                                                          \
    # add Bash for our entrypoint.sh, and ca-certificates so Duplicacy doesn't complain about certs
    apk update                                                                                         && \
    apk add --no-cache bash ca-certificates                                                            && \
                                                                                                          \
    # download, check, and install duplicacy
    wget --quiet -O /usr/local/bin/duplicacy                                                              \ 
 https://github.com/gilbertchen/duplicacy/releases/download/v${VERSION_DUPLICACY}/duplicacy_${ARCHITECTURE}_${VERSION_DUPLICACY} && \
    echo "${SHA256_DUPLICACY}  /usr/local/bin/duplicacy" | sha256sum -s -c -                           && \
    chmod +x /usr/local/bin/duplicacy                                                                  && \
                                                                                                          \
    # downlooad, check, and install the web UI
    wget --quiet -O /usr/local/bin/duplicacy_web                                                          \
        https://acrosync.com/duplicacy-web/duplicacy_web_${ARCHITECTURE}_${VERSION_DUPLICACY_WEB}      && \
    echo "${SHA256_DUPLICACY_WEB}  /usr/local/bin/duplicacy_web" | sha256sum -s -c -                   && \
    chmod +x /usr/local/bin/duplicacy_web                                                              && \
                                                                                                          \
    # duplicacy_web expects to find the CLI binary in a certain location
    # https://forum.duplicacy.com/t/run-web-ui-in-a-docker-container/1505/2
    mkdir -p ~/.duplicacy-web/bin                                                                      && \
    ln -s /usr/local/bin/duplicacy ~/.duplicacy-web/bin/duplicacy_${ARCHITECTURE}_${VERSION_DUPLICACY} && \
                                                                                                              \
    # create the logs directory and touch a log so that we can tail it before we start the server
    mkdir ~/.duplicacy-web/logs                                                                        && \
    touch ~/.duplicacy-web/logs/duplicacy_web.log                                                      && \
                                                                                                          \
    # listen on all interfaces
    echo '{"listening_address":"0.0.0.0:3875"}' > ~/.duplicacy-web/settings.json

COPY ./entrypoint.sh /usr/local/bin/entrypoint.sh

ENTRYPOINT [ "/usr/local/bin/entrypoint.sh" ]

entrypoint.sh looks something like this:

#!/usr/bin/env bash
tail -f ~/.duplicacy-web/logs/duplicacy_web.log &
duplicacy_web &

It also includes a Bash trap to catch Ctrl-C to shut down the server, but I’ve left that out for clarity.

The trick for me was to get the web UI to listen on 0.0.0.0 by writing an initial duplicacy.json, otherwise the server listens only on localhost and is therefore inaccessible outside of the container.


#10

Could you share the additions/edits or an example of your duplicacy.json (redacted of course)? I don’t see how to add the listening address.