Backups stopped working, segmentation fault in saspus docker image

Support
#1

I noticed I’m no longer getting backup reports, and it looks like my @saspus docker stopped working at some point.

Docker logs from Portainer:

init.sh runing as user root:root(0:0)
Remote available versions... latest: 1.6.3 stable: 1.6.3
Newest cached local version... 1.6.3
Using version 1.6.3
Using cached duplicacy_web binary /config/bin/duplicacy_web_linux_x64_1.6.3
Using machine-id = 30baf6a3e56b852d944d4bfb5d6ae876
duplicacy_web runing as user root:root(0:0)
Logging tail of the log from this moment on
Starting duplicacy_web
/usr/local/bin/init.sh: line 148:    28 Segmentation fault      su-exec $USR_ID:$GRP_ID launch.sh

Ansible config that used to work:

---
# Install Duplicacy
# https://hub.docker.com/r/saspus/duplicacy-web
# https://bitbucket.org/saspus/duplicacy-web-docker-container/src/master/
# https://duplicacy.com/licenses

# https://docs.ansible.com/ansible/latest/modules/file_module.html
- name: "Create Duplicacy Directory"
  file:
    path: "{{ item }}"
    state: directory
  with_items:
    - "{{ appdata_dir }}/duplicacy/config"
    - "{{ appdata_dir }}/duplicacy/logs"
    - "{{ appdata_dir }}/duplicacy/cache"

# https://docs.ansible.com/ansible/latest/collections/community/docker/docker_container_module.html
- name: "Install Duplicacy"
  community.docker.docker_container:
    name: duplicacy
    image: saspus/duplicacy-web:mini
    pull: yes
    hostname: "duplicacy-{{ ansible_hostname }}"
    domainname: "{{ ansible_domain }}"
    restart_policy: unless-stopped
    env:
      TZ: "{{ local_timezone }}"
      # Run as root
      # USR_ID: "{{ user_id }}"
      # GRP_ID: "{{ group_id }}"
      DWE_PASSWORD: "{{ duplicacy_password }}"
    volumes:
      - "{{ appdata_dir }}/duplicacy/config:/config"
      - "{{ appdata_dir }}/duplicacy/logs:/logs"
      - "{{ appdata_dir }}/duplicacy/cache:/cache"
      - "/data:/backuproot:ro"
    networks:
      - name: "{{ docker_local_network }}"
    # published_ports:
    # External routing via traefik
    # - 3875:3875
    labels:
      traefik.enable: "true"
      traefik.http.routers.duplicacy.rule: "Host(`duplicacy.{{ ansible_domain }}`) || Host(`duplicacy.{{ external_services_domain }}`)"
      traefik.http.routers.duplicacy.entrypoints: "websecure"
      com.centurylinklabs.watchtower.enable: "true"
    recreate: "{{ docker_container_recreate }}"

Any ideas?

Duplicacy Web Segmentation Fault
#2

Looks like this: Duplicacy Web Segmentation Fault

If you are using “mini” tag, switch to previous duplicacy version (by setting the DUPLICACY_WEB_VERSION environment variable to desired version, e.g. 1.6.1, and restarting the container) until that is fixed.

@gchen, please change what the ‘https://duplicacy.com/latest_web_version’ reports for stable version. Because this one isn’t stable at all

#3

That worked, thank you.

#4

These 2 segmentation faults are different. This one is in the web GUI while the other in the CLI.

I don’t see any changes between 1.6.1 and 1.6.3 could lead to segmentation faults, which means it may happen again with 1.6.1. Is it possible to configure the docker to dump core on a segmentation fault?

#5

It’s running the alpine linux image. These seem to be instructions to setup coredumps. bash - How to generate a core dump in Linux on a segmentation fault? - Stack Overflow

I would open shell in the container, set that, and run the duplicacy_web to collect the core.

I’ll try to reproduce later tonight, if this happens on x86_64. @pieter , what is the architecture of the machine this is running on?

#6

Actually, this points to the fault likely caused by some of the changed dependencies. When you have built the 1.6.3, did you pin the dependencies or fetched updated ones? If not, can you rebuild the 1.6.3 but with the dependencies versions you’ve used with 1.6.1 (e.g. check out them by date). What about compiler version itself?

#7

Intel x64, Proxmox 7.2, Docker, saspus/duplicacy-web:mini

#8 
2022/11/12 20:54:17 A new license has been downloaded for duplicacy-web-docker-instance-1
2022/11/12 20:54:17 Temporary directory set to /cache
2022/11/12 20:54:17 Duplicacy Web Edition 1.6.3 (31BDE1)
2022/11/12 20:54:17 Duplicacy CLI 3.0.1

On macOS in docker, (which is linux vm essentially), it works fine, I’ve checked just now, the container

OS that gets installed is:

Linux duplicacy-web-docker-instance-1 5.10.124-linuxkit #1 SMP Thu Jun 30 08:19:10 UTC 2022 x86_64 Linux

Maybe proxmox adds variability here? I’ll try docker in proxmox in VMWare fusion on my x86 Mac now :slight_smile: Not sure what I expect, but it it fails, it would be an interesting data point.

#9

Yep, also works in proxmox.

image
image2060×1088 353 KB

So, I guess this would be next steps, with the intent to test it on your system.

To rule out system issues, maybe it is worth running memory test as well, however unlikely it might be.

#10

128GB ECC memory on a server running 23 other docker containers and 2 VM’s with no memory issues.

I tested the image on the same system, with no existing data or configuration, and it runs:

pieter@server-1:~$ docker run -it --rm --pull always --name Testing saspus/duplicacy-web:mini
mini: Pulling from saspus/duplicacy-web
Digest: sha256:3cc88e1d8b0b8ef80c167d180603576b0f17edff9abb443b76bb79973fa3c4bd
Status: Image is up to date for saspus/duplicacy-web:mini
init.sh runing as user root:root(0:0)
Remote available versions... latest: 1.6.3 stable: 1.6.3
Newest cached local version... None
Using version 1.6.3
Downloading executable from https://acrosync.com/duplicacy-web/duplicacy_web_linux_x64_1.6.3
--2022-11-13 09:36:52--  https://acrosync.com/duplicacy-web/duplicacy_web_linux_x64_1.6.3
Resolving acrosync.com (acrosync.com)... 198.199.87.9
Connecting to acrosync.com (acrosync.com)|198.199.87.9|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21735437 (21M) [application/x-troff-man]
Saving to: '/config/bin/duplicacy_web_linux_x64_1.6.3'

/config/bin/duplicacy_web_linux_x64_1.6.3                           100%[=================================================================================================================================================================>]  20.73M  6.60MB/s    in 3.1s

2022-11-13 09:36:55 (6.60 MB/s) - '/config/bin/duplicacy_web_linux_x64_1.6.3' saved [21735437/21735437]

Using machine-id = 3e900c54936c42c89a88f01763712b37
duplicacy_web runing as user root:root(0:0)
Logging tail of the log from this moment on
Starting duplicacy_web
Log directory set to /logs
Duplicacy Web Edition 1.6.3 (31BDE1)
Starting the web server at http://[::]:3875
2022/11/13 09:36:55 A new license has been downloaded for 72eb856c581f
2022/11/13 09:36:55 Temporary directory set to /cache
2022/11/13 09:36:55 Duplicacy Web Edition 1.6.3 (31BDE1)
2022/11/13 09:36:55 Downloading the CLI executable from https://github.com/gilbertchen/duplicacy/releases/download/v3.0.1/duplicacy_linux_x64_3.0.1
2022/11/13 09:36:58 Duplicacy CLI 3.0.1
^C
Terminating pid 30....
Exiting.

If I run with mapped data, then I get a segfault:

pieter@server-1:~$ docker run -it --rm --pull always --name Testing \
    --env TZ="America/Los_Angeles" \
    --env DWE_PASSWORD="foo" \
    --volume "/data/appdata/duplicacy/config:/config" \
    --volume "/data/appdata/duplicacy/logs:/logs" \
    --volume "/data/appdata/duplicacy/cache:/cache" \
    --volume "/data:/backuproot:ro" \
    saspus/duplicacy-web:mini
mini: Pulling from saspus/duplicacy-web
Digest: sha256:3cc88e1d8b0b8ef80c167d180603576b0f17edff9abb443b76bb79973fa3c4bd
Status: Image is up to date for saspus/duplicacy-web:mini
init.sh runing as user root:root(0:0)
Remote available versions... latest: 1.6.3 stable: 1.6.3
Newest cached local version... 1.6.3
Using version 1.6.3
Using cached duplicacy_web binary /config/bin/duplicacy_web_linux_x64_1.6.3
Using machine-id = 30baf6a3e56b852d944d4bfb5d6ae876
duplicacy_web runing as user root:root(0:0)
Logging tail of the log from this moment on
Starting duplicacy_web
/usr/local/bin/init.sh: line 148:    28 Segmentation fault      su-exec $USR_ID:$GRP_ID launch.sh

If I replace the entrypoint and manually run the init, I get the same error:

bash-5.1# /usr/local/bin/init.sh
init.sh runing as user root:root(0:0)
Remote available versions... latest: 1.6.3 stable: 1.6.3
Newest cached local version... 1.6.3
Using version 1.6.3
Using cached duplicacy_web binary /config/bin/duplicacy_web_linux_x64_1.6.3
Using machine-id = 30baf6a3e56b852d944d4bfb5d6ae876
duplicacy_web runing as user root:root(0:0)
Logging tail of the log from this moment on
Starting duplicacy_web
/usr/local/bin/init.sh: line 148:    76 Segmentation fault      su-exec $USR_ID:$GRP_ID launch.sh
bash-5.1#

If I skip init.sh and run launch.sh directly, all is fine:

bash-5.1# su-exec /user/local/bin launch.sh &
[1] 83
bash-5.1# duplicacy_web runing as user root:root(0:0)
Logging tail of the log from this moment on
Starting duplicacy_web

So I suspect the problem is with the code crashing when it access the existing configuration and data?

Adding, I looked at the cached bins, and I suspect the downloaded 1.6.3 is not correct, much smaller?

pieter@server-1:~$ ls /data/appdata/duplicacy/config/bin -ls
total 260981
11105 -rwxrwxr-x 1 root root 26382872 Jan  7  2020 duplicacy_linux_x64_2.2.3
14993 -rwxrwxr-x 1 root root 26806950 Jan  7  2020 duplicacy_linux_x64_2.3.0
14985 -rwxrwxr-x 1 root root 26807132 Mar  7  2020 duplicacy_linux_x64_2.4.0
15009 -rwxrwxr-x 1 root root 26806861 Mar 13  2020 duplicacy_linux_x64_2.4.1
16669 -rwxrwxr-x 1 root root 31654969 Apr 12  2020 duplicacy_linux_x64_2.5.0
16673 -rwxrwxr-x 1 root root 31650873 Apr 19  2020 duplicacy_linux_x64_2.5.1
16669 -rwxrwxr-x 1 root root 31650873 May 12  2020 duplicacy_linux_x64_2.5.2
16761 -rwxrwxr-x 1 root root 31740199 Jul  6  2020 duplicacy_linux_x64_2.6.0
16765 -rwxrwxr-x 1 root root 31740199 Jul  9  2020 duplicacy_linux_x64_2.6.1
17057 -rwxrwxr-x 1 root root 32383675 Dec 27  2020 duplicacy_linux_x64_2.7.2
19289 -rwx------ 1 root root 34202683 Nov 12 12:09 duplicacy_linux_x64_3.0.1
17049 -rwxrwxr-x 1 root root 31359571 Jun 21  2020 duplicacy_web_linux_x64_1.3.0
17085 -rwxrwxr-x 1 root root 31419238 Aug  8  2020 duplicacy_web_linux_x64_1.4.0
17125 -rwxrwxr-x 1 root root 31436080 Aug 24  2020 duplicacy_web_linux_x64_1.4.1
17141 -rwxrwxr-x 1 root root 31469417 Jan 19  2021 duplicacy_web_linux_x64_1.5.0
13037 -rwxr-xr-x 1 root root 21721144 May  2  2022 duplicacy_web_linux_x64_1.6.1
 3569 -rwxrwxr-x 1 root root  6324224 Aug 26 07:22 duplicacy_web_linux_x64_1.6.3
pieter@server-1:~$

Yep, just deleted all cached bins, and now it works again.

init.sh runing as user root:root(0:0)
Remote available versions... latest: 1.6.3 stable: 1.6.3
Newest cached local version... None
Using version 1.6.3
Downloading executable from https://acrosync.com/duplicacy-web/duplicacy_web_linux_x64_1.6.3
--2022-11-13 11:24:58--  https://acrosync.com/duplicacy-web/duplicacy_web_linux_x64_1.6.3
Resolving acrosync.com (acrosync.com)... 198.199.87.9
Connecting to acrosync.com (acrosync.com)|198.199.87.9|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21735437 (21M) [application/x-troff-man]
Saving to: '/config/bin/duplicacy_web_linux_x64_1.6.3'
     0K .......... .......... .......... .......... ..........  0%  249K 85s
    50K .......... .......... .......... .......... ..........  0%  373K 71s
   100K .......... .......... .......... .......... ..........  0%  102M 47s
   150K .......... .......... .......... .......... ..........  0%  747K 42s
   200K .......... .......... .......... .......... ..........  1%  752K 39s
   250K .......... .......... .......... .......... ..........  1%  115M 33s
   300K .......... .......... .......... .......... ..........  1%  135M 28s
   350K .......... .......... .......... .......... ..........  1%  757K 28s
   400K .......... .......... .......... .......... ..........  2%  119M 25s
   450K .......... .......... .......... .......... ..........  2%  113M 22s
   500K .......... .......... .......... .......... ..........  2%  113M 20s
   550K .......... .......... .......... .......... ..........  2%  448K 22s
   600K .......... .......... .......... .......... ..........  3%  134M 21s
   650K .......... .......... .......... .......... ..........  3%  245M 19s
   700K .......... .......... .......... .......... ..........  3%  278M 18s
   750K .......... .......... .......... .......... ..........  3%  244M 17s
   800K .......... .......... .......... .......... ..........  4% 2.14M 16s
   850K .......... .......... .......... .......... ..........  4%  138M 15s
   900K .......... .......... .......... .......... ..........  4% 1.14M 15s
   950K .......... .......... .......... .......... ..........  4% 2.15M 15s
  1000K .......... .......... .......... .......... ..........  4%  196M 14s
  1050K .......... .......... .......... .......... ..........  5%  212M 14s
  1100K .......... .......... .......... .......... ..........  5%  271M 13s
  1150K .......... .......... .......... .......... ..........  5%  244M 12s
  1200K .......... .......... .......... .......... ..........  5% 68.5M 12s
  1250K .......... .......... .......... .......... ..........  6%  125M 11s
  1300K .......... .......... .......... .......... ..........  6%  159M 11s
  1350K .......... .......... .......... .......... ..........  6%  787K 11s
  1400K .......... .......... .......... .......... ..........  6% 42.0M 11s
  1450K .......... .......... .......... .......... ..........  7% 65.2M 11s
  1500K .......... .......... .......... .......... ..........  7% 49.3M 10s
  1550K .......... .......... .......... .......... ..........  7% 98.4M 10s
  1600K .......... .......... .......... .......... ..........  7% 84.4M 10s
  1650K .......... .......... .......... .......... ..........  8% 1.18M 10s
  1700K .......... .......... .......... .......... ..........  8% 2.18M 10s
  1750K .......... .......... .......... .......... ..........  8% 56.9M 9s
  1800K .......... .......... .......... .......... ..........  8% 42.2M 9s
  1850K .......... .......... .......... .......... ..........  8%  109M 9s
  1900K .......... .......... .......... .......... ..........  9%  112M 9s
  1950K .......... .......... .......... .......... ..........  9% 1.19M 9s
  2000K .......... .......... .......... .......... ..........  9% 2.16M 9s
  2050K .......... .......... .......... .......... ..........  9% 61.4M 9s
  2100K .......... .......... .......... .......... .......... 10% 47.0M 8s
  2150K .......... .......... .......... .......... .......... 10% 74.7M 8s
  2200K .......... .......... .......... .......... .......... 10%  106M 8s
  2250K .......... .......... .......... .......... .......... 10% 1.20M 8s
  2300K .......... .......... .......... .......... .......... 11% 2.13M 8s
  2350K .......... .......... .......... .......... .......... 11% 70.8M 8s
  2400K .......... .......... .......... .......... .......... 11% 42.3M 8s
  2450K .......... .......... .......... .......... .......... 11% 32.3M 8s
  2500K .......... .......... .......... .......... .......... 12% 87.1M 7s
  2550K .......... .......... .......... .......... .......... 12%  111M 7s
  2600K .......... .......... .......... .......... .......... 12% 1.21M 7s
  2650K .......... .......... .......... .......... .......... 12% 2.15M 7s
  2700K .......... .......... .......... .......... .......... 12%  105M 7s
  2750K .......... .......... .......... .......... .......... 13% 28.5M 7s
  2800K .......... .......... .......... .......... .......... 13% 58.5M 7s
  2850K .......... .......... .......... .......... .......... 13% 74.1M 7s
  2900K .......... .......... .......... .......... .......... 13% 1.22M 7s
  2950K .......... .......... .......... .......... .......... 14% 2.15M 7s
  3000K .......... .......... .......... .......... .......... 14% 61.6M 7s
  3050K .......... .......... .......... .......... .......... 14% 73.3M 7s
  3100K .......... .......... .......... .......... .......... 14% 29.4M 7s
  3150K .......... .......... .......... .......... .......... 15% 38.5M 6s
  3200K .......... .......... .......... .......... .......... 15%  127M 6s
  3250K .......... .......... .......... .......... .......... 15% 1.22M 6s
  3300K .......... .......... .......... .......... .......... 15% 2.17M 6s
  3350K .......... .......... .......... .......... .......... 16% 65.9M 6s
  3400K .......... .......... .......... .......... .......... 16% 29.9M 6s
  3450K .......... .......... .......... .......... .......... 16% 39.8M 6s
  3500K .......... .......... .......... .......... .......... 16% 53.6M 6s
  3550K .......... .......... .......... .......... .......... 16% 1.25M 6s
  3600K .......... .......... .......... .......... .......... 17% 2.15M 6s
  3650K .......... .......... .......... .......... .......... 17% 45.5M 6s
  3700K .......... .......... .......... .......... .......... 17% 76.7M 6s
  3750K .......... .......... .......... .......... .......... 17% 37.8M 6s
  3800K .......... .......... .......... .......... .......... 18% 37.2M 6s
  3850K .......... .......... .......... .......... .......... 18% 56.6M 6s
  3900K .......... .......... .......... .......... .......... 18% 1.23M 6s
  3950K .......... .......... .......... .......... .......... 18% 2.20M 6s
  4000K .......... .......... .......... .......... .......... 19% 41.3M 6s
  4050K .......... .......... .......... .......... .......... 19% 80.4M 6s
  4100K .......... .......... .......... .......... .......... 19% 44.8M 6s
  4150K .......... .......... .......... .......... .......... 19% 37.8M 5s
  4200K .......... .......... .......... .......... .......... 20% 48.0M 5s
  4250K .......... .......... .......... .......... .......... 20% 1.24M 5s
  4300K .......... .......... .......... .......... .......... 20% 2.18M 5s
  4350K .......... .......... .......... .......... .......... 20% 47.7M 5s
  4400K .......... .......... .......... .......... .......... 20% 47.0M 5s
  4450K .......... .......... .......... .......... .......... 21% 71.2M 5s
  4500K .......... .......... .......... .......... .......... 21% 35.9M 5s
  4550K .......... .......... .......... .......... .......... 21% 49.1M 5s
  4600K .......... .......... .......... .......... .......... 21% 1.23M 5s
  4650K .......... .......... .......... .......... .......... 22% 2.19M 5s
  4700K .......... .......... .......... .......... .......... 22% 51.9M 5s
  4750K .......... .......... .......... .......... .......... 22%  116M 5s
  4800K .......... .......... .......... .......... .......... 22% 23.3M 5s
  4850K .......... .......... .......... .......... .......... 23% 79.0M 5s
  4900K .......... .......... .......... .......... .......... 23% 55.4M 5s
  4950K .......... .......... .......... .......... .......... 23% 1.23M 5s
  5000K .......... .......... .......... .......... .......... 23% 2.19M 5s
  5050K .......... .......... .......... .......... .......... 24% 54.4M 5s
  5100K .......... .......... .......... .......... .......... 24% 45.4M 5s
  5150K .......... .......... .......... .......... .......... 24%  103M 5s
  5200K .......... .......... .......... .......... .......... 24% 31.2M 5s
  5250K .......... .......... .......... .......... .......... 24% 1.22M 5s
  5300K .......... .......... .......... .......... .......... 25% 85.0M 5s
  5350K .......... .......... .......... .......... .......... 25% 2.19M 5s
  5400K .......... .......... .......... .......... .......... 25% 59.8M 5s
  5450K .......... .......... .......... .......... .......... 25% 49.3M 5s
  5500K .......... .......... .......... .......... .......... 26% 75.2M 5s
  5550K .......... .......... .......... .......... .......... 26% 34.2M 4s
  5600K .......... .......... .......... .......... .......... 26% 1.22M 5s
  5650K .......... .......... .......... .......... .......... 26% 91.2M 4s
  5700K .......... .......... .......... .......... .......... 27% 2.17M 5s
  5750K .......... .......... .......... .......... .......... 27% 64.6M 4s
  5800K .......... .......... .......... .......... .......... 27%  100M 4s
  5850K .......... .......... .......... .......... .......... 27% 46.5M 4s
  5900K .......... .......... .......... .......... .......... 28% 35.4M 4s
  5950K .......... .......... .......... .......... .......... 28% 71.6M 4s
  6000K .......... .......... .......... .......... .......... 28% 1.22M 4s
  6050K .......... .......... .......... .......... .......... 28% 2.17M 4s
  6100K .......... .......... .......... .......... .......... 28% 85.5M 4s
  6150K .......... .......... .......... .......... .......... 29% 91.2M 4s
  6200K .......... .......... .......... .......... .......... 29% 41.0M 4s
  6250K .......... .......... .......... .......... .......... 29% 31.7M 4s
  6300K .......... .......... .......... .......... .......... 29% 98.3M 4s
  6350K .......... .......... .......... .......... .......... 30% 1.22M 4s
  6400K .......... .......... .......... .......... .......... 30% 2.15M 4s
  6450K .......... .......... .......... .......... .......... 30%  127M 4s
  6500K .......... .......... .......... .......... .......... 30% 61.4M 4s
  6550K .......... .......... .......... .......... .......... 31% 57.2M 4s
  6600K .......... .......... .......... .......... .......... 31% 25.5M 4s
  6650K .......... .......... .......... .......... .......... 31%  161M 4s
  6700K .......... .......... .......... .......... .......... 31% 1.22M 4s
  6750K .......... .......... .......... .......... .......... 32%  133M 4s
  6800K .......... .......... .......... .......... .......... 32% 2.15M 4s
  6850K .......... .......... .......... .......... .......... 32% 83.1M 4s
  6900K .......... .......... .......... .......... .......... 32% 52.0M 4s
  6950K .......... .......... .......... .......... .......... 32% 29.0M 4s
  7000K .......... .......... .......... .......... .......... 33% 54.6M 4s
  7050K .......... .......... .......... .......... .......... 33% 1.23M 4s
  7100K .......... .......... .......... .......... .......... 33%  108M 4s
  7150K .......... .......... .......... .......... .......... 33% 2.15M 4s
  7200K .......... .......... .......... .......... .......... 34%  116M 4s
  7250K .......... .......... .......... .......... .......... 34% 58.7M 4s
  7300K .......... .......... .......... .......... .......... 34% 56.4M 4s
  7350K .......... .......... .......... .......... .......... 34% 33.4M 4s
  7400K .......... .......... .......... .......... .......... 35% 65.4M 4s
  7450K .......... .......... .......... .......... .......... 35% 1.23M 4s
  7500K .......... .......... .......... .......... .......... 35% 2.16M 4s
  7550K .......... .......... .......... .......... .......... 35% 64.3M 4s
  7600K .......... .......... .......... .......... .......... 36% 59.3M 4s
  7650K .......... .......... .......... .......... .......... 36%  104M 4s
  7700K .......... .......... .......... .......... .......... 36% 30.9M 3s
  7750K .......... .......... .......... .......... .......... 36% 69.4M 3s
  7800K .......... .......... .......... .......... .......... 36% 1.22M 3s
  7850K .......... .......... .......... .......... .......... 37% 73.6M 3s
  7900K .......... .......... .......... .......... .......... 37% 2.19M 3s
  7950K .......... .......... .......... .......... .......... 37% 64.3M 3s
  8000K .......... .......... .......... .......... .......... 37% 79.1M 3s
  8050K .......... .......... .......... .......... .......... 38% 30.7M 3s
  8100K .......... .......... .......... .......... .......... 38% 60.8M 3s
  8150K .......... .......... .......... .......... .......... 38% 86.4M 3s
  8200K .......... .......... .......... .......... .......... 38% 1.22M 3s
  8250K .......... .......... .......... .......... .......... 39% 2.15M 3s
  8300K .......... .......... .......... .......... .......... 39% 38.1M 3s
  8350K .......... .......... .......... .......... .......... 39% 98.2M 3s
  8400K .......... .......... .......... .......... .......... 39%  101M 3s
  8450K .......... .......... .......... .......... .......... 40% 55.6M 3s
  8500K .......... .......... .......... .......... .......... 40% 47.2M 3s
  8550K .......... .......... .......... .......... .......... 40% 1.23M 3s
  8600K .......... .......... .......... .......... .......... 40% 2.15M 3s
  8650K .......... .......... .......... .......... .......... 40% 49.0M 3s
  8700K .......... .......... .......... .......... .......... 41% 57.3M 3s
  8750K .......... .......... .......... .......... .......... 41% 97.1M 3s
  8800K .......... .......... .......... .......... .......... 41% 50.9M 3s
  8850K .......... .......... .......... .......... .......... 41% 68.7M 3s
  8900K .......... .......... .......... .......... .......... 42% 60.9M 3s
  8950K .......... .......... .......... .......... .......... 42% 1.23M 3s
  9000K .......... .......... .......... .......... .......... 42% 2.17M 3s
  9050K .......... .......... .......... .......... .......... 42% 30.3M 3s
  9100K .......... .......... .......... .......... .......... 43%  140M 3s
  9150K .......... .......... .......... .......... .......... 43%  110M 3s
  9200K .......... .......... .......... .......... .......... 43% 56.7M 3s
  9250K .......... .......... .......... .......... .......... 43% 68.0M 3s
  9300K .......... .......... .......... .......... .......... 44% 1.22M 3s
  9350K .......... .......... .......... .......... .......... 44% 71.5M 3s
  9400K .......... .......... .......... .......... .......... 44% 2.12M 3s
  9450K .......... .......... .......... .......... .......... 44% 17.0M 3s
  9500K .......... .......... .......... .......... .......... 44%  236M 3s
  9550K .......... .......... .......... .......... .......... 45%  242M 3s
  9600K .......... .......... .......... .......... .......... 45%  197M 3s
  9650K .......... .......... .......... .......... .......... 45%  244M 3s
  9700K .......... .......... .......... .......... .......... 45% 1.23M 3s
  9750K .......... .......... .......... .......... .......... 46% 2.16M 3s
  9800K .......... .......... .......... .......... .......... 46% 63.2M 3s
  9850K .......... .......... .......... .......... .......... 46% 16.2M 3s
  9900K .......... .......... .......... .......... .......... 46% 90.1M 3s
  9950K .......... .......... .......... .......... .......... 47%  731K 3s
 10000K .......... .......... .......... .......... .......... 47% 1.22M 3s
 10050K .......... .......... .......... .......... .......... 47% 2.13M 3s
 10100K .......... .......... .......... .......... .......... 47%  152M 3s
 10150K .......... .......... .......... .......... .......... 48%  230M 3s
 10200K .......... .......... .......... .......... .......... 48%  193M 3s
 10250K .......... .......... .......... .......... .......... 48%  243M 3s
 10300K .......... .......... .......... .......... .......... 48%  159M 3s
 10350K .......... .......... .......... .......... .......... 48%  199M 3s
 10400K .......... .......... .......... .......... .......... 49% 36.8M 3s
 10450K .......... .......... .......... .......... .......... 49%  110M 3s
 10500K .......... .......... .......... .......... .......... 49% 1.21M 3s
 10550K .......... .......... .......... .......... .......... 49%  144M 3s
 10600K .......... .......... .......... .......... .......... 50% 2.13M 3s
 10650K .......... .......... .......... .......... .......... 50% 18.0M 3s
 10700K .......... .......... .......... .......... .......... 50%  106M 3s
 10750K .......... .......... .......... .......... .......... 50%  103M 3s
 10800K .......... .......... .......... .......... .......... 51% 1.22M 3s
 10850K .......... .......... .......... .......... .......... 51% 2.11M 3s
 10900K .......... .......... .......... .......... .......... 51% 82.2M 3s
 10950K .......... .......... .......... .......... .......... 51% 18.8M 2s
 11000K .......... .......... .......... .......... .......... 52%  114M 2s
 11050K .......... .......... .......... .......... .......... 52% 1.22M 2s
 11100K .......... .......... .......... .......... .......... 52%  111M 2s
 11150K .......... .......... .......... .......... .......... 52% 2.12M 2s
 11200K .......... .......... .......... .......... .......... 53% 18.8M 2s
 11250K .......... .......... .......... .......... .......... 53% 82.5M 2s
 11300K .......... .......... .......... .......... .......... 53% 78.2M 2s
 11350K .......... .......... .......... .......... .......... 53% 1.23M 2s
 11400K .......... .......... .......... .......... .......... 53% 2.11M 2s
 11450K .......... .......... .......... .......... .......... 54%  104M 2s
 11500K .......... .......... .......... .......... .......... 54% 21.8M 2s
 11550K .......... .......... .......... .......... .......... 54% 83.3M 2s
 11600K .......... .......... .......... .......... .......... 54% 1.21M 2s
 11650K .......... .......... .......... .......... .......... 55% 78.1M 2s
 11700K .......... .......... .......... .......... .......... 55% 2.14M 2s
 11750K .......... .......... .......... .......... .......... 55% 20.5M 2s
 11800K .......... .......... .......... .......... .......... 55% 84.3M 2s
 11850K .......... .......... .......... .......... .......... 56% 93.2M 2s
 11900K .......... .......... .......... .......... .......... 56% 1.21M 2s
 11950K .......... .......... .......... .......... .......... 56% 58.8M 2s
 12000K .......... .......... .......... .......... .......... 56% 2.14M 2s
 12050K .......... .......... .......... .......... .......... 57% 23.6M 2s
 12100K .......... .......... .......... .......... .......... 57% 89.1M 2s
 12150K .......... .......... .......... .......... .......... 57% 97.7M 2s
 12200K .......... .......... .......... .......... .......... 57% 1.21M 2s
 12250K .......... .......... .......... .......... .......... 57% 65.0M 2s
 12300K .......... .......... .......... .......... .......... 58% 2.15M 2s
 12350K .......... .......... .......... .......... .......... 58% 23.4M 2s
 12400K .......... .......... .......... .......... .......... 58% 61.2M 2s
 12450K .......... .......... .......... .......... .......... 58% 1.22M 2s
 12500K .......... .......... .......... .......... .......... 59% 71.2M 2s
 12550K .......... .......... .......... .......... .......... 59% 2.18M 2s
 12600K .......... .......... .......... .......... .......... 59% 59.5M 2s
 12650K .......... .......... .......... .......... .......... 59% 21.6M 2s
 12700K .......... .......... .......... .......... .......... 60% 82.5M 2s
 12750K .......... .......... .......... .......... .......... 60% 1.21M 2s
 12800K .......... .......... .......... .......... .......... 60% 57.7M 2s
 12850K .......... .......... .......... .......... .......... 60% 2.18M 2s
 12900K .......... .......... .......... .......... .......... 61% 14.4M 2s
 12950K .......... .......... .......... .......... .......... 61% 95.1M 2s
 13000K .......... .......... .......... .......... .......... 61%  133M 2s
 13050K .......... .......... .......... .......... .......... 61% 1.24M 2s
 13100K .......... .......... .......... .......... .......... 61% 63.2M 2s
 13150K .......... .......... .......... .......... .......... 62% 2.16M 2s
 13200K .......... .......... .......... .......... .......... 62% 15.8M 2s
 13250K .......... .......... .......... .......... .......... 62%  109M 2s
 13300K .......... .......... .......... .......... .......... 62%  109M 2s
 13350K .......... .......... .......... .......... .......... 63% 1.23M 2s
 13400K .......... .......... .......... .......... .......... 63% 54.3M 2s
 13450K .......... .......... .......... .......... .......... 63% 2.18M 2s
 13500K .......... .......... .......... .......... .......... 63% 16.3M 2s
 13550K .......... .......... .......... .......... .......... 64%  107M 2s
 13600K .......... .......... .......... .......... .......... 64% 74.4M 2s
 13650K .......... .......... .......... .......... .......... 64% 1.23M 2s
 13700K .......... .......... .......... .......... .......... 64% 74.1M 2s
 13750K .......... .......... .......... .......... .......... 65% 2.16M 2s
 13800K .......... .......... .......... .......... .......... 65% 15.6M 2s
 13850K .......... .......... .......... .......... .......... 65%  111M 2s
 13900K .......... .......... .......... .......... .......... 65% 86.5M 2s
 13950K .......... .......... .......... .......... .......... 65% 1.24M 2s
 14000K .......... .......... .......... .......... .......... 66% 34.0M 2s
 14050K .......... .......... .......... .......... .......... 66% 2.22M 2s
 14100K .......... .......... .......... .......... .......... 66% 16.5M 2s
 14150K .......... .......... .......... .......... .......... 66% 80.1M 2s
 14200K .......... .......... .......... .......... .......... 67% 75.4M 2s
 14250K .......... .......... .......... .......... .......... 67% 1.24M 2s
 14300K .......... .......... .......... .......... .......... 67% 65.3M 2s
 14350K .......... .......... .......... .......... .......... 67% 2.17M 2s
 14400K .......... .......... .......... .......... .......... 68% 16.1M 2s
 14450K .......... .......... .......... .......... .......... 68% 68.3M 2s
 14500K .......... .......... .......... .......... .......... 68% 69.5M 2s
 14550K .......... .......... .......... .......... .......... 68% 1.25M 2s
 14600K .......... .......... .......... .......... .......... 69% 75.9M 2s
 14650K .......... .......... .......... .......... .......... 69% 2.15M 2s
 14700K .......... .......... .......... .......... .......... 69% 18.3M 2s
 14750K .......... .......... .......... .......... .......... 69% 49.0M 2s
 14800K .......... .......... .......... .......... .......... 69% 54.8M 2s
 14850K .......... .......... .......... .......... .......... 70% 1.26M 2s
 14900K .......... .......... .......... .......... .......... 70% 41.7M 2s
 14950K .......... .......... .......... .......... .......... 70% 2.19M 2s
 15000K .......... .......... .......... .......... .......... 70% 17.4M 1s
 15050K .......... .......... .......... .......... .......... 71% 75.9M 1s
 15100K .......... .......... .......... .......... .......... 71% 66.1M 1s
 15150K .......... .......... .......... .......... .......... 71% 1.25M 1s
 15200K .......... .......... .......... .......... .......... 71% 41.5M 1s
 15250K .......... .......... .......... .......... .......... 72% 2.17M 1s
 15300K .......... .......... .......... .......... .......... 72% 20.2M 1s
 15350K .......... .......... .......... .......... .......... 72% 54.2M 1s
 15400K .......... .......... .......... .......... .......... 72% 60.9M 1s
 15450K .......... .......... .......... .......... .......... 73% 1.25M 1s
 15500K .......... .......... .......... .......... .......... 73%  110M 1s
 15550K .......... .......... .......... .......... .......... 73% 2.12M 1s
 15600K .......... .......... .......... .......... .......... 73% 21.0M 1s
 15650K .......... .......... .......... .......... .......... 73% 45.7M 1s
 15700K .......... .......... .......... .......... .......... 74% 71.4M 1s
 15750K .......... .......... .......... .......... .......... 74% 1.25M 1s
 15800K .......... .......... .......... .......... .......... 74%  107M 1s
 15850K .......... .......... .......... .......... .......... 74% 2.11M 1s
 15900K .......... .......... .......... .......... .......... 75% 22.7M 1s
 15950K .......... .......... .......... .......... .......... 75% 48.4M 1s
 16000K .......... .......... .......... .......... .......... 75% 70.1M 1s
 16050K .......... .......... .......... .......... .......... 75% 1.24M 1s
 16100K .......... .......... .......... .......... .......... 76%  123M 1s
 16150K .......... .......... .......... .......... .......... 76% 2.10M 1s
 16200K .......... .......... .......... .......... .......... 76% 24.9M 1s
 16250K .......... .......... .......... .......... .......... 76% 45.1M 1s
 16300K .......... .......... .......... .......... .......... 77% 85.3M 1s
 16350K .......... .......... .......... .......... .......... 77% 46.4M 1s
 16400K .......... .......... .......... .......... .......... 77% 1.25M 1s
 16450K .......... .......... .......... .......... .......... 77% 2.10M 1s
 16500K .......... .......... .......... .......... .......... 77% 64.4M 1s
 16550K .......... .......... .......... .......... .......... 78% 23.7M 1s
 16600K .......... .......... .......... .......... .......... 78% 76.2M 1s
 16650K .......... .......... .......... .......... .......... 78% 57.9M 1s
 16700K .......... .......... .......... .......... .......... 78% 1.24M 1s
 16750K .......... .......... .......... .......... .......... 79% 86.8M 1s
 16800K .......... .......... .......... .......... .......... 79% 2.09M 1s
 16850K .......... .......... .......... .......... .......... 79% 25.1M 1s
 16900K .......... .......... .......... .......... .......... 79% 77.8M 1s
 16950K .......... .......... .......... .......... .......... 80% 66.3M 1s
 17000K .......... .......... .......... .......... .......... 80% 1.23M 1s
 17050K .......... .......... .......... .......... .......... 80% 78.8M 1s
 17100K .......... .......... .......... .......... .......... 80% 2.12M 1s
 17150K .......... .......... .......... .......... .......... 81% 32.0M 1s
 17200K .......... .......... .......... .......... .......... 81% 42.8M 1s
 17250K .......... .......... .......... .......... .......... 81% 70.0M 1s
 17300K .......... .......... .......... .......... .......... 81% 1.23M 1s
 17350K .......... .......... .......... .......... .......... 81% 78.4M 1s
 17400K .......... .......... .......... .......... .......... 82% 2.11M 1s
 17450K .......... .......... .......... .......... .......... 82% 41.3M 1s
 17500K .......... .......... .......... .......... .......... 82% 38.6M 1s
 17550K .......... .......... .......... .......... .......... 82% 98.1M 1s
 17600K .......... .......... .......... .......... .......... 83% 1.22M 1s
 17650K .......... .......... .......... .......... .......... 83% 89.0M 1s
 17700K .......... .......... .......... .......... .......... 83% 2.11M 1s
 17750K .......... .......... .......... .......... .......... 83% 91.0M 1s
 17800K .......... .......... .......... .......... .......... 84% 26.6M 1s
 17850K .......... .......... .......... .......... .......... 84% 69.7M 1s
 17900K .......... .......... .......... .......... .......... 84% 47.3M 1s
 17950K .......... .......... .......... .......... .......... 84% 1.23M 1s
 18000K .......... .......... .......... .......... .......... 85% 2.12M 1s
 18050K .......... .......... .......... .......... .......... 85% 72.3M 1s
 18100K .......... .......... .......... .......... .......... 85% 36.4M 1s
 18150K .......... .......... .......... .......... .......... 85% 41.7M 1s
 18200K .......... .......... .......... .......... .......... 85% 56.3M 1s
 18250K .......... .......... .......... .......... .......... 86% 1.24M 1s
 18300K .......... .......... .......... .......... .......... 86% 69.1M 1s
 18350K .......... .......... .......... .......... .......... 86% 2.13M 1s
 18400K .......... .......... .......... .......... .......... 86% 32.9M 1s
 18450K .......... .......... .......... .......... .......... 87% 22.6M 1s
 18500K .......... .......... .......... .......... .......... 87%  119M 1s
 18550K .......... .......... .......... .......... .......... 87% 1.25M 1s
 18600K .......... .......... .......... .......... .......... 87% 54.9M 1s
 18650K .......... .......... .......... .......... .......... 88% 2.14M 1s
 18700K .......... .......... .......... .......... .......... 88% 75.9M 1s
 18750K .......... .......... .......... .......... .......... 88% 23.3M 1s
 18800K .......... .......... .......... .......... .......... 88% 34.9M 1s
 18850K .......... .......... .......... .......... .......... 89% 1.25M 1s
 18900K .......... .......... .......... .......... .......... 89% 68.3M 1s
 18950K .......... .......... .......... .......... .......... 89% 76.5M 1s
 19000K .......... .......... .......... .......... .......... 89% 2.15M 1s
 19050K .......... .......... .......... .......... .......... 89% 36.3M 0s
 19100K .......... .......... .......... .......... .......... 90% 21.5M 0s
 19150K .......... .......... .......... .......... .......... 90% 78.5M 0s
 19200K .......... .......... .......... .......... .......... 90% 1.25M 0s
 19250K .......... .......... .......... .......... .......... 90% 68.9M 0s
 19300K .......... .......... .......... .......... .......... 91% 2.15M 0s
 19350K .......... .......... .......... .......... .......... 91% 86.3M 0s
 19400K .......... .......... .......... .......... .......... 91% 18.5M 0s
 19450K .......... .......... .......... .......... .......... 91% 61.0M 0s
 19500K .......... .......... .......... .......... .......... 92% 1.25M 0s
 19550K .......... .......... .......... .......... .......... 92% 55.4M 0s
 19600K .......... .......... .......... .......... .......... 92% 2.16M 0s
 19650K .......... .......... .......... .......... .......... 92% 58.4M 0s
 19700K .......... .......... .......... .......... .......... 93% 21.3M 0s
 19750K .......... .......... .......... .......... .......... 93% 54.3M 0s
 19800K .......... .......... .......... .......... .......... 93% 53.3M 0s
 19850K .......... .......... .......... .......... .......... 93% 1.26M 0s
 19900K .......... .......... .......... .......... .......... 93% 56.0M 0s
 19950K .......... .......... .......... .......... .......... 94% 2.17M 0s
 20000K .......... .......... .......... .......... .......... 94% 46.1M 0s
 20050K .......... .......... .......... .......... .......... 94% 21.2M 0s
 20100K .......... .......... .......... .......... .......... 94% 46.3M 0s
 20150K .......... .......... .......... .......... .......... 95% 1.25M 0s
 20200K .......... .......... .......... .......... .......... 95% 73.5M 0s
 20250K .......... .......... .......... .......... .......... 95% 2.14M 0s
 20300K .......... .......... .......... .......... .......... 95%  108M 0s
 20350K .......... .......... .......... .......... .......... 96% 21.1M 0s
 20400K .......... .......... .......... .......... .......... 96% 41.7M 0s
 20450K .......... .......... .......... .......... .......... 96% 1.24M 0s
 20500K .......... .......... .......... .......... .......... 96% 98.9M 0s
 20550K .......... .......... .......... .......... .......... 97% 82.4M 0s
 20600K .......... .......... .......... .......... .......... 97% 2.15M 0s
 20650K .......... .......... .......... .......... .......... 97% 83.0M 0s
 20700K .......... .......... .......... .......... .......... 97% 19.3M 0s
 20750K .......... .......... .......... .......... .......... 97% 70.9M 0s
 20800K .......... .......... .......... .......... .......... 98% 1.14M 0s
 20850K .......... .......... .......... .......... .......... 98%  111M 0s
 20900K .......... .......... .......... .......... .......... 98% 2.49M 0s
 20950K .......... .......... .......... .......... .......... 98% 92.3M 0s
 21000K .......... .......... .......... .......... .......... 99% 23.5M 0s
 21050K .......... .......... .......... .......... .......... 99% 43.9M 0s
 21100K .......... .......... .......... .......... .......... 99% 1.14M 0s
 21150K .......... .......... .......... .......... .......... 99% 98.1M 0s
 21200K .......... .......... ......                          100% 97.2M=4.9s
2022-11-13 11:25:04 (4.20 MB/s) - '/config/bin/duplicacy_web_linux_x64_1.6.3' saved [21735437/21735437]
Using machine-id = 30baf6a3e56b852d944d4bfb5d6ae876
duplicacy_web runing as user root:root(0:0)
Logging tail of the log from this moment on
Starting duplicacy_web
Log directory set to /logs
Duplicacy Web Edition 1.6.3 (31BDE1)
Starting the web server at http://[::]:3875
2022/11/13 11:25:04 Temporary directory set to /cache
2022/11/13 11:25:04 Schedule Daily next run time: 2022-1114 00:00
2022/11/13 11:25:04 Duplicacy Web Edition 1.6.3 (31BDE1)
2022/11/13 11:25:04 Downloading the CLI executable from https://github.com/gilbertchen/duplicacy/releases/download/v3.0.1/duplicacy_linux_x64_3.0.1
2022/11/13 11:25:06 Duplicacy CLI 3.0.1

@saspus maybe it would be good to have some kind of validation that the cached bins have 1 correctly downloaded and 2 have not changed or are still valid.

1 Like
#11

Whoa. Ok, this is the culprit then. It shall be around 20MB as well.

It’s fascinating that this partial executable still runs, but apparently fails when the code in the missing part of the image is trying to get accessed.

its unclear why was partial file saved by wget. The init script checks it’s return value and was supposed to delete the partial file.

To recover — delete that partial file, and restart the container.

@gchen, perhaps duplicacy_web_version URL shall report hash of the executables as well (in a backwards compatible way, e.g. add .latest_hash, and .stable_hash, and I’ll add checks for it during downloads in the container.

#12

Sorry, I missed this last paragraph somehow

Yes definitely. There is a check for a download success:

https://bitbucket.org/saspus/duplicacy-web-docker-container/src/35716f81d92e43dec396ae224c7858275304b8e1/init.sh#lines-124

And it seemed to work when I was testing it by interrupting connection. Not sure why it failed here, but either way we can no longer rely on it.

If @gchen adds hash to the json reported by https://duplicacy.com/latest_web_version, e.g.


{
    "latest":"1.6.3",
    "stable":"1.6.3",
    "hashes":  {
    	"latest": {
            "linux_x64": "abf34242eb......544bb",
		    "osx_x64": "abf34242eb......544bb",
    		"osx_aarm64": "3234bf53......a8f729",
    		"freebsd_x64": "ac7d76s7......891f57",
    		...
    	}
		"stable": {
		...
		}
}

I will add the check to the container.

Checking that the executable hasn’t changed between invocations is a great idea!

Automatically update duplicacy web
#13

https://duplicacy.com/latest_web_version now returns:

{
   "hashes" : {
      "1.6.3" : {
         "freebsd_x64" : "69c7338c5864d4cefbf5ffb6f2745475d6806191cbddf229b7a2690c0b329217",
         "linux_arm" : "28033dafc2b6dfa6c78ea028736f947b8c5dc138b4f9629d0e0616c26cfd05f7",
         "linux_arm64" : "43d09b28a3b1a8ec06369ee86982c9a74a817f2e52cb8e7d18122fd808f93f95",
         "linux_x64" : "7e20fefb806578792002199246596d86246a013a11892c68c9b7365d3b080661"
      }
   },
   "latest" : "1.6.3",
   "stable" : "1.6.3"
}
2 Likes
#14

Awesome!! Thank you for such a lighting quick response, during the weekend no less!

I’m going add checks for that to the container tonight.

Which hash is it? Sha256? Md5?

#15

Sorry I forgot to mention that it is Sha256.

1 Like
#16

So the reason for the segmentation fault ( whatever that is) in this thread was that the binary download was incomplete but the binary still worked partially?

#17

Yep, this seems to be the case. Weird, but who knows what optimizations go compiler is doing :slight_smile:

#18

All, I’ve added the check for hash in the :mini container: Bitbucket

New mini one has been pushed to docker hub.

Thank you @pieter for discovering and hunting down the issue, and @gchen for a quick turnaround publishing the hashes on the web service.

I’ve noticed, not all available binaries have hash published, for example macOS is missing, but for the purposes of the docker container updates only linux is required anyway.

I will update my TrueNAS plugin shortly as well.

#19

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.