HTTPS Certificates for Private Servers

Hi everyone,

New Duplicacy user – trying to switch over from Arq – purchased a personal license yesterday. I’ll need more licenses if I can successfully get my PoC working and resolve open issues. Here’s one that I’ve not been able to solve through searching the forum:

I’ve tried multiple approaches to having the Duplicacy Web edition issue Let’s Encrypt certificates for the server I have running on my local network that is using RFC1918 private addresses.

Forum posts I found that didn’t resolve my question:

Here’s the situation:

  • I have an instance of Duplicacy Web 1.3.0 running in a jail on my FreeBSD NAS (FreeNAS).
  • I have Duplicacy setup and running scheduled backup jobs to my Backblaze B2 account.
  • Duplicacy is being accessed at 10.10.10.10:80 but I’ve also tried 10.10.10.10:3875
  • I created a DNS A Record (fictitious but representative example) of duplicacy . home . mydomain . com that is globally (and publicly) resolvable to 10.10.10.10.

All is working just fine and I can access my Duplicacy web instance insecurely (HTTP) on ports 80 or 3875 using a settings.json such as:

{
    "listening_address": ":80",
    "temporary_directory": "/root/.duplicacy-web/repositories",
    "log_directory": "/root/.duplicacy-web/logs",
    "dark_mode": false
}

However, if I change this to:

{
    "listening_address": ":80",
    "https_address": ":443",
    "https_domain": "duplicacy.home.mydomain.com",
    "temporary_directory": "/root/.duplicacy-web/repositories",
    "log_directory": "/root/.duplicacy-web/logs",
    "dark_mode": false
}

I see log entries such as:

2020/07/27 08:01:45 http: TLS handshake error from 10.10.10.238:56442: acme/autocert: unable to satisfy "https://acme-v02.api.letsencrypt.org/acme/authz-v3/6142500006" for domain "duplicacy.home.mydomain.com": no viable challenge type found
2020/07/27 08:01:45 http: TLS handshake error from 10.10.10.238:56447: acme/autocert: missing certificate

Note: 10.10.10.238 is the laptop on the same private network I’m access the web interface from.

I realize that typically people are using Let’s Encrypt to create certificates for web services running on the public Internet but there is no reason that you can’t use it for securing servers on a private local network. I have done this successfully with 7 other servers I have running in my home without any problem since I can create A Records that point to my 10...* addresses. However, in those cases, I’ve setup the Let’s Encrypt certbot myself and use a DNS-01 challenge since it does not involve Let’s Encrypt’s servers trying to reach the server on HTTP Port 80 looking for the token (which of course can’t work for 10...* private servers.

My question is whether it is:

  1. It it possible for me to get visibility into the token the Duplicacy certbot is looking for during the challenge? I could put this up on a web server that is publicly accessible so that Let’s Encrypt’s servers can check for the token both during certificate issuance and subsequent renewals.
  2. If #1 isn’t possible, can Duplicacy consider supporting DNS-01 challenges? This would be a lot easier for individuals to host the DNS records for the challenge.

Either way, I think this would be a useful feature as I have no interest (for security reasons) in exposing any Duplicacy servers I setup to the public Internet. However, I don’t want to leave traffic between my Duplicacy web servers and any workstations I’m managing them from as insecure HTTP.

I tried to search through GitHub as well to see if I could find the associated code to assess the level of difficulty in adding DNS-01 challenges (I’m a SW dev as my day job) – but it appears that only the duplicacy CLI source is publicly available.

Thanks in advance for the assistance.

1 Like

The solution: Web-UI security: HTTPS, sessions, and logout button

In my opinion let’s encrypt support for a local application is a pointless and useless gimmick anyway. Custom certificates (issued from my own CA) would have been the way to go, but until that is supported ssh tunneling is good enough if not better by allowing to separate security from the application.

I appreciate you sharing your approach @saspus. I disagree that let’s encrypt support for a local application is a pointless and useless gimmick.

In my opinion, LE certs for HTTPS are valuable and useful because they are simple to use and are intended for exactly this purpose: to encrypt the communication between a web browser and a web server. Once setup, there is also no overhead – the certs are automatically renewed prior to expiration requiring almost 0 overhead on my part.

What you are suggesting is, IMO, a workaround that achieves the desired outcome (encrypting communications) but with an inferior user experience. To access the Duplicacy web interface, you would have to launch a terminal and call a function before the web browser launches (a function defined in your shell profile).

A super UX could be accomplished by supporting DNS-01 challenges. It isn’t clear whether the acme/autocert Go package used by Duplicacy supports dns-01 challenges (although the parent acme package does). Issue here: x/crypto/acme/autocert: Manager should support DNS-01 verification · Issue #23198 · golang/go · GitHub

This package supports http-01, dns-01, and tls-alpn-01 challenges: GitHub - go-acme/lego: Let's Encrypt client and ACME library written in Go

To add, are certificates pointless/useless when used within a corporation or government’s internal network? Not every web server on the planet is accessible via the public Internet. Moreover, the addressing on the public Internet vs. the addressing on a private network are functionally no different in terms of TCP/IP. It only happens to be the case that Internet BGP ASN routers don’t accept RFC1918 prefixes.

In my opinion, LE certs for HTTPS are valuable and useful because they are simple to use and are intended for exactly this purpose: to encrypt the communication between a web browser and a web server. Once setup, there is also no overhead

I don’t argue against a certificate; I’m arguing against lets encrypt specifically. Maintaining web or DNS validation infrastructure and involving third party service just to keep issuing certificates that is going to be used between LAN machines is counterproductive. This also introduces avoidable dependencies on your (DNS) and third party (LE) services and may lead to unintended information disclosure though externally visible domain names (yes, you can get wildcard certificates to mitigate that, but the rest of the points remain and now you have to support synchronizing the certificates with your other services. All for no reason.).

What you are suggesting is, IMO, a workaround that achieves the desired outcome (encrypting communications) but with an inferior user experience.

Correct, this is a workaround until duplicacy supports for custom certificates. Certificates that I can generate myself without requiring third party service.

To access the Duplicacy web interface, you would have to launch a terminal and call a function before the web browser launches (a function defined in your shell profile).

Not really, it can be done in a tmux session on login once, wrapped into auto reconnect loop. Absolutely transparent for any user. The script in that post is just a proof of concept; the presence of sleep and not proper wait for conectivity (with nc for example) should have hinted at that.

A super UX could be accomplished by supporting DNS-01 challenges.

Yeah, no. I don’t want/need to support DNS updates for internal tools. That would be worse workaround. We need support for custom certificates in Duplicacy. And even then, I would prefer SSH solution – SSH is already there and no need to deal with certificates updates.

To add, are certificates pointless/useless when used within a corporation or government’s internal network?

Never said that.

Not every web server on the planet is accessible via the public Internet.

This is precisely why I’m against LE. There is no need to involve third party to issue certificates between two local hosts. LE is not suited for internal use, it’s useful for public facing services only.

At the risk of sounding repetitive, introducing avoidable external dependencies to your internal services is a terrible idea.

Moreover, the addressing on the public Internet vs. the addressing on a private network are functionally no different in terms of TCP/IP.

I"m not sure what this has to do with anything.

We’ll have to agree to disagree. Your opinion on what makes sense for you is just as valid as mine is for what makes sense for me.

I have been managing internal tools with LE certs for years, and it has been very effective. I am not the only one using LE certs for internal servers (e.g., Datto with 90,000 private servers), so my questions as to whether this can be done with Duplicacy remain unanswered.

  1. It it possible for me to get visibility into the token the Duplicacy certbot is looking for during the challenge? I could put this up on a web server that is publicly accessible so that Let’s Encrypt’s servers can check for the token both during certificate issuance and subsequent renewals.
  2. If #1 isn’t possible, can Duplicacy consider supporting DNS-01 challenges? This would be a lot easier for individuals to host the DNS records for the challenge.

Sure. But for the sake of discussion and my education lets talk a bit more. I genuinely want to understand the appeal of LE here.

I have been managing internal tools with LE certs for years, and it has been very effective. I am not the only one using LE certs for internal servers (e.g., Datto with 90,000 private servers)

“very effective” is a subjective and does not explain anything. and that you are not the only one hardly justifies anything.

Looking at the article you linked to I fail to see how creating and maintaining all that is easier than running a script to issue certificate to your appliance once every 823 days from your own CA and copying a file to the target. All I see is a desire to use LE just for the sake of using LE. And what if my appliance does not have internet access? I would need to run certbot elsewhere and deploy certificates to it. Why not deploy my own certificates instead that don’t have to be limited to 3 months validity?

I could put this up on a web server that is publicly

This is a hack on a hack… don’t do that.

If #1 isn’t possible, can Duplicacy consider supporting DNS-01 challenges?

This would be question for @gchen, but I would have removed support for LE altogether and instead support manually specifying path to the certificates. If users want to use LE to obtain those – it’s on them. Duplicacy is a backup program, it should have no business managing certificates, talking to web servers and updating DNS.

@TechPolymath The certificates are stored under ~/.duplicacy-web/certs, so I wonder if it is possible to make it work by downloading the certificates manually and then placing them under that folder.

What should be the format and naming for the certificate and private key?

There are usually 2 files under that directory. One is acme_account+key which I guess is only needed when you renew the certificates. The other is named with your domain name and contains the private key and all certificates concatenated together.

That did not work; with the compound certificate file with extension pem containing the private key and the certificate chain up to the root Duplicacy said this on start:

2020/07/30 03:32:59 http: TLS handshake error from 127.0.0.1:55366: acme/autocert: server name component count invalid