Failed to send the email: x509: certificate signed by unknown authority

Getting this error when trying to set up ‘Send email after completion’ in the Web UI.

The issue seems to have been mentioned before, but with no resolution.

Basically, I’m migrating a client from the legacy UI to the new Web UI.

Email was set up before and working just fine with the legacy UI. Using the exact same server details as before - i.e. companymail.companygroup.local:25 which is internal and self-signed, but also tried using its external FQDN mail.companydomain.com with a proper SSL cert.

This is a local Exchange server, with a special receive connector set up to allow anonymous authentication on port 25 for certain IPs within the local LAN. Thus username and password fields are left blank, as before.

Not much else said in duplicacy-web.log. What is the Web UI doing differently than the legacy UI?

The legacy UI is written in C++ and uses libcurl to send email, while the web UI is written in go and uses the standard net/smtp library.

The legacy UI doesn’t check the certificate at all so that is probably why you’re getting the error with the web GUI. If the certificate for companymail.companygroup.local is self-signed then I think you just need to add the certificate to the trusted root CA list.

Why then doesn’t it work with mail.companydomain.com which has a proper SSL cert? Is there any way to get more debug info about what’s happening during the SMTP authentication?

This go program uses the same code in the web UI to send emails. If you can get this program to work with your SMTP server, the web UI should too.

// This file is copied and modified from /usr/local/go/src/net/smtp/smtp.go
//
// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.

package main

import (
	"crypto/tls"
	"encoding/base64"
	"errors"
	"fmt"
	"io"
	"os"
	"net"
	"net/smtp"
	"net/textproto"
	"strings"
)

// A SMTPClient represents a client connection to an SMTP server.
type SMTPClient struct {
	// Text is the textproto.Conn used by the Client. It is exported to allow for
	// clients to add extensions.
	Text *textproto.Conn
	// keep a reference to the connection so it can be used to create a TLS
	// connection later
	conn net.Conn
	// whether the Client is using TLS
	tls        bool
	serverName string
	// map of supported extensions
	ext map[string]string
	// supported auth mechanisms
	auth       []string
	localName  string // the name to use in HELO/EHLO
	didHello   bool   // whether we've said HELO/EHLO
	helloError error  // the error from the hello
}

// Dial returns a new Client connected to an SMTP server at addr.
// The addr must include a port, as in "mail.example.com:smtp".
func Dial(addr string) (*SMTPClient, error) {
	conn, err := net.Dial("tcp", addr)
	if err != nil {
		return nil, err
	}
	host, _, _ := net.SplitHostPort(addr)
	return NewSMTPClient(conn, host)
}

// NewSMTPClient returns a new Client using an existing connection and host as a
// server name to be used when authenticating.
func NewSMTPClient(conn net.Conn, host string) (*SMTPClient, error) {
	text := textproto.NewConn(conn)
	_, _, err := text.ReadResponse(220)
	if err != nil {
		text.Close()
		return nil, err
	}
	c := &SMTPClient{Text: text, conn: conn, serverName: host, localName: "acrosync"}
	_, c.tls = conn.(*tls.Conn)
	return c, nil
}

// Close closes the connection.
func (c *SMTPClient) Close() error {
	return c.Text.Close()
}

// hello runs a hello exchange if needed.
func (c *SMTPClient) hello() error {
	if !c.didHello {
		c.didHello = true
		err := c.ehlo()
		if err != nil {
			c.helloError = c.helo()
		}
	}
	return c.helloError
}

// Hello sends a HELO or EHLO to the server as the given host name.
// Calling this method is only necessary if the client needs control
// over the host name used. The client will introduce itself as "duplicacy"
// automatically otherwise. If Hello is called, it must be called before
// any of the other methods.
func (c *SMTPClient) Hello(localName string) error {
	if err := validateSMTPLine(localName); err != nil {
		return err
	}
	if c.didHello {
		return errors.New("smtp: Hello called after other methods")
	}
	c.localName = localName
	return c.hello()
}

// cmd is a convenience function that sends a command and returns the response
func (c *SMTPClient) cmd(expectCode int, format string, args ...interface{}) (int, string, error) {
	id, err := c.Text.Cmd(format, args...)
	if err != nil {
		return 0, "", err
	}
	c.Text.StartResponse(id)
	defer c.Text.EndResponse(id)
	code, msg, err := c.Text.ReadResponse(expectCode)
	return code, msg, err
}

// helo sends the HELO greeting to the server. It should be used only when the
// server does not support ehlo.
func (c *SMTPClient) helo() error {
	c.ext = nil
	_, _, err := c.cmd(250, "HELO %s", c.localName)
        fmt.Printf("helo: %v\n", err)
	return err
}

// ehlo sends the EHLO (extended hello) greeting to the server. It
// should be the preferred greeting for servers that support it.
func (c *SMTPClient) ehlo() error {
        fmt.Printf("ehol()\n")
	_, msg, err := c.cmd(250, "EHLO %s", c.localName)
	if err != nil {
                return fmt.Errorf("ehlo err: %v", err)
	}
	ext := make(map[string]string)
	extList := strings.Split(msg, "\n")
	if len(extList) > 1 {
		extList = extList[1:]
		for _, line := range extList {
			args := strings.SplitN(line, " ", 2)
			if len(args) > 1 {
				ext[args[0]] = args[1]
			} else {
				ext[args[0]] = ""
			}
		}
	}
	if mechs, ok := ext["AUTH"]; ok {
		c.auth = strings.Split(mechs, " ")
	}
	c.ext = ext
	return err
}

// StartTLS sends the STARTTLS command and encrypts all further communication.
// Only servers that advertise the STARTTLS extension support this function.
func (c *SMTPClient) StartTLS(config *tls.Config) error {
	if err := c.hello(); err != nil {
		return err
	}
	_, _, err := c.cmd(220, "STARTTLS")
	if err != nil {
		return err
	}
	c.conn = tls.Client(c.conn, config)
	c.Text = textproto.NewConn(c.conn)
	c.tls = true
	return c.ehlo()
}

// Auth authenticates a client using the provided authentication mechanism.
// A failed authentication closes the connection.
// Only servers that advertise the AUTH extension support this function.
func (c *SMTPClient) Auth(a smtp.Auth) error {
	if err := c.hello(); err != nil {
		return err
	}
	encoding := base64.StdEncoding
	mech, resp, err := a.Start(&smtp.ServerInfo{c.serverName, c.tls, c.auth})
	if err != nil {
		c.Quit()
		return err
	}
	resp64 := make([]byte, encoding.EncodedLen(len(resp)))
	encoding.Encode(resp64, resp)
	code, msg64, err := c.cmd(0, strings.TrimSpace(fmt.Sprintf("AUTH %s %s", mech, resp64)))
	for err == nil {
		var msg []byte
		switch code {
		case 334:
			msg, err = encoding.DecodeString(msg64)
		case 235:
			// the last message isn't base64 because it isn't a challenge
			msg = []byte(msg64)
		default:
			err = &textproto.Error{Code: code, Msg: msg64}
		}
		if err == nil {
			resp, err = a.Next(msg, code == 334)
		}
		if err != nil {
			// abort the AUTH
			c.cmd(501, "*")
			c.Quit()
			break
		}
		if resp == nil {
			break
		}
		resp64 = make([]byte, encoding.EncodedLen(len(resp)))
		encoding.Encode(resp64, resp)
		code, msg64, err = c.cmd(0, string(resp64))
	}
	return err
}

// Mail issues a MAIL command to the server using the provided email address.
// If the server supports the 8BITMIME extension, Mail adds the BODY=8BITMIME
// parameter.
// This initiates a mail transaction and is followed by one or more Rcpt calls.
func (c *SMTPClient) Mail(from string) error {
	if err := validateSMTPLine(from); err != nil {
		return err
	}
	if err := c.hello(); err != nil {
		return err
	}
	cmdStr := "MAIL FROM:<%s>"
	if c.ext != nil {
		if _, ok := c.ext["8BITMIME"]; ok {
			cmdStr += " BODY=8BITMIME"
		}
	}
	_, _, err := c.cmd(250, cmdStr, from)
	return err
}

// Rcpt issues a RCPT command to the server using the provided email address.
// A call to Rcpt must be preceded by a call to Mail and may be followed by
// a Data call or another Rcpt call.
func (c *SMTPClient) Rcpt(to string) error {
	if err := validateSMTPLine(to); err != nil {
		return err
	}
	_, _, err := c.cmd(25, "RCPT TO:<%s>", to)
	return err
}

type dataCloser struct {
	c *SMTPClient
	io.WriteCloser
}

func (d *dataCloser) Close() error {
	d.WriteCloser.Close()
	_, _, err := d.c.Text.ReadResponse(250)
	return err
}

// Data issues a DATA command to the server and returns a writer that
// can be used to write the mail headers and body. The caller should
// close the writer before calling any more methods on c. A call to
// Data must be preceded by one or more calls to Rcpt.
func (c *SMTPClient) Data() (io.WriteCloser, error) {
	_, _, err := c.cmd(354, "DATA")
	if err != nil {
		return nil, err
	}
	return &dataCloser{c, c.Text.DotWriter()}, nil
}

var testHookStartTLS func(*tls.Config) // nil, except for tests

// SendMail connects to the server at addr, switches to TLS if
// possible, authenticates with the optional mechanism a if possible,
// and then sends an email from address from, to addresses to, with
// message msg.
// The addr must include a port, as in "mail.example.com:smtp".
//
// The addresses in the to parameter are the SMTP RCPT addresses.
//
// The msg parameter should be an RFC 822-style email with headers
// first, a blank line, and then the message body. The lines of msg
// should be CRLF terminated. The msg headers should usually include
// fields such as "From", "To", "Subject", and "Cc".  Sending "Bcc"
// messages is accomplished by including an email address in the to
// parameter but not including it in the msg headers.
//
// The SendMail function and the net/smtp package are low-level
// mechanisms and provide no support for DKIM signing, MIME
// attachments (see the mime/multipart package), or other mail
// functionality. Higher-level packages exist outside of the standard
// library.
func SendMail(addr string, a smtp.Auth, from string, to []string, msg []byte) error {
	if err := validateSMTPLine(from); err != nil {
		return err
	}
	for _, recp := range to {
			if err := validateSMTPLine(recp); err != nil {
					return err
			}
	}

	c, err := Dial(addr)
	if err != nil {
		return fmt.Errorf("dial err: %v", err)
	}
	defer c.Close()
	if err = c.hello(); err != nil {
		return fmt.Errorf("hello err: %v", err)
	}
	if ok, _ := c.Extension("STARTTLS"); ok {
		config := &tls.Config{ServerName: c.serverName}
		if testHookStartTLS != nil {
			testHookStartTLS(config)
		}
		if err = c.StartTLS(config); err != nil {
		    return fmt.Errorf("starttls err: %v", err)
		}
	}
	if a != nil && c.ext != nil {
		if _, ok := c.ext["AUTH"]; ok {
			if err = c.Auth(a); err != nil {
		                return fmt.Errorf("auth err: %v", err)
			}
		}
	}
	if err = c.Mail(from); err != nil {
		return fmt.Errorf("mail err: %v", err)
	}
	for _, addr := range to {
		if err = c.Rcpt(addr); err != nil {
		        return fmt.Errorf("rcpt err: %v", err)
		}
	}
	w, err := c.Data()
	if err != nil {
		return fmt.Errorf("data err: %v", err)
	}
	_, err = w.Write(msg)
	if err != nil {
		return fmt.Errorf("write err: %v", err)
	}
	err = w.Close()
	if err != nil {
		return err
	}
	return c.Quit()
}

// Extension reports whether an extension is support by the server.
// The extension name is case-insensitive. If the extension is supported,
// Extension also returns a string that contains any parameters the
// server specifies for the extension.
func (c *SMTPClient) Extension(ext string) (bool, string) {
	if err := c.hello(); err != nil {
		return false, ""
	}
	if c.ext == nil {
		return false, ""
	}
	ext = strings.ToUpper(ext)
	param, ok := c.ext[ext]
	return ok, param
}

// Reset sends the RSET command to the server, aborting the current mail
// transaction.
func (c *SMTPClient) Reset() error {
	if err := c.hello(); err != nil {
		return err
	}
	_, _, err := c.cmd(250, "RSET")
	return err
}

// Noop sends the NOOP command to the server. It does nothing but check
// that the connection to the server is okay.
func (c *SMTPClient) Noop() error {
	if err := c.hello(); err != nil {
		return err
	}
	_, _, err := c.cmd(250, "NOOP")
	return err
}

// Quit sends the QUIT command and closes the connection to the server.
func (c *SMTPClient) Quit() error {
	if err := c.hello(); err != nil {
		return err
	}
	_, _, err := c.cmd(221, "QUIT")
	if err != nil {
		return err
	}
	return c.Text.Close()
}

// validateSMTPLine checks to see if a line has CR or LF as per RFC 5321
func validateSMTPLine(line string) error {
	if strings.ContainsAny(line, "\n\r") {
			return errors.New("smtp: A line must not contain CR or LF")
	}
	return nil
}

type EmailAuth struct {
	username, password string
	useLogin bool
}

func (auth *EmailAuth) Start(server *smtp.ServerInfo) (string, []byte, error) {
	for _, mechanism := range server.Auth {
		if mechanism == "LOGIN" && auth.username != "" && auth.password != "" {
			auth.useLogin = true
			break
		}
	}

	if !auth.useLogin {
		resp := []byte("\x00" + auth.username + "\x00" + auth.password)
		return "PLAIN", resp, nil
	} else {
		return "LOGIN", []byte{}, nil
	}
}

func (auth *EmailAuth) Next(fromServer []byte, more bool) ([]byte, error) {
	if more {
		if !auth.useLogin {
			return nil, fmt.Errorf("unexpected server challenge")
		}
		switch string(fromServer) {
		case "Username:":
			return []byte(auth.username), nil
		case "Password:":
			return []byte(auth.password), nil
		default:
			return nil, fmt.Errorf("Unkown message from server: %s", fromServer)
		}
	}
	return nil, nil
}


func main() {

    if len(os.Args) < 6 {
        fmt.Printf("Usage: %s server username password from to\n", os.Args[0])
        return
    }
  
    server := os.Args[1]
    username := os.Args[2]
    password := os.Args[3]
    from := os.Args[4]
    to := os.Args[5] 
                
    var recipients [] string

	for _, recipient := range strings.FieldsFunc(to, func (r rune) bool {
		return r == ',' || r == ';' || r == ' '
	}) {
		if recipient != "" {
			recipients = append(recipients, recipient)
		}
	}

	message := "To: " + strings.Join(recipients, ", ") + "\r\n" +
		       "From: " + from + "\r\n" +
		       "Subject: This is a test email\r\n" +
	           "\r\n" +
	           "This is for testing only\r\n"

	auth := &EmailAuth { username: username, password: password }

	fmt.Printf("Sending email to %s\n", strings.Join(recipients, ", ") )
	err := SendMail(server, auth, from, recipients, []byte(message))
        if err != nil {
            fmt.Printf("Failed to send email: %v\n", err)
        } else {
            fmt.Printf("Email has been sent successfully\n")
        }
   
}


@gchen, is the web UI using the SendMail function? Or a custom function to send mails?

From the issue below, it sounds like non-ideal certificate setups might require something different when using golang’s net/smtp.

SendMail is only a convenience function, if you want to tweak the tls configuration,
you can smtp.Dail and then do the StartTLS yourself to be able to pass in a
crypto/tls.Config (where you can set InsecureSkipVerify to true to skip the check)

@Droolio This issue is also older, but it makes it sound like macOS might have issues with handling custom certs. Do you happen to be using macOS?

There’s a workaround that may allow skipping the TLS cert verification, but I’m more interested in fixing the mail server…

Turns out the mail server isn’t responding with the correct certificates, and this seems to be a specific issue with Exchange 2013.

Remember, I’m testing this with a public SSL certificate as well as self-signed. BOTH self-signed cert and the real cert are valid and installed correctly, but it appears the previously expired certificates are still bound to the SMTP service and Exchange isn’t following the proper chain.

I believe I need to Remove the SSL Certificate from Exchange but I want to do that in a test environment, so I’m currently restoring a backup of the same server, into a test VM, before tinkering with production. :slight_smile:

1 Like

The web UI uses a slightly modified version of the standard SendMail function. The only change is to support anonymous login. You can find the code in the program I posted above.