Verifying releases is not possible currently (no checksums, no reproducible build info, and building from source fails)

dev
#1

My whole life is in backups, so I need to verify releases when I setup duplicacy on a new machine. I tried today to verify a downloaded binary release using a checksum.

Since there are no published checksums (that I know of), I instead tried to rebuild it locally. Not wanting to guess about which command was used to build the source, I tried gorepro, and was greeted with an error message:

$ go install github.com/capnspacehook/gorepro@latest
$ go/bin/gorepro Downloads/duplicacy_linux_x64_3.1.0
"Downloads/duplicacy_linux_x64_3.1.0" was built with go1.16.15, only go1.18 or newer embeds build metadata that is required by gorepro

So, either add checksums anywhere (could be in a forum post from a trusted user for all I care), or update to a newer version of Go, or give guidance (straight in the project README) on how to verify releases if you have another preferred way. Or provide OS packages. Anything so that I don’t just download a binary and run it without being able to verify it’s the correct one. And no, I don’t trust duplicacy on my previous machine, because I’m rotating away from it right now, and the old machine is now untrusted.

#2

go install also does not seem to be possible. The latest version 3.1.0 is not ‘seen’. So we’re back to verifying release binaries or reproducing the builds, no matter how I look at it.

$ go list -versions -m github.com/gilbertchen/duplicacy
github.com/gilbertchen/duplicacy v0.1.1 v0.1.5 v0.1.6 v0.1.7 v0.1.8 v0.1.10 v1.0.0 v1.0.1 v1.1.0 v1.1.3
 v1.1.4 v1.1.5 v1.1.6 v1.1.8 v1.1.9 v1.2.0 v1.2.1 v1.2.3 v1.2.4 v1.2.5 v2.0.0+incompatible 
v2.0.2+incompatible v2.0.3+incompatible v2.0.4+incompatible v2.0.5+incompatible v2.0.6+incompatible 
v2.0.7+incompatible v2.0.9+incompatible v2.0.10+incompatible v2.1.0+incompatible v2.1.1+incompatible 
v2.1.2+incompatible v2.2.0+incompatible v2.2.1+incompatible v2.2.2+incompatible v2.2.3+incompatible 
v2.3.0+incompatible v2.4.0+incompatible v2.4.1+incompatible v2.5.0+incompatible v2.5.1+incompatible 
v2.5.2+incompatible v2.6.0+incompatible v2.6.1+incompatible v2.6.2+incompatible v2.7.0+incompatible 
v2.7.1+incompatible v2.7.2+incompatible
#3

I’m also having difficulty building from source. So I don’t really have any way to ensure I have a binary hat I can tell for sure it’s coming from the official repository. Unclear how much of this pain is due to my not understanding how building Go projects is supposed to be done. You’ll see from the various attempts that it’s not a plug and play affair.

$ echo "layout go" > .envrc # clean .direnv GOPATH
$ direnv allow
...
$ go version
go version go1.18.1 linux/amd64

First try: go get

$ GO111MODULE=off go get -u -v github.com/gilbertchen/duplicacy/duplicacy
cannot find package "github.com/golang-jwt/jwt/v4" in any of:
	/usr/lib/go-1.18/src/github.com/golang-jwt/jwt/v4 (from $GOROOT)
	/SCRUBBED/.direnv/go/src/github.com/golang-jwt/jwt/v4 (from $GOPATH)
code in directory /SCRUBBED/.direnv/go/src/github.com/spacemonkeygo/monkit expects import "github.com/spacemonkeygo/monkit/v3"

Second try: go install v2.7.2 (the one I actually wanted)

$ go install github.com/gilbertchen/duplicacy/duplicacy@v2.7.2 2>&1 | grep -vE 'finding|found|downloading'
# github.com/gilbertchen/duplicacy/src
.direnv/go/pkg/mod/github.com/gilbertchen/duplicacy@v2.7.2+incompatible/src/duplicacy_dropboxstorage.go:28:48: not enough arguments in call to dropbox.NewConfig
	have (string)
	want (string, string, string)

Third try: go install v3.1.0

$ go install github.com/gilbertchen/duplicacy/duplicacy@v3.1.0 2>&1 | grep -vE 'finding|found|downloading'
go: github.com/gilbertchen/duplicacy/duplicacy@v3.1.0: github.com/gilbertchen/duplicacy@v3.1.0: invalid version: module contains a go.mod file, so module path must match major version ("github.com/gilbertchen/duplicacy/v3")

I then gave up.

#4

Your first attempt was almost correct. You need to run this instead:

GO111MODULE=off go get -v -u github.com/gilbertchen/duplicacy/...

#5

4th try: go get with ‘…’

Same as in the first try, it just takes longer to get there because it tries to build even more things.

But the result is the same, it fails.

$ GO111MODULE=off go get -u github.com/gilbertchen/duplicacy/...
cannot find package "github.com/golang-jwt/jwt/v4" in any of:
	/usr/lib/go-1.18/src/github.com/golang-jwt/jwt/v4 (from $GOROOT)
	/SCRUBBED/.direnv/go/src/github.com/golang-jwt/jwt/v4 (from $GOPATH)
code in directory /SCRUBBED/.direnv/go/src/github.com/spacemonkeygo/monkit expects import "github.com/spacemonkeygo/monkit/v3"

It is possible that go get is a waste of time if the main branch does not currently build.

go install takes a version tag, and would be my favourite way to build from source. But as I showed, that does not work either. I’m pretty dumbfounded.

#6

You’re right, something is off. It used to work just a few weeks ago, but is now broken. It seems that for some reason get pulls the latest (?) version of golang-jwt, and in the latest version they just switched to experimental v5, so that jwt/v4 imports fail. Weird, as explicit references seem to require v4@v4.5.0.

I mean, if all you want is to build, you can replace v5 with v4 in github.com/golang-jwt/jwt/go.mod (golang-jwt v5 will pretend to be v4) and this will compile, but it is certainly not a proper solution.

@gchen will probably need to take a closer look at dependencies.

#7

Latest master builds for me with go build duplicacy/duplicacy_main.go from a local clone of the repo.

go version reports go version go1.18.9 linux/amd64. (on AlmaLinux 8)

Sorry I don’t know much else; I’m not familiar with Go, just Googled enough to get a working dev environment.

#8

This should work:

git clone https://github.com/gilbertchen/duplicacy
cd duplicacy
go build duplicacy/duplicacy_main.go

Go modules has been enabled so go get -u github.com/gilbertchen/duplicacy/... won’t work. It gives the following error:

go: go.mod file not found in current directory or any parent directory.
	'go get' is no longer supported outside a module.
	To build and install a command, use 'go install' with a version,

go install won’t work either. I need to figure out what needs to be done to support it.

Build Duplicacy from source
#9

I switched to using a nix flake.

$ cat flake.nix 
{
  description = "Duplicacy 2.7.2";

  inputs = {
    # https://github.com/NixOS/nixpkgs/commit/2a1fcb02eb8d8c589f851b49f4e8b59a0e2d2eec
    nixpkgs.url = "github:nixos/nixpkgs/2a1fcb02eb8d8c589f851b49f4e8b59a0e2d2eec";
  };

  outputs = { self, nixpkgs }: {
    packages.x86_64-linux.duplicacy = nixpkgs.legacyPackages.x86_64-linux.duplicacy;
    packages.x86_64-linux.default = self.packages.x86_64-linux.duplicacy;
  };
}
$ /usr/bin/nix --version
nix (Nix) 2.6.0
$ /usr/bin/nix --extra-experimental-features 'nix-command flakes' build --impure .#default
$ readlink -e result/bin/duplicacy 
/nix/store/8blg5l4mxnv46qygicfiv1dj3575gn3k-duplicacy-2.7.2/bin/duplicacy
$ result/bin/duplicacy -help | grep -A1 VERSION | tail -1
   2.7.2 (unofficial)
$ uname -si
Linux x86_64
$ sha256sum result/bin/duplicacy
415aaca6d0f7824ff7de95c5372fcd3baf254f8cc7522bdfbdb3fe32473e432c  result/bin/duplicacy

I’m now trying to understand how to use nix to produce a build of 2.7.2 (intentionally not the latest version) with upgraded dependencies (in this case glibc) to remove scanned vulnerabilities.

$ vulnxscan result/bin/duplicacy 
INFO     Generating SBOM for target '/nix/store/8blg5l4mxnv46qygicfiv1dj3575gn3k-duplicacy-2.7.2/bin/duplicacy'
INFO     Loading runtime dependencies referenced by '/nix/store/8blg5l4mxnv46qygicfiv1dj3575gn3k-duplicacy-2.7.2/bin/duplicacy'
INFO     Using SBOM '/tmp/vulnxscan_ib_mlj78.json'
INFO     Running vulnix scan
INFO     Running grype scan
INFO     Running OSV scan
INFO     Querying vulnerabilities
INFO     Console report

Potential vulnerabilities impacting 'result/bin/duplicacy' or some of its runtime dependencies:

| vuln_id        | url                                             | package   | version   |  grype  |  osv  |  vulnix  |  sum  |
|----------------+-------------------------------------------------+-----------+-----------+---------+-------+----------+-------|
| CVE-2022-23219 | https://nvd.nist.gov/vuln/detail/CVE-2022-23219 | glibc     | 2.32-10   |    0    |   0   |    1     |   1   |
| CVE-2022-23218 | https://nvd.nist.gov/vuln/detail/CVE-2022-23218 | glibc     | 2.32-10   |    0    |   0   |    1     |   1   |
| CVE-2021-38604 | https://nvd.nist.gov/vuln/detail/CVE-2021-38604 | glibc     | 2.32-10   |    0    |   0   |    1     |   1   |
| CVE-2021-27645 | https://nvd.nist.gov/vuln/detail/CVE-2021-27645 | glibc     | 2.32-10   |    0    |   0   |    1     |   1   |

INFO     Wrote: vulns.csv

It does not matter to me whether they are false positives or not, it’s about being able to build a version with upgraded dependencies in some reproducible way. The curent setup for the developer who is not proficient in go is too janky. I’ll report what I find. And then I’ll apply that knowledge in building a more recent version. I can’t use anything other than 2.7.2 because I migrate laptops at the moment, and upgrading duplicacy was not part of the plan. But I still won’t compromise on security (the subject of this post).