Occasional SIGSEGV with Duplicacy 2.7.2 on Ubuntu 20.04.2 LTS

dchenbecker · 5 April 2021 19:18

Please describe what you are doing to trigger the bug:

I have a set of cron jobs that run every hour to back things up. Starting today (April 5, 2021), I’ve seen two different jobs fail due to a SIGSEGV in Duplicacy (2.7.2). I did update some packages (apt upgrade) two days ago, but these cron jobs have run succesfully ~ 40 times since, so I don’t think that’s related. I’ve also been using 2.7.2 since March 13, 2021, so that didn’t change, either.

I’m using a mix of local and Backblaze B2 storage.

Please describe what you expect to happen (but doesn’t):

I expect the backup to complete successfully as it always has

Please describe what actually happens (the wrong behaviour):

I get the following (I’ve replaced “github.com” with GHC below to avoid the forum’s limit on “links”):

unexpected fault address 0x0
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x80 addr=0x0 pc=0xd6e3d8]

goroutine 1 [running]:
runtime.throw(0x101f156, 0x5)
/usr/local/go/src/runtime/panic.go:617 +0x72 fp=0xc001aa0420 sp=0xc001aa03f0 pc=0x42d712
runtime.sigpanic()
/usr/local/go/src/runtime/signal_unix.go:397 +0x401 fp=0xc001aa0450 sp=0xc001aa0420 pc=0x442c81
GHC/gilbertchen/duplicacy/src.(*SnapshotManager).ListAllFiles(0xc0003f4640, 0x1305740, 0xc0001722d0, 0x102124e, 0x7, 0x0, 0x26000, 0x2, 0x10, 0x0, …)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/src/duplicacy_snapshotmanager.go:613 +0x3a8 fp=0xc001aa45d0 sp=0xc001aa0450 pc=0xd6e3d8
GHC/gilbertchen/duplicacy/src.(*BackupManager).CopySnapshots(0xc000030120, 0xc00011c300, 0x0, 0x0, 0x0, 0x0, 0x0, 0x1, 0x1, 0x0)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/src/duplicacy_backupmanager.go:1757 +0x992 fp=0xc001aa4d78 sp=0xc001aa45d0 pc=0xd1e192
main.copySnapshots(0xc00040c240)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/duplicacy/duplicacy_main.go:1275 +0x9fe fp=0xc001aa51f8 sp=0xc001aa4d78 pc=0xdaf0fe
GHC/gilbertchen/duplicacy/vendor/GHC/gilbertchen/cli.Command.Run(0x101e50d, 0x4, 0x0, 0x0, 0x0, 0x0, 0x0, 0x104ad7e, 0x2a, 0x0, …)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/vendor/GHC/gilbertchen/cli/command.go:160 +0x74d fp=0xc001aa5430 sp=0xc001aa51f8 pc=0x7961cd
GHC/gilbertchen/duplicacy/vendor/GHC/gilbertchen/cli.(*App).Run(0xc00016fd40, 0xc0000b0120, 0x6, 0x6, 0x0, 0x0)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/vendor/GHC/gilbertchen/cli/app.go:179 +0x64f fp=0xc001aa5928 sp=0xc001aa5430 pc=0x79357f
main.main()
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/duplicacy/duplicacy_main.go:2201 +0x65b6 fp=0xc001aa7f98 sp=0xc001aa5928 pc=0xdb67d6
runtime.main()
/usr/local/go/src/runtime/proc.go:200 +0x20c fp=0xc001aa7fe0 sp=0xc001aa7f98 pc=0x42f07c
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1337 +0x1 fp=0xc001aa7fe8 sp=0xc001aa7fe0 pc=0x45c8e1

goroutine 18 [syscall, 2 minutes]:
os/signal.signal_recv(0x0)
/usr/local/go/src/runtime/sigqueue.go:139 +0x9c
os/signal.loop()
/usr/local/go/src/os/signal/signal_unix.go:23 +0x22
created by os/signal.init.0
/usr/local/go/src/os/signal/signal_unix.go:29 +0x41

goroutine 49 [chan receive, 2 minutes]:
main.main.func1(0xc0003f6540)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/duplicacy/duplicacy_main.go:2195 +0x60
created by main.main
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/duplicacy/duplicacy_main.go:2194 +0x6584

goroutine 34 [select]:
GHC/gilbertchen/duplicacy/vendor/go.opencensus.io/stats/view.(*worker).start(0xc000172780)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/vendor/go.opencensus.io/stats/view/worker.go:154 +0x100
created by GHC/gilbertchen/duplicacy/vendor/go.opencensus.io/stats/view.init.0
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/vendor/go.opencensus.io/stats/view/worker.go:32 +0x57

goroutine 25 [select]:
net/http.(*persistConn).writeLoop(0xc001aa8120)
/usr/local/go/src/net/http/transport.go:1978 +0x113
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1360 +0xb0d

goroutine 66 [select, 2 minutes]:
GHC/gilbertchen/duplicacy/src.CreateChunkDownloader.func1(0xc001b300b0, 0x0)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/src/duplicacy_chunkdownloader.go:85 +0x162
created by GHC/gilbertchen/duplicacy/src.CreateChunkDownloader
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/src/duplicacy_chunkdownloader.go:82 +0x1e6

goroutine 24 [IO wait]:
internal/poll.runtime_pollWait(0x7f65b1ae3e68, 0x72, 0xffffffffffffffff)
/usr/local/go/src/runtime/netpoll.go:182 +0x56
internal/poll.(*pollDesc).wait(0xc001abe018, 0x72, 0xe900, 0xe969, 0xffffffffffffffff)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x9b
internal/poll.(*pollDesc).waitRead(…)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc001abe000, 0xc003c00000, 0xe969, 0xe969, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:169 +0x19b
net.(*netFD).Read(0xc001abe000, 0xc003c00000, 0xe969, 0xe969, 0x203000, 0x0, 0xc000000034)
/usr/local/go/src/net/fd_unix.go:202 +0x4f
net.(*conn).Read(0xc000125e48, 0xc003c00000, 0xe969, 0xe969, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:177 +0x69
crypto/tls.(*atLeastReader).Read(0xc00149ce00, 0xc003c00000, 0xe969, 0xe969, 0x6, 0xa, 0xc00186a938)
/usr/local/go/src/crypto/tls/conn.go:761 +0x60
bytes.(*Buffer).ReadFrom(0xc0001293d8, 0x12dd780, 0xc00149ce00, 0x409d55, 0xeef1c0, 0xfeb640)
/usr/local/go/src/bytes/buffer.go:207 +0xbd
crypto/tls.(*Conn).readFromUntil(0xc000129180, 0x12deb80, 0xc000125e48, 0x5, 0xc000125e48, 0x203000)
/usr/local/go/src/crypto/tls/conn.go:783 +0xf8
crypto/tls.(*Conn).readRecordOrCCS(0xc000129180, 0x1068300, 0xc0001292b8, 0xc00186ab88)
/usr/local/go/src/crypto/tls/conn.go:590 +0x125
crypto/tls.(*Conn).readRecord(…)
/usr/local/go/src/crypto/tls/conn.go:558
crypto/tls.(*Conn).Read(0xc000129180, 0xc003be6000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/crypto/tls/conn.go:1236 +0x137
net/http.(*persistConn).Read(0xc001aa8120, 0xc003be6000, 0x1000, 0x1000, 0xc00186ac88, 0x404fa5, 0xc0018395c0)
/usr/local/go/src/net/http/transport.go:1526 +0x7b
bufio.(*Reader).fill(0xc001681140)
/usr/local/go/src/bufio/bufio.go:100 +0x10f
bufio.(*Reader).Peek(0xc001681140, 0x1, 0x0, 0x0, 0x1, 0xc01514d800, 0x0)
/usr/local/go/src/bufio/bufio.go:138 +0x4f
net/http.(*persistConn).readLoop(0xc001aa8120)
/usr/local/go/src/net/http/transport.go:1679 +0x1a3
created by net/http.(*Transport).dialConn
/usr/local/go/src/net/http/transport.go:1359 +0xae8

dchenbecker · 5 April 2021 19:19

Just to clarify, the issue is also intermittent. I just confirmed that the latest run succeeded.

gchen · 6 April 2021 14:04

duplicacy_snapshotmanager.go:613 is this line:

github.com

gilbertchen/duplicacy/blob/175adb14cbd86c2bb716c72b0a4bb8e476014a66/src/duplicacy_snapshotmanager.go#L613


			LOG_ERROR("LIST_FILES", "Failed to list the directory %s: %v", dir, err)
			return nil, nil
		}

		if len(dir) > len(top) {
			allFiles = append(allFiles, dir[len(top):])
			allSizes = append(allSizes, 0)
		}

		for i, file := range files {
			if len(file) > 0 && file[len(file)-1] == '/' {
				directories = append(directories, dir+file)
			} else {
				allFiles = append(allFiles, (dir + file)[len(top):])
				allSizes = append(allSizes, sizes[i])
			}
		}
	}

	return allFiles, allSizes
}

I don’t know how this line can cause a SIGSEGV.

Did the other job crash at the same place?

dchenbecker · 6 April 2021 18:22

The other job crashed at duplicacy/duplicacy_entry.go at v2.7.2 · gilbertchen/duplicacy · GitHub, so it doesn’t look like it:

goroutine 1 [running]:
runtime.throw(0x101f156, 0x5)
/usr/local/go/src/runtime/panic.go:617 +0x72 fp=0xc0005ee3f8 sp=0xc0005ee3c8 pc=0x42d712
runtime.sigpanic()
/usr/local/go/src/runtime/signal_unix.go:397 +0x401 fp=0xc0005ee428 sp=0xc0005ee3f8 pc=0x442c81
GHC/gilbertchen/duplicacy/src.(*Entry).Compare(0xc00b626780, 0xc00b226800, 0xffffffffffffffff)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/src/duplicacy_entry.go:345 +0x3d fp=0xc0005ee438 sp=0xc0005ee428 pc=0xd39d1d
GHC/gilbertchen/duplicacy/src.(*SnapshotManager).CheckSnapshot(0xc000156840, 0xc0003081e0, 0x1b0fca0, 0x3a2e)
/Users/chgang/zincbox/go/src/GHC/gilbertchen/duplicacy/src/duplicacy_snapshotmanager.go:2496 +0x34a fp=0xc0005ee7b8 sp=0xc0005ee438 pc=0xd82cfa