Unexpected results

Support
#1

I’ve been running a series of tests to help me understand how Duplicacy works. I’m using a simple folder structure, where repo-1 has a couple of files, and, to start with, storage-1 and storage-2 are empty.

C:\TEMP\DUP-TEST
├───repos
│   └───repo-1
└───storages
    ├───storage-1
    └───storage-2

Below are two sequences of commands, each starting with a fresh copy of the structure above. The only difference between the two sets is that the second one uses the -e option to enable encryption. The encryption key I’m using for everything is “aaaaaaaa”.

I expect that after each batch of commands has run, the folder structure and files of storage-1 and storage-2 will be identical. For the fist batch, without encryption, this holds true. For the second, with encryption, they are different.

In the second case, there are the same number of chunk files in the two storage folders, with the same names and sizes, but the contents are different. The config files are also different. I was under the impression that using the -bit-identical option would ensure that everything remained the same, and that encryption would only effect the config file. Also, by using the same password, I would expect the encrypted config files to be the same.

What am I misunderstanding?

duplicacy  -log  init -storage-name repo-1-storage-1 repo-1 c:\temp\dup-test\storages\storage-1
duplicacy  -log  backup   -storage c:\temp\dup-test\storages\storage-1
duplicacy  -log  add -copy repo-1-storage-1 -bit-identical repo-1-storage-2 repo-1 c:\temp\dup-test\storages\storage-2
duplicacy  -log  backup   -storage c:\temp\dup-test\storages\storage-2

duplicacy  -log  init -e -storage-name repo-1-storage-1 repo-1 c:\temp\dup-test\storages\storage-1
duplicacy  -log  backup   -storage c:\temp\dup-test\storages\storage-1
duplicacy  -log  add -copy repo-1-storage-1 -bit-identical -e repo-1-storage-2 repo-1 c:\temp\dup-test\storages\storage-2
duplicacy  -log  backup   -storage c:\temp\dup-test\storages\storage-2
#2

I think it’s expected result: duplicacy uses AES-GCM for encryption, and therefore you would get different cipher text each time you perform encryption due to different Nonce (initialization vector) used and/or authentication tag. The plaintext is of course the same, as you confirmed in the unencrypted case, and chunk name is derived from plaintext contents.

The config file is different because although encryption keys are copied, storage ID is different.

#3

This would make sense, except the discussion around -bit-identical talks about using rsync or rclone to copy chunks around that makes them sound interchangeable.

If they are not interchangeable, what is the purpose of -bit-identical?

Or are you saying that they are interchangeable, just different (which, I suppose, is something I could test pretty easily with this setup).

#4

Yes, they are both different and interchangeable

#5

I created a little script to randomly combine files from each of my storages (which I verified were different as per my question), and lo-and-behold, I was able to restore my files. It kind of makes sense, but it also seems amazing to me.

So yes, the files are both different and interchangeable. As long as you have the required chunks, they can come from anywhere.

Thank you for your help!

1 Like
#6

It’s math :slight_smile: This is not a bad explanation: https://moxso.com/blog/glossary/nonce

#7

I even verified you can mix-and-match chunks when my two repos have different passwords… no real use case for that (for me), but interesting nonetheless, and confirms what I thought I knew about the encryption.

#8

No, that’s different: The data is encrypted with keys that are randomly generated when the storage is first initialized, and saved in the config file.

The config file itself is then encrypted with the password you provide (with -e option).

When you create bit-identical storage, those same keys are copied to the config file of the storage being created, so the its data is still encrypted with the same keys.

A small tangent: When you create “copy-compatible” storage, as opposed to “bit identical” then only the chunk generation parameters are copied but not encryption keys. This makes it impossible to just copy chunk files around between these storages. duplciacy copy in this case decrypts chunks and re-encrypts them with the target storage credentials.

#9

In my testing, I was setting -bit-identical and -copy, which is probably why I got the behavior I observed, but I was using different passwords with the -e option, so it all tracks.

1 Like