Compression?

Does duplicacy have a compression option?

You can set the compression level when initializing the storage:

SYNOPSIS:
   duplicacy init - Initialize the storage if necessary and the current directory as the repository

USAGE:
   duplicacy init [command options] <snapshot id> <storage url>

OPTIONS:
   -encrypt, -e                    encrypt the storage with a password
   -chunk-size, -c 4M              the average size of chunks
   -max-chunk-size, -max 16M       the maximum size of chunks (defaults to chunk-size * 4)
   -min-chunk-size, -min 1M        the minimum size of chunks (defaults to chunk-size / 4)
   -compression-level, -l <level>  compression level (defaults to -1)
The compression level parameter is passed to the zlib library. Valid values are -1 through 9, with 0 meaning no compression, 9 best compression (slowest), and -1 being the default value (equivalent to level 6).

and how about if one uses the Windows GUI? I didn’t see any compression options there.

There is no compression option in the GUI version. You can initialize the storage with the CLI version first, and then when you run the GUI version, once you select the repository directory it will load all settings correctly.

when I try using the -l9 for example I get “Incorrect usage”

What if you try with -l 9 ? (observe the space)

Thanks for the reply…

I tried the space with the same results.

With the latest CLI release Releases · gilbertchen/duplicacy · GitHub – 2.1.2 the init command doesn’t have any compression options.

Here are the options:

D:\vm>z init fdas
The init command requires 2 arguments.

NAME:
   duplicacy init - Initialize the storage if necessary and the current directory as the repository

USAGE:
   duplicacy init [command options] <snapshot id> <storage url>

OPTIONS:
   -encrypt, -e                         encrypt the storage with a password
   -chunk-size, -c <size>               the average size of chunks (defaults to 4M)
   -max-chunk-size, -max <size>         the maximum size of chunks (defaults to chunk-size*4)
   -min-chunk-size, -min <size>         the minimum size of chunks (defaults to chunk-size/4)
   -iterations <i>                      the number of iterations used in storage key derivation (default is 16384)
   -pref-dir <path>                     alternate location for the .duplicacy directory (absolute or relative to current directory)
   -storage-name <name>                 assign a name to the storage
   -repository <path>                   initialize a new repository at the specified path rather than the current working directory


D:\vm>

I think it’s the case that compression support was removed: now only a default compression exists and the user cannot change it.

@TheBestPessimist I think that it’s still possible change compression. If I changed value config file (before first backup)
from
“compression-level”: 100,
to
“compression-level”: 0,

I tested it on duplicacy_win_x64_2.1.2 with text file. “chunks” folder have after backup same size as source text file. Co compression was not applied. My lead was this source code

So missing option in init is maybe just a bug?

1 Like

This is not a bug. It is for backward compatibility. At the beginning Duplicacy used zlib for compression and SHA256 for hash, but later I found out LZ4 and BLAKE have better performance. So a new compression level 100 was introduced to switch to LZ4 and BLAKE which became not only the default but also the only choice.

It should be fairly easy to add back zlib by assigning new compression levels (for example 200-209) to it. There is no reason to switch the hash algorithm to SHA256 instead BLAKE I think.

3 Likes

Using the Mac GUI version the default compression is applied?

The default compression is well… default. It is always applied.
There’s no point is turning it off since it is quite very fast and doesn’t use many resources!

Seems like this thread is out of date now.

Didn’t start a new one since I have exactly the same question. What compression does duplicacy use?

Also, is there a way to see how much is saved via compression. Would be nice to see that in the -tabular option.

… but later I found out LZ4 and BLAKE have better performance …

^this was the key bit; LZ4 compression is used, though not sure how BLAKE fits into the whole picture

in any case, there is some good-enough compression performed by default (ie always) which cannot be tuned. expect storage savings. even if it could be better, there’s nothing to be done here atm (thread tl;dr)

Also, is there a way to see how much is saved via compression. Would be nice to see that in the -tabular option.

+1, would be nice to see more real data on all the value Duplicacy provides

Would be neat to not readd zlib but zstd instead. :slight_smile:

Do I understand correctly that by default there is compression enabled?

Yes…