Backup with or without "-hash"

[maccra]

I usually run my backups without using the "-hash" option. However every now and again I run a backup with the "-hash" option just to check that everything is backed up.

My backup are running fine without problems and today when I ran my backup without -hash there was no changes and therefore nothing to backup. Right after, I ran the backup with the -hash and to my surprise duplicacy backed up about 1.6 GB of chunks (of a total of 65GB).

Does that mean my normal backups without -hash, do not back all my files.

What does that tell me? Can anyone explain to me what is happening.

Cheers and thanks

[Christoph]

Does this previous topic answer your question?

Should we use -hash with every backup?

With the -hash argument Duplicacy will need to read the entire content for each file, and this can be very slow for a large directory. That is why it is not enabled by default. Without the -hash argument, Duplicacy will only back up files with different timestamps from last backup. This may be a problem if there is some file whose content has been modified but the modified time somehow remains unchanged. Another less obvious problem without the -hash argument is that it may leave ‘holes’ in …

[maccra]

Thanks for the link. I am not sure if it answers my question or even if I have a problem, but I am assuming that the reason for uploading much data by using the -hash is due to the 2nd issue as mentioned in your link. I have about 300 or so small files (50-150Kb) that change nearly every day, so by using the -hash I am filling up the “holes” in the chunks and thus need to upload these new chunks. It makes sense in my scenario. Now I just need to consider if I should run -hash backup more often or less often.

Thank you very much for you asistance.

[TheBestPessimist]

From my own usage (2 windows machine and 1 mac) i can say that -hash was never needed. I’m doing restores (of the latest revision) about once every 3-4 months, and so far all the files in the backup match the files on the drives.

[maccra]

Ok… Thank you for the feedback. Sounds good. I am not aware that I have any software which does not modify the file timestamp so not using -hash should not be a problem. The only software I have ever used which does not modify the time stamp is Veracrypt (or Truecrypt) but there is also a setting to let it change the timestamp. As I currenly am not using Veracrypt then I should not have any problems not using -hash.

Again… Thank you both for the responses and feedback… truly welcome.

[Christoph]

have about 300 or so small files (50-150Kb) that change nearly every day,

As a side note: regardless of the --hash option, duplicacy is currently not performing as good as it could when it comes to small files. See here:

New deduplication algorithm: bundle-or-chunk

I don’t think this has been mentioned on the forum yet, so I’m doing it here because I think this is a fantastic improvement of duplicacy’s already excellent deduplication algorithm, especially because it solves the problem that copying, renaming and moving of files may currently lead to new uploads even though the content of the files has not changed. The above pull request (not yet merged) is the result of this discussion: