Persist command ignored

Please describe what you are doing to trigger the bug:
Rerstoring a backup with long filenames and the “-persist” option

Please describe what you expect to happen (but doesn’t):
restore continues

Please describe what actually happens (the wrong behaviour):
restore is canceled

./duplicacy restore -r 687 -threads 5 -overwrite -stats -persist
Failed to create the file for in-place writing

How long are the long filenames (with and without directory names)?

What type of filesystem are the files being restored to?

I figured out that encrypted folders in synology only support 143 characters per file name (not path).Filesystem ist btrfs

Based on the info so far, there seems to be a couple of unrelated issues that make it appear as though there’s a problem with the -persist option in Duplicacy.

The -persist option is behaving as it should (skipping bad chunks and optionally stopping on existing files unless the -overwrite option is also specified, then it really only ignores bad chunks).

Because Duplicacy generates the filename for a chunk by hashing it contents, the filename is always 63 characters (which is equal to 63 bytes since it only contains the ASCII set).

And because the original filenames are stored in the snapshot log, they aren’t affected by the storage filesystem. Unless the backup chunks are being transferred to a FAT-formatted filesystem (which only supports up to 9 character ASCII filenames), their 63-character filenames are also no problem for pretty much every filesystem created during the past 40 years or so.

Right now the only scenario I can see where Duplicacy aborts with a “Failed to create the file for in-place writing” error during a restore is if the destination is an encrypted directory on the Synology (which uses eCryptfs).

Its not about the chunks, its about the files restored from a backup.
Synology has a limitation foe encrypted folders:

So the worakroud would be to provide restore filter to exclude files longer than 143 symbols: something like e:^(.*\/)?[^\/]{144,}$ (easy to tweak to also skip long directory names if needed)

Wether -persist shall ignore fatal filesystem errors is debatable.

1 Like

Yes, I know :wink:, but because someone might try to use Duplicacy to upload chunks to an eCryptfs volume on a Synology, it seemed relevant to mention it.

As for restoring files, I agree with @saspus that whether Duplicacy should ignore filesystem errors is up for debate. I think if Duplicacy follows the typical behavior of other user mode apps when failing to open a file descriptor, then it definitely should abort (perhaps after a few attempts). Since the documented purpose of -persist doesn’t include ignoring OS errors, is it a bug in Duplicacy or a limitation of eCryptfs?

1 Like

But then you don’t know which flies are missing in the restore

You can do duplicacy list -files | egrep <that regex> beforehand to know which ones will be skipped.

On a separate note, encryption on Synology is a snake oil, I would not use it, and as a result, not have to deal with this noncence. My file names are humongous :slight_smile: