Restore: weird inclusions and how to exclude

Feature
#1

I want to verify that duplicacy can restore things if I lose a boot drive due to a crash or whatever. On a test machine, I backed up my / drive (Mac), excluding the common system paths. I wiped the drive and restored the OS first (El Capitan, it’s an old MacBook pro).

I’m having two problems restoring.

1. Duplicacy seems to be trying to restore things that shouldn’t have been backed up, and aren’t listed when I do a ‘duplicacy list -files’. For example:

storage set to sftp://xxx@yyy/x/y/z
Skipped non-regular file dev/afsc_type5
Skipped non-regular file dev/auditsessions
...

However, I excluded dev/* from the backup.
When I search the files listing from “duplicacy list -files” there is no “auditsessions” or “afsc_type5” listed.

So the question is: Why is duplicacy apparently trying to restore the dev/ files, even though they weren’t backed up?

2. I could not figure out how to use the exclusion patterns on the command line for restore. I finally figured it out, so I have turned this question into an example for future reference:

e.g. I backed up the Applications folder. However, upon restore, it is trying to restore some of the system applications, and due to System Integrity Protection, that is not allowed. So I am trying to exclude them.

For example, upon first try I got the error:
Restoring / to revision 2
mkdir /Applications/Safari.app/Contents/PlugIns: operation not permitted

I tried many different ways of doing exclusion. I finally figured it out using shell escapes (i.e. ) to specify the exclusions, like this:

sudo duplicacy restore -stats -r2 -- \-*Safari\*

i.e. escaping both the leading ‘-’ and the two ‘*’ so they don’t get interpreted by the shell first

Second problem down, one more to go:

3. Only restore non-existent files.

This is what I thought duplicacy was supposed to do if -overwrite was not specified.

However, that is not what it does.

It chokes at the first file that it finds already exists, e.g.

Restoring / to revision 2
Downloaded chunk 1 size 8393236, 8.00MB/s 08:33:45 0.0%    
File installer.failurerequests already exists. Please specify the -overwrite option to continue.

Then the restore process quits at that point.

How can I force duplicacy to do a restore of all files that don’t already exist on the restore destination?

While I can do a workaround of restoring to a separate directory, it’s going to be a PITA to then move everything into place by hand. If this were a real system crash, it would be very slow and inconvenient to do so.

Thanks
Morgan

#2

Why is duplicacy apparently trying to restore the dev/ files, even though they weren’t backed up?

It is not trying to restore the dev/ files; it is just listing these files. Sure, it shouldn’t list these files if they are not included in the backup, but currently it is not smart enough to know it.

How can I force duplicacy to do a restore of all files that don’t already exist on the restore destination?

Currently there is no option for this. The concern is that if it only restores files that don’t exist, then the restored repository may be in an inconsistent state, as files that already exist may be different from those in the backup.

#3

Hi, okay the logic makes sense for why you wouldn’t restore only flies that aren’t present, but can I request this as a feature? i.e. It seems like a common scenario where someone may have a system restore where they are adding in the restored personal files like I’m doing here. In that scenario, the consistency isn’t a concern. We just want to “add in” all the customizations and personal files that aren’t there on the native system.

Meanwhile, I will keep playing with the “filters” file upon backup to try to get it only backing up unique stuff that is not included with the system.

#4

Can you submit a git issue for that feature request on https://github.com/gilbertchen/duplicacy so I won’t forget it later?

#5

Will do - thanks.

#6

What’s the status of this request?