Exclude rules help

Support
#1

I am trying to get my first backup happening but this exclude/include rule is confusing me.

As an example, I want to back up my Docker config in /data/docker/ and there is a bunch of build stuff under /data/docker/esphome/xx/xx/xx/xx that I do not want included. Am I right in thinking that to exclude everything but the .yaml config files in /data/docker/esphome that I have to explicitly include both /data/docker/esphome/config/ and /data/docker/esphome/ as below:

-*._*
-._.DS_Store
-*.DS_Store
-*.log
-*.iso
+esphome/config/*.yaml
+esphome/config/
+esphome/
-esphome/config/*

I have done some -dry-run tests and think I am still getting all of the other stuff I want but just want to make sure I am understanding these exclude rules properly.

#2

You’re almost there…

+esphome/config/*.yaml

Assuming that /data/docker is the root of your backup, the rule above says “include any files under the directory path /data/docker/esphome/config that have a .yaml extension”…

But for the file include rule to work, each directory leading up to the matching files must also be explicitly included (it’s often easier to follow if it’s written in reverse order)…

+esphome/config/
+esphome/

However, the last rule you added to your filter – a file exclude – doesn’t limit the scope of the directory include rule immediately above it enough (e.g., /data/docker/esphome/junk.txt would be included in backups):

-esphome/config/*

It’s because “+esphome/” is saying “include the directory ‘esphome’ and its contents”, which includes any files and subdirectories.

Duplicacy needs to know to “ignore everything else under the ‘esphome’ directory” that wasn’t already explicitly included:

+esphome/config/*.yaml
+esphome/config/
+esphome/
-esphome/*

Note that the filter above as written only captures .yaml files under /data/docker/esphome/config, so any additional subdirectories with .yaml files would be ignored, e.g., “me.yaml”:

data
    └── docker
        └── esphome
            └── config
                └── 1.yaml
                └── ignore
                    └── me.yaml
#3

OK - I think I am understanding!

So to capture sub-directory .yaml files I would need:

+esphome/config/*/*.yaml

If this is correct how do I then included the directories that get me to that sub-directory?

+esphome/config/*/*

???

#4

No, the pattern above won’t work with the simple include/exclude syntax because an asterisk (*) matches any character, including the path separator.

In other words, this…

+esphome/config/*/*.yaml

… is equivalent to this:

+esphome/config/*.yaml

Likewise, this…

+esphome/config/*/*

… is equivalent to this:

+esphome/config/*

If I understand correctly, you want to grab any .yaml file under the path /data/docker/esphome regardless of how deep the subdirectories go, but at the same time ignore all other files and subdirectories that aren’t a match or contain at least one .yaml file?

#5

That’s what I am trying to understand how to do!

#6

The * matches everything, including path separator

So, why not just

+*/
+*.yaml

The first line will match all directories and the second line - yaml files. You need the fist one for the parser to visit every directory, otherwise it won’t be matching files there. And the second obviously will match on the file once it is visited.

I haven’t tried it but it seems that this shall work. You can add prefix before the * as needed too.

For example

+data/
+data/docker/
+data/docker/esphome/
+data/docker/esphome/*/
+data/docker/esphome/*.yaml

Same logic

#7

Expanding upon @saspus’ recommendations…

If any YAML filenames are in uppercase, then use the regular expression matching syntax:

i:.+/$
i:(?i)\.yaml$

For both filter pattern versions, the reason why they work and a very important aspect to keep in mind from the Filters/Include exclude patterns page:

If a match is not found, the path will be excluded if all patterns are include patterns, but included otherwise.

In other words, no mixing includes and excludes (e.g., -*.iso) – it’s either all inclusions or nothing.

#8

You can mix includes and excludes. That rule only talks about default behavior if nothing matched: it will be “exclude” of all patterns are “include” and “include” otherwise.

For example, to only include two sub folders you would combine include and exclude:

#### ---------------------------- Pictures ----------------------------  >>>>
+Pictures/

# Only pictures and originals from Photo Booth 
+Pictures/Photo Booth Library/
+Pictures/Photo Booth Library/Originals/*
+Pictures/Photo Booth Library/Pictures/*
-Pictures/Photo Booth Library/*

This will include Originals, Pictures, and everything else from ~, except other files and folders under photo booth library.

You can of course Invert the behavior anyway by adding -* in the end.

#9

Thanks for the replies, I think I understand now.

#10

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.