Patterns for exclusion/inclusion are confusing

[marekhwd]

Filters/Include exclude patterns

When matching a path against a list of patterns, the path is compared with the part after “+” or “-”, one pattern at a time. Therefore, the order of the patterns is significant. If a match with an include pattern is found, the path is said to be included without further comparisons. If a match with an exclude pattern is found, the path is said to be excluded without further comparison.

…

Patterns ending with “*” and “?”, however, apply to both directories and files. When a directory is excluded, all files and subdirectories under it will also be excluded. Therefore, to include a subdirectory, all parent directories must be explicitly included.
For instance, the following pattern list doesn’t do what is intended, since the foo directory will be excluded so the foo/bar will never be visited:

+foo/bar/*
-*

The correct way is to include foo as well:

+foo/bar/*
+foo/
-*

This is confusing. According to the first paragraph foo/bar/baz.txt will be matched with first include pattern +foo/bar/* and therefore shouldn’t be excluded by -*. Is it a mistake in documentation or am I missing something?

[Christoph]

foo/bar/baz.txt will be matched with first include pattern +foo/bar/* and therefore shouldn’t be excluded by -*

It’s not excluded by -*. If it was matched with a previous inclusion pattern it is included. Full stop. No further processing occurs. That’s why inclusions have to be listed first in order to make sure that they are not “overwritten” by some exclusion (and vice versa, of course: exclusions that should not be trumped by inclusions should come first):

Filters/Include exclude patterns

the order of the patterns is significant. If a match with an include pattern is found, the path is said to be included without further comparisons. If a match with an exclude pattern is found, the path is said to be excluded without further comparison.

It takes a while to wrap your head around this. Check out the topics tagged #filters for more discussions around this.

That said: the documentation may well be improved. Once you figured out what information might have helped you to grasp things faster, feel free to edit it with clarifications (it’s a wiki).

Check out this one, in particular:

Improving the instructions for include/exclude patterns (filters)

I think the instructions for the include/exclude patterns could be improved, but I am myself too unsure of things to make any edits in the wiki. So I’m starting this thread to clarify some things first and once I (or anyone else) has a clear view of things, I/they can edit the wiki. So, let’s start with the first example under “1. Wildcard matching”: +foo/bar/* -* It is introduced by saying the following pattern list doesn’t do what is intended, since the foo directory will be excluded so t…

[marekhwd]

It’s not excluded by -*. If it was matched with a previous inclusion pattern it is included. Full stop. No further processing occurs. That’s why inclusions have to be listed first in order to make sure that they are not “overwritten” by some exclusion (and vice versa, of course: exclusions that should not be trumped by inclusions should come first):

That’s what I understand too from the first paragraph I cited (which you also cited just below your paragraph I cited). Then just to make it clear, the second paragraph I cited in my previous post is false, is that correct? I’ll cite it again:

Filters/Include exclude patterns

When a directory is excluded, all files and subdirectories under it will also be excluded. Therefore, to include a subdirectory, all parent directories must be explicitly included.
For instance, the following pattern list doesn’t do what is intended, since the foo directory will be excluded so the foo/bar will never be visited:

+foo/bar/*
-*

The correct way is to include foo as well:

+foo/bar/*
+foo/
-*

This is not true, right?

[marekhwd]

OK, this comment from the post you linked explains why it IS true:

Improving the instructions for include/exclude patterns (filters)

Assuming foo/bar/file.txt is the only file you want to back up, the correct patterns are: +foo/bar/file.txt +foo/bar/ +foo/ -* Duplicacy lists the root of the repository. The foo/ folder matches the third pattern, while all others will match the fourth one and thus get excluded. In the next step Duplicacy lists the foo/ folder and only foo/bar/ will be included. Finally, Duplicacy lists foo/bar/ and finds foo/bar/file.txt which is the only file with a matching include pattern while all othe…

[noah.e.miller]

In other words, duplicacy starts by visiting the top-level directories in your repo, and runs through your include/exclude patterns on them. So, first, duplicacy will decide whether to include foo:

• Does the rule +foo/bar/* match foo/? No. Try the next rule.
• Does the rule -* match foo/? Yes. Exclude foo.

Because foo is excluded, foo/bar will never be visited, therefore foo/bar/baz.txt will never be considered.