Include filters help please. WebUI

Support
#1

Hi guys,

Filters are a common point of confusion it seems :slight_smile: i’ve read a few posts but it hasn’t yet helped me with this question.

I’ve set the backup ‘Directory’ to C:\ (or C:/ as it’s displayed), but i only want to include about a dozen directories, so i’ve set these to ‘include’ using the UI only, i didn’t manually enter any filters e.g.
c:\PortableApps
c:\users\humanbean\Desktop
c:\users\humanbean\Documents
c:\users\humanbean\AppData
etc.

I’ve done a -dry-run and only PortableApps was logged. It doesn’t appear to have read the humanbean directories.

I gather i can probably create 2 different backup sets:
One for PortableApps
One for humanbean

But that’s more work. So can i do it the way i planned to and if so, what did i get wrong? p.s. it is installed as a service per the installation guide.

Thanks.

#2

You will also need to include the c:\users and c:\users\humanbean directories. If you don’t want to include their other subdirectories, exclude them with a wildcard after the respective includes

#3

Thanks. I decided to go with the second option of a couple of different backup sets instead of using excludes. I find that Windows apps tend to put stuff all over the place and soon enough i’ll have a bunch of backups i don’t want because i haven’t specifically excluded them. This works perfectly well, just a tiny bit more labour intensive. I’m unfit thanks to Covid - so now i’m sweating :slight_smile:

#4

OK, that didn’t quite work as i’d hoped. It didn’t want to include my AppData\Local\Mozilla directory. Using these didn’t get that path, all the others worked ok. The base is c:\Users\humanbean. Then i include these:

+AppData/Local/Mozilla/*
+Desktop/*
+Documents/*
+Downloads/*
+Music/*
+Pictures/*
+Videos/*

I’ve read stuff about these filters but i just haven’t managed to understand them :frowning: Is it possible to include the Mozilla directory there?

#5

You would need to include all folders up to the parent one. Like so:

+AppData/
+AppData/Local/
+AppData/Local/Mozilla/*
+Desktop/*
...

The thing ending with / matches folder, not files.

The way to understand it: rules are applies one by one while traversing the directory tree. So if there is no match to include AppData/ duplicacy would not bother looking inside. In the same way, if AppData/Local/ is not included it won’t look inside to check for Mozilla.

This requirement for the human to have to cater to the peculiarities of the parser is annoying; it has been rightfully brought up before multiple times; and luckily is on a roadmap to get addressed: Adding folders to backup set using UI still requires knowledge of filters behavior

It’s especially bad when you click in the UI to include the folder and … nothing happens. UI should know better to make what user wanted happen.

Edit. In your original question:

you would do

+PortableApps/*
+users/
+users/humanbean/
+users/humanbean/Desktop/*
+users/humanbean/Documents/*
+users/humanbean/AppData/*

But take care to drill down further and exclude the bulk of things from AppData – most of it is transient and temp data.

3 Likes
Backups failing "No files under the repository to be backed up" in directory with files
#6

Hmmm…

So if i had multiple users, could i save writing each of the full paths, once i have their parents, and have something like this? Assuming C:\Users is the base directory:

+humanbean\
+bloodbottler\
+Public\
+*\AppData\
+*\AppData\Local\
+*\AppData\Local\Mozilla\*
+*\Desktop\*
+*\Documents\*
+*\Downloads\*
+*\Music\*
+*\Pictures\*
+*\Videos\*
#7

Yes, you can.

(not sure about / vs \ though – I think even though windows uses \ Duplicacy expects /. But I may be wrong – haven’t used windows for ages now)

However note that * matches everything, including path separator. so it may add affect more stuff than you think. (e.g. c:\Users\test\hello\AppData\Local\some_unneeded_stuff\ will match, because \AppData\Local\ is in the middle while star will consume \Users\test\hello\

At this point I would use regular expressions instead. (with e: and i:) – there you can specify exactly what you want matched.

Edit. Just for kicks, here is the single regex that will match everything you listed above:

i:(?i)^[^/]*/((AppData(/Local(/Mozilla(.*)?)?)?|Documents|Downloads|Pictures|Videos|Public|bloodbottler|humanbean)/.*)?

The way it works:

  • i: – include
  • (?i) – case insensitive
  • ^ – match start of path
  • [^/]* – match any number (star) of anything but (the hat) path separators (slash)-- ie. your user home in this case
  • / followed by path separator
  • (A|B|C) – match A or B or C
  • ? means optional (repeats 0 or 1 times)
  • . matches any symbol
  • * repeats 0 or more times
1 Like
#8

Oh i think Duplicacy ‘fixes’ it automagically. :slight_smile:

Yeah, i’m aware of that tradeoff.

God, they’re even more complicated :smiley:. Let’s give this a go. btw i’ve just found out the Firefox and Thunderbird profiles are in roaming, not local. Local is cache.

So this time going all the way down to C:\ as the base directory…

i:(?i)Users/[^/]AppData/Roaming+/Mozilla/
i:(?i)Users/[^/]AppData/Roaming+/Thunderbird/
i:(?i)Users/[^/]/Desktop/
i:(?i)Users/[^/]/Downloads/
i:(?i)PortableApps/

OMG that literally gave me a headache. :pleading_face: And it’s probably wrong.

Can i mix the filters? use both i/e and +/- depending on the directory? It might just be easier for me to do it the longer way and list every user with every one of their sub-dirs i’d like to backup.

#9

Cheeky bugger, you just added that didn’t you :stuck_out_tongue:

That’s a pretty damned efficient way of doing it. Thank you. I’ll try it out and let you know. It’s getting late now, so i’ll report back tomorrow.

#10

While you were getting a headache I was getting one too. Updated my comment above :slight_smile:

Yes, you can mix-and-match in any order – meaning regardless of what filters are they will be processed one by one from the top.

#11

Honestly I would split it into separate simpler regexes for actual use – it would be maybe less efficient in the runtime but will be much easier to maintain. And who cares about runtime – duplicacy is ridiculously fast anyway. Otherwise it becomes write-only unmanageable monstrosity very quickly

#12

You said you went with your second approach - to not use excludes, two different backup sets. So with individual base folders, why do you even need a filters file?

Otherwise, if you want an ‘include-only-these’ filter, I believe you have to add an exclude for everything else - i.e. -* at the end.

#13

Yeah, i decided to do that after i took another look at your crazy line :slight_smile: I needed to add another 2 users, and i didn’t know how and decided that it would just be easier to do it the verbose way. Which worked btw - so thank you for the explanation up the top somewhere which i’ll mark as the solution. Once you know, it’s not complex - just a little lengthy :slight_smile:

You’re right on this. If you select it, Duplicacy first shows \, then when you save and go back it, it’s changed to /. I assumed it would work with manually inputted, but it didn’t seem to - so / is the way to go.

#14

For me the best approach was using @saspus’s (marked) solution. More verbose but better for me on that PC.