Need Help With Filters

Support
#1

I am really trying to understand how filtering works via examples and docs, but it just isn’t behaving in a way that I can make work the way I want.

This is my directory tree

 +-- os
 |    
 +-- pool1
 |  |  
 |  +-- wantdir
 |  +-- dontwantdir
 |  +-- wantdir2
 |    
 +-- pool2

I do not want os or pool2, but do want everything under pool1 except dontwantdir. So in the end I would backup pool1/wantdir and pool1/wantdir2. I thought something like these would work

-os/
-pool2/
-pool1/dontwantdir/

or

-os/
-pool2/
+pool1/
-pool1/dontwantdir/

but these fail to load anything when backing up with -enum-only. I also tried

+pool1/
+pool1/wantdir/
+pool1/wantdir2/
-*

But this also produces an empty set. I know the backup works in general because if I do this,

-os/
-pool2/

I start to get files listed from pool1, but it has directories I don’t want included.

Ack…I am just not understanding how to make this work. (using the web gui if that matters)

#2

Since you’re testing with -enum-only, are you using -d to see exactly why a folder was rejected? I believe this could help you figure it out.

#3

I don’t think can’t use -d in the web version, it doesn’t handle global parameters correctly.

I’ll give that a try and figure out how to test it by hand.

1 Like
#4

If I understood correctly, what you need would be this?

-os/
-pool2/
-pool1/dontwantdir/
+*
#5

Ah, sry, when you mentioned -enum-only I though you used the CLI.

#6

Your first example should work. If you’re doing excludes only and not messing around with include patterns or things like ‘exclude everything except X’, simply listing directories to be excluded should work fine - so long as your repository root is correct relative to those paths.

Are any of those paths symbolic links? If so, you may have include patterns with and without trailing slash.

In your example, wouldn’t it be better to point the repository at pool1?

Also, if you’re testing with -enum-only, switch to the CLI and use the provided binary and repository directories, something like:

cd ~\.duplicacy-web\repositories\localhost\0
..\..\..\bin\duplicacy_linux_x64_2.3.0 -d -log backup -enum-only -stats

The exact location of your .duplicacy-web may be different and you may need to change \0 to a different number representing the real repository. Delve in that structure to find out.

But yea, in the Web Edition, you can’t add -v or -d switches to options and I believe -d is required to see anything useful with -enum-only.

How to test filters in web ui
#7

If I understood correctly, what you need would be this?

I tried that and it produces an empty backup set …though I am starting to suspect there is more at play here than just filters (i.e. web gui shenanigans)…will have to find time to test.

#8

Ah, sry, when you mentioned -enum-only I though you used the CLI.

The gui produces a list of files to be backed up when using -enum-only (i.e. when I have no filter set). I hope that it would work in a similar way to the CLI.

#9

Are any of those paths symbolic links? If so, you may have include patterns with and without trailing slash.
In your example, wouldn’t it be better to point the repository at pool1 ?

No symbolic links and yes, that would make sense to make pool1 root, except the example is simplifying what i’m actually doing. I do things in various other dirs, but I can’t get the simple example working at all so didn’t want to add any other complexity.

switch to the CLI and use the provided binary

Exactly what i’m doing now, thanks. Need to find time to explore this a bit more…currently wading through the env variables needed to make the runs a little easier to do manually.

#10

I checked the filters I’m using and (I’m not sure why, probably due to the tests I did at the beginning and it ended up like this) the common part that all filters use, designed to “include everything else”, is using regex, and not wildcard. Something like:

-os/
-pool2/
-pool1/dontwantdir/
i:.*
1 Like
#11

Ok (sigh), I figured out what the problem is. The UI kind of lures you into thinking that when you choose something like this

+pool1/wantdir

that the directory and all of the contents underneath will be included in the backup. This is not the case if you decide to use a rejection pattern like -* in the end. It turns out the line above says “include pool1/wantdir, but since nothing else is matched, apply the rejection pattern to everything in that directory”

The fact is the UI doesn’t allow you to add the correct syntax needed which is

+pool1/wantdir/*

which will add everything in that directory including files and other sub directories. Well, you could add it manually…but that wasn’t clear initially. That was an important factor in my misunderstanding.

So to solve my problem, I ended up using this and it works the way I want it:

+pool1/
+pool1/wantdir/*
+pool1/wantdir2/*
-*

It includes only the directories I want, recursively walking those directories, and then reject anything else it comes across. I had to enter it by hand.

Thanks everyone for the help.

2 Likes
#12

Thank you for this information! When the command is run, where does the log file save to? It doesn’t appear to be in my …/logs folder

Do you know how the -log parameter works? I assumed “-log backup” means to save the output to a log with the file name backup.txt

../bin/duplicacy_linux_x64_2.3.0 -d -log backup -enum-only -stats

#13

With the CLI, I don’t think logs are automatically generated with the backup command (only for prune I think), though you just need to echo the output to a file by appending > log.txt at the end of the command.

-log merely formats the output of the command into a more readable form, with timestamps at the beginning of each line; your usage seems fine.

2 Likes