As I mentioned in my earlier reply, this does not appear to be effective in getting the contents to be included - i.e. I still see the excluded message for the symlinks.
This means test1 is a file. Is it a symlink? The current implementation will apply the patterns before converting symlinks to directories, so this may be the issue.
Yes, test1 is a symlink to another dir.
For grins and giggles, as a test, - WITHOUT making any changes to filters as described above - I went ahead and mounted the dir that the test1 symbolic link refers to as follows:
mount --bind /path/to/test1 test1
Not too surprisingly, the directory is then included:
test1/ is included by pattern +test1/
It would appear that I am either doing something incorrectly with the include filters for symlinks or that there is a problem with them? Hard mounts like this are discouraged and quite problematic, so this is really not a good solution, but I just thought I would pass this info along.
@gchen, does this help to confirm that this is a bug? Is there anything else I can try with the symlink approach not already mentioned above?
@gchen unfortunately, as mentioned above, this does NOT work - even when adding in the +test1 pattern.
This was from my test and it worked as expected.
Loaded 3 include/exclude pattern(s)
Pattern: +test
Pattern: +test/*
Pattern: -*
Listing
duplicacy_win.exe is excluded by pattern -*
duplicacy_win_x64_2.1.1.exe is excluded by pattern -*
file1 is excluded by pattern -*
test is included by pattern +test
Listing test/
test/file2 is included by pattern +test/*
Can you post the output from duplicacy -d backup -dry-run if you think it still doesn’t work?
@gchen, just to put this all in one post, my filters file includes:
+test1 # This should not be needed, but is to follow symlink
+test1/
+test1/Qmultimedia/
+test1/Qmultimedia/*
+test2 # This should not be needed, but is to follow symlink
+test2/
-test2/home/*/tmp/
-test2/opt/google/
+test2/*
-*
Yet, as requested, the output from the -d backup command (either with or without the -dry-run option) reports the following:
...
test1 is excluded by pattern -*
...
test2 is excluded by pattern -*
...
Even more telling, is the fact that I now see (I didn’t see this in previous run with --bind version) a large number of lines in the output similar to the following:
Delete chunk 0082d8c5ca25330221ad4dc148da74bfd60446ca2cc952ab7459f4b1034d65cd from the snapshot cache
Delete chunk 00ee77e0de606dd6c8f0812b5a464f95ca9273e190a543ec9dadb4fb03f51db2 from the snapshot cache
This backup run against the symlinks has removed the dirs/files that had been included by the immediately preceding mount --bind backup run that had been completed just before the symlink version (as there were no ‘organic’ or other deletes that could otherwise explain this). This is further confirmed by checking the history of one such file with the following command:
duplicacy -v history -r 22 -r $last "test1/Qmultimedia/Photos/2018/Spring/20180516_084431.jpg"
and the associated output:
Using 16384 iterations for key derivation
22: 2051696 2018-05-16 08:44:31 ce1a37c6188699daa9fa9f42d9a97652d3d0fbaf339a1d6c23ea4fa9e8579ed4 test1/Qmultimedia/Photos/2018/Spring/20180516_084431.jpg
23:
current: 2051696 2018-05-16 08:44:31 test1/Qmultimedia/Photos/2018/Spring/20180516_084431.jpg
The links are as follows:
>:~$ ls -ld test1 test2
lrwxrwxrwx 1 gouldt gouldt 23 Aug 19 17:49 test1 -> /media/gouldt/test1
lrwxrwxrwx 1 gouldt gouldt 22 Aug 19 17:50 test2 -> /media/gouldt/test2
This is all with the very latest 2.1.1 version on an Ubuntu 16.04 LTS system.
Please let me know what else I can do to help you.
+test1 # This should not be needed, but is to follow symlink
If this is the exact line in your filter file, then I can see where the issue is: the whole line including the comment is interpreted as the pattern. A # will be treated as a comment only when it starts the line; otherwise it will be part of the pattern.
Other than that this I can’t think of any other reason why it won’t work for you.
2 Likes
@gchen, I understand what you are saying, but this does not seem to line up with my actual results. Why would it work for the --bind version but not the symlinks (same filter file used for both)?
I am wondering if the fact that the symbolic links are to mounted directories (one is Windows share via cifs and the others is an NFS mount) may be effecting this. All dirs, subdirs and files under them behave as expected via the symbolic links, so I am hoping not. However, I am attempting to discover why duplicacy is not able to deal with them appropriately in my environment?
Did you try removing the comment in that line? Why not give it a try? I would be very surprised if it doesn’t work.
The reason the --bind version worked was that in that case test1 became a directory and was therefore excluded by the pattern +test1/.
.@gchen nailed it, of course. I never would have thought that inline comments were not supported/allowable. Thanks! I suppose the only remaining issues to consider are:
- documenting the unexpected behavior of comments (looks like @TheBestPessimist is already on this)
- determining why (and perhaps removing the need for) a redundant +test1 entry is required fro symbolic links
1 Like
I think the redundancy is because on some systems, a symbolic link to a folder, is actually seen as a file?