Follow symlinks in subfolders?

Support
#1

Is there a way to get duplicacy to follow Windows symlinks (instead of backing them up as symlinks) in subfolders? Or at least tell it how “deep” it can go?

I’m setting up backups for multiple locations on various drives, and don’t want to have to maintain multiple repositories and backup jobs. So as per other suggestions in the forum and documentation, I created a main “Backups” folder, and will be setting up symlinks to the various folders I actually want backed up.

However these are in different places and in different drives, and I would like a structure that looks like:

\Backups
    \C
        \Folder1       (symlink to C:\Folder1)
        \Folder2       (symlink to C:\Folder2)
    \D
        \Folder1       (symlink to D:\Folder1)
        \Folder2       (symlink to D:\Folder2)
(etc)

Instead, what I think I have to end up doing is:

\Backups
    \C_Folder1       (symlink to C:\Folder1)
    \C_Folder2       (symlink to C:\Folder2)
    \D_Folder1       (symlink to D:\Folder1)
    \D_Folder2       (symlink to D:\Folder2)

It would be nice if it were only 4-5 folders in total, in which case keeping things in a flat structure wouldn’t be much of an issue, but in actuality it’s closer to 30-40, from 4 different drives, and likely to grow to double that size in the next year or so!

Any thoughts or workarounds would be appreciated!

Thanks,
-Alex

Why does this filter block any backup, no matter what?
#2

I don’t really understand what you are trying to do, but duplicacy follows only the first symlink in a path and backs up subsequent ones as symlinks.

#3

Not wanting to take this off topic, but: that seems a strange rule. I can understand the rationale if the symlink is the first element of the path, but what if it’s buried several levels deep? Should it really be treated specially in that case?

#4

I don’t think it is treated differently if it occurs deeper down in the path. What counts is whether it’s the first symlink or not. Right, @gchen?

#5

I’m trying to create a folder structure that loosely reflects what’s being backed up and from which drive, instead of having a “flattened” view of everything. Ideally I’d like symlinks to be 2-3 folders deep, e.g. “C:\Backups\D\Ongoing\Project1”, where “Project1” is a symlink to the actual folder “D:\Ongoing\Project1”.

If there’s a way to tell duplicacy to follow symlinks regardless of where they are (and not back them up as symlinks), that would solve my problem.

#6

What happens is as follows: symlinks are only followed (treated as if they are folders) if they are at the root of the repository (if they are placed in the same folder as .duplicacy\). The rest of the symlinks are treated just as normal files (eg. they remain symlinks but are not followed).

This is a really good rationale because you may have deep down in the folder path symlinks which point to the root of the drive and this means that if duplicacy follows everything, it will go in a continuous loop because of that one symlink.

3 Likes
The snapshot contains an error: The entry X/ appears before the entry Y/
#7

Thanks for clarifying, @TheBestPessimist - that is a sound and easy-to-remember rule.

#8

Should this become a feature request then? I understand the case of getting into an infinite loop, but let the user decide whether to follow all symlinks or not (default can be just top-level symlinks).

Or, even better - the ability to backup folders outside the repository path without using symlinks! :wink:

#9

Nope. That is the point of a repository: to have specific “categories” of data you can easily backup/restore located in different points in your system.

There is work though in the ways to have a repository somewhere (eg. in a specific folder) but the actual .duplicacy/ folder to be located someplace else. This is still just WIP.

I have seen this waaay to many times: the user decides he wants something which he doesn’t understand. Then he gets into trouble, then blames the app.

My question is: if symlinking is not useful for you for every folder, could you try and symlink the roots of your drives, then use the #filters file (with regex maybe for easier management) to select what folders you want?

To reiterate/clarify: i don’t speak for gchen. It is just my rather strong belief that this option is too risky to have and the current means should suffice ( i mean: when i found out that i could use the symlink trick for backups i was like WOOOOW that’s f*kin’ genius!).
You can still make a feature request, of course…

2 Likes
#10

Well, perhaps coming from other software (I’m a CrashPlan refugee, but have also used “enterprisey” software like Veritas etc at work), I’ve gotten used to a single instance/config to backup everything regardless of where it is on a system. I understand the need for multiple repositories if you have different backup strategies or destinations (daily, hourly, retention policies, etc), but I’m pretty sure that’s not the case for most users…

And then helpful people such as yourself tell the user it’s PEBKAC :wink:

Speaking generally, I’m always for more user choice not less… I guess someone using a CLI tool should have enough know-how to understand the various switches provided. Non-technical users can always revert to a GUI which will always (one hopes) be more user-friendly.

You know, I actually didn’t consider that! This may do the trick!

1 Like
#11

I am not sure I agree with this rational, as posted here :-

#12

My folder structures have many symlinks. So Duplicati not allowing to follow symlinks is a some kind of risk. If it does not backup because of a Symlink, it may be unnoticed.
I’m aware Symlinks are some kind of risk for software (e.g., endless symlink following, seen that builtin in Windows XP?7?/ “Anwendungsdaten”) but Duplicacy could put some intelligence into it to not follow traps.

1 Like