Remind me.. why did i not use Duplicacy on Linux?

Support
#1

OK, strange question but i’ve hunted through my previous posts and the reason/s i didn’t go with Duplicacy eludes me.

I remember i had already invested in, and was happy with Borg, so that’s one reason. But there was something that is itching my brain - something about files?

I have a (perhaps false) recollection that Duplicacy drops .duplicacy files (or something) all over the filesystem. Or something. And that irked me. But i’ve installed d-web on a VM and i don’t see such files.

Was i dreaming?

I’m now looking at using Duplicacy on my unRAID box to backup to the cloud (i already use d-web on 3xWindows machines for the family).

#2

Duplicacy CLI creates a hidden .duplicacy folder in the repo root to keep configuration, logs, and temp files;

Later the configuration option was added (-repository, iirc) that allows to run it in one place and backup another. WebUI used this approach and as a result can be made to not add stuff to the repo root and generally keep logs, caches, and config is separate places. (Which should be setup correctly by default but isn’t. At least it’s possible now).

Other than that — why knows why did you go with something else? There is no shortage of shortfalls and no shortage of great features. If you can tolerate the specific set of the former and need the latter — you will be a happy user. otherwise — not :slight_smile:

FWIF for me it was simplicity of design (single level CAS and lack of central database), performance (regex engine is phenomenal — thanks whoever wrote that go library), and resilience to failures — it was the only tool that surviving my barbaric testing. That’s what mattered to me then.

#3

Thanks saspus. When i did two separate backup sets (downloads; and desktop). I only see ~/.duplicacy-web. Is that the directory you’re referring to?

I think i might have misinterpreted something, or maybe the default behaviour was different nearly 3 years ago.

I like Duplicacy. Pretty sure my use of it will expand.

#4

Hi, could you elaborate this point?

#5

Modern OSes designate special locations to place application data, caches, and logs.

For example on macOS application data goes to Library/Application Data, logs go to Library/Logs and caches to Library/Caches. It’s a standard and expected system wide convention that other apps rely on. For example, Time Machine skips Logs and Caches folders.

Furthermore, transient data on macOS must be marked with a special extended attribute to prevent time machine from picking it up.

Today default installation of duplicacy-web does none of that: it dumps everything into a single hidden directory and does not mark anything for exclusion.

This, among other things, results in data loss: when turbulent stuff that does not need backed up gets backed up other, important data stays unprotected longer and due to limited space on the target drive results in version history shortening.

This is at least one easy to understand tangible reason for following OS guidelines. There are many more ways how duplicacy_web behaves and feels foreign on that OS.

Similar deal on Windows.

Can’t speak for other OSes.

With duplicacy_web users can fix those issues manually in settings and command line.
But for that users need to:

  • understand their OS design.
  • understand duplicacy design.
  • know to do that in the first place.

Most therefore won’t do it. And even if 100% of them did — asking thousands of users to do work that can be fixed once in the app is of questionable utility.

2 Likes
#6

Well than i suppose this is another valid reason to run Duplicacy in a container :grinning:

#7

It’s not. You still need to map out the container volumes manually, and you have to know even more: not only about OS but also docker or podman or whathaveyou; might as well configure that for duplicacy. Furthermore, golang apps rarely if ever benefit from containerization. The main appeal is dependencies isolation but go programs have none. They are self contained monolithic executables. Adding entire new user space environment there is pure waste. And on some OSes this is not even a lightweight user more stuff — it’s actually a VM under the hood… too much overhead for absolutely nothing in return.

The solution is not yet another layer of complexity but trivial fixes in the app to address these shortcomings. For duplicacy CLI anyone can submit PR but for GUI it’s up to the author.

1 Like
#8

I think the norm for Linux user-level programs is to put the executable in ~/.local/bin and configs in ~/.config. For system-wide, /usr/bin. But i don’t know for sure.

To illustrate your point - I didn’t know that! After running some test backups, i changed that setting. It didn’t move all the files, or remove the old path. I renamed the old path and rebooted, to see what it would do. When i ran the executable again (from the same path as before, because i had manually copied it to ~/.local/bin originally) it re-created ~/duplicacy-web - my changed settings and path were not present and it wanted new credentials like it was the first time used :frowning:

#9

With some searching I found this: XDG Base Directory Specification and lsb:fhs [Wiki] but I’m not sure if there are newer/better/custom conventions per distribution, etc. What I know for sure is polluting user home with temp data is a no go.

#10

What is the temporary data you’re referring to? I was thinking configurations and the executable.

Would /tmp be the better place for such data? Or should it remain post reboot?

#11

For duplicacy CLI it’s .duplicacy/logs, .duplicacy/caches.

For duplicacy_web it’s the whole .duplicacy_web/repositories. Those contain subfolders with temp repositories created for duplicacy CLI to feel at home, along with their own logs and caches; e.g ~/.duplicacy_web/repositories/0/.duplicacy/caches

They shall not be in tmp becuase it’s desirable for them to survive the reboot for performance optimization, but they are not critical and can be deleted any time. It’s the definition of cache data. On MacOS it shall be in ~/Library/Caches, on linux – perhaps somewhere under /var?

On the other hand I’ve seen some games on windows place 2GB of GPU shader caches into user’s Documents folder (which is synced to OneDrive). So I’m not surprised anymore. People tend to not care. But I do.

2 Likes