Docker is just fine. I really just want a quick overview from time to time if everything is still working as expected without googling commands. Read-only gui would be fine aswell. I also need encryption.
I agree about the points related to the GUI version.
But about this:
just use the CLI version with Healthchecks or equivalent, it works perfectly and you have the best of each tool: the robustness of Duplicacy and the ease of use and look of Healthchecks
2 Likes
Ok I will try cli since it integrates better with backing up databases. I can just execute a pgdump before the backup starts with the same script. Healthcheks looks pretty cool, but it adds another online service.
Why did I even start with the UI version in the first place xD
1 Like
I was just checking out Kopia while searching for an official duplicacy cli container, but the Kopia UI is so much more useable than duplicacy. I can even define scripts prior to executing backups. Any red flags on Kopia?
Looks good but is only really maintained by one guy
1 Like
Duplicacy supports pre- and post- backup scripts.
There is no other way to accomplish this task without external services: if duplicacy fails to start, or lost network, how can it report to you that it failed to backup?
Because it’s reasonable to have a working gui for a tool that you use once in your life to setup and never touch again. It’s unreasonable to expect people to read pages of documentation. And yet, here we are.
In case of duplicacy however the CLI interface is very good, clean, and mostly self-explanatory. Use that.
1 Like
TLDR that project is over engineered monster, with the sole purpose to tickle its developers egos. It suffers from featurecreep (support per file compression algorithm?!), while lacking basic necessities (support for filesystem snapshots), and the UI is written on Electron — just look at resource utilization of a simple two page UI.
Duplicacy is simple, and there lies its strength. I trust duplicacy, but I don’t trust Kopia. In addition, Kopia is still version zero, and they periodically break backwards compatibility, only maintaining one with the previous release. It’s a no go, it’s a toy project.
I’ve played with anlmosy every backup tool in existence, and earlier Kopia allowed me to corrupt its datastore, twice. I found the most resilient app is duplicacy. So, there is that. When there is a choice between nice UI and reliable backup — the latter wins.
But at first I did reject duplicacy after one look on the ui. Only after every other tool had failed to actually do backup reliably i revisited duplicacy again.
So yes, duplicacy’s UI is damaging reputation.
3 Likes
Btw there is no “official” container. Likely because duplicacy does need one - it’s self contained monolithic executable with no dependencies. There are no benefits in container overheard.
Some use them as a form of a scheduler — but it’s silly. Every OS comes with a built-on scheduler already.
Most of this type of single-job software in the world is maintained by a proverbial “one guy”. Duplicacy, Arq, restic, tarsnap, rclone, borg, attic, duplicity, ….
So, you can spend time to test every single one, in various scenarios, review the past history, developer responsiveness at fixing issues over long periods of time, resilience to corruption of the tool itself, etc and make your choice, or you can ask for advice of other folks who have done it, and then research validity of their advice and aggregate it into the decision.
My conclusion so far and for the past many years was and still is as following
- FreeBSD and Linux: duplicacy CLI (CLI only, free for personal use, paid for commercial)
- macOS and Windows: Arq7 (commercial, excellent UI)
The next best thing in each category is so far behind that it’s not with considering it.
Ultimately, any of those products can cease to exist any time once 1) developer loses interest 2) product requires more support than income it generated. This is very important to consider when you pick a solution for long term backup, to mitigate risk of needing early migration.
1 Like
Huge thanks for your detailed explanation, I will avoid kopia.
Btw there is no “official” container.
I am using unraid as my main nas system and don’t intend to switch any time soon, so I need to use docker or a vm.
How does duplicati rank on your list? I have tried it today and to me it seems like a good compromise with a huge community and ok web ui for my needs.
Not necessarily. See this recent thread: How to generate ssh keys in dupkicacy web ui docker container on unraid - #6 by saspus
It’s not on my list. It used to be when I started, but now it’s beyond bottom of the list. It’s not a piece of software worthy consideration. Besides obviously lacking a stable version (why EoLs old codebase before new one is stable?) it’s absolute garbage in the one task it is designed to do — keeping your backups intact. Its database gets corrupted if you just look at it wrong, and reliance on Mono framework does not help overal reliability.
Pretend it does not exist.
The problems were very well detailed by @saspus: they are essentially unfinished software (one in zero version and the other in eternal beta). They have one huge point of failure in common: they use databases.
One of the main reasons for Duplicacy’s robustness is that it only relies on the storage filesystem.
1 Like
I don’t think Kopia uses central databases; its design is very similar to duplicacy’s: they use content addressable storage (CAS).
1 Like
My conclusion so far and for the past many years was and still is as following
- FreeBSD and Linux: duplicacy CLI (CLI only, free for personal use, paid for commercial)
- macOS and Windows: Arq7 (commercial, excellent UI)
Are there any reasons, outside of the nice GUI, to prefer Arq7 over Duplicacy CLI? (Windows user)
1 Like
I don’t want to advertise the competitor’s product here, but since this can be framed as “why did duplciacy lost customer” and ultimately help improve the product – I’ll do.
I’m macOS user but some things will still apply to windows too.
Dealbreakers (has been requested from duplicacy many times). One is enough:
- Support for AWS Archival storage. This is a full stop. Forcing users to use hot storage for backup is recklessly wasteful.
- Support for user-mounted filesystems (via helper process and impersonation). On windows this is a separate disaster because the OS does not support concurrent connections to the same server with different user. (Microsoft invented SMB – and yet, here we are)
- Support for macOS multi-user environment (SIP, access: I made a workaround for myself here).
Nice-to-haves (not dealbreakers, but have been requested many times too):
- CPU throttling
- Prevent/pause/throttle backup with on battery power
- AWS cost management
- AWS object lock management
New developments:
3 Likes
Thank you very much, appreciated!
With Arq 6 i lost all of my backups. it was a terrible disaster.
The publisher has raised the bar with Arq 7, which is now beginning to mature, however, it remains weak on reliability. For example, I encountered a lot of VMDK corruption issues on powered off Workstation VM backups, an issue I never had with Duplicacy.
But the most annoying thing is that there is no procedure or possibility of repair in case of file corruption. If some files are corrupted, it is not uncommon to lose the roof of the backup set because you cannot restore it.
Apart from the rigid and unintuitive side of the Arq 7 interface, it remains interesting as a second backup software.
The big difference that makes me put Duplicacy in front is its incredible robustness, and all the possibilities of being able to recover your data when a backup is corrupted, because the huge advantage is that there is no database system to store meta data or whatever, everything happens at the file system level, it’s incredibly powerful and robust.
And with Duplicacy, the only limitation on performance is only hardware, CPU, RAM or disk IO. Arq 7, sometimes, does nothing for hours, no CPU load, no disk IO, no full RAM, then it goes back to its slow speed until the next shutdown.
With Duplicacy, I almost halved my backup times.
3 Likes
I believe Kopia does use central databases, or what they called index. I briefly looked at how it worked when it just came out. In my opinion their way of handling the databases is too complicated, even more complicated than other competitors.
This has changed, I think around 0.9, now indexes are append-only and also live in CAS.
Can’t comment on arq6. It was very short lived, electron based UI, experiment. Not even a contender. It was rightfully scraped shortly after.
I doubt it had anything to do with a backup tool. Any backup solution is as reliable as underlying storage. On the other hand, with encryption enabled with any tool, including Arq and duplicacy, there is no way for you to end up with bad data even if storage rots. You either get back exactly what was backed up or nothing.
There are two factors that affect reliability (assuming your ram and cpu are working correctly): storage and networking.
For storage it’s easy: Use reliable storage that guarantees data consistency. Repairing data rot is a job of a filesystem, not applications. Datastore corruption is an impossibility.
Duplicacy does support erasure coding, so it can tolerate some rot — but the problem is, if even one byte is allowed to rot, who is to say that it will be limited to that one byte? Ultimately, the storage either guarantees data integrity or it does not. In the former case you don’t need to worry’s about corruption. In the latter — you don’t have any guarantees, so don’t use that storage.
The other factor — resilience to network interruptions — is indeed important, and both tools handle it very well, according to my limited synthetic testing and years of use.
Performance is never a selling point of a backup tool. It’s simply irrelevant. Yes, duplicacy is very fast, which is nice, but I take CPU throttling over any super-speed backup utilizing full resources. There is no reason to hurry up to finish backup in 5 minutes to then wait for 12 hours for another backup.
Duplicacy benefits from a very fast regex engine, this helps a lot with scanning when exclusions are specified. Arq5 used to be very, ridiculously slow for the reasons I did not look into. I don’t know about arq7, because I no longer use manual filters. Everything gets backed up indiscriminately.
This statement applies to any piece of software on earth 
Arq 7, sometimes, does nothing for hours, no CPU load, no disk IO, no full RAM, then it goes back to its slow speed until the next shutdown
Have you got to the root cause of this? Apps don’t just do nothing, when they appear to be not doing anything with all stacks idle they are likely waiting for some asynchronous request, network, or disk IO. This is not specific to ark or duplicacy or Microsoft word. Nobody inserts delineate delays in the code
For the VMDK corruption with the support we think the corruption was during restore. We search but never find why.
I make a new test just few minutes and with last version of Arq 7 this seems working. If i reinstall the old Arq 7 that i use when i have the troubles, problem occurs again with a new one,
if I launch several backups taking care to have modifications in the VMDK (just start then stop the VM) in one snapshot out of two I have no files displayed and therefore nothing to restore.
This doesn’t seem to happen with the latest version, I couldn’t find anything interesting in the changelogs that might relate to this issue but it’s not very reassuring.
So the easiest thing for me was to go with Duplicacy, much less obscure, very practical in its CLI version, and just as little to work under linux, which allows me to have a single backup format for my three platforms.
For performance I don’t agree, nowadays it’s not up to humans to wait for tools, so I prefer a reliable and fast tool, than a reliable but slow tool. When I launch a new backup, I need the time needed to achieve it to be realistic, Arq 7 has improved on this point but not enough for my taste and given the articles found on the internet I am not not the only one to think that.
But fortunately there are many tools, this makes it possible to satisfy more people and find the tool best suited to their needs.
PS: I really like restic too which seems a very good tool 
3 Likes