Dont buy this scam

[AZERTY]

With Arq 6 i lost all of my backups. it was a terrible disaster.

The publisher has raised the bar with Arq 7, which is now beginning to mature, however, it remains weak on reliability. For example, I encountered a lot of VMDK corruption issues on powered off Workstation VM backups, an issue I never had with Duplicacy.

But the most annoying thing is that there is no procedure or possibility of repair in case of file corruption. If some files are corrupted, it is not uncommon to lose the roof of the backup set because you cannot restore it.

Apart from the rigid and unintuitive side of the Arq 7 interface, it remains interesting as a second backup software.

The big difference that makes me put Duplicacy in front is its incredible robustness, and all the possibilities of being able to recover your data when a backup is corrupted, because the huge advantage is that there is no database system to store meta data or whatever, everything happens at the file system level, it’s incredibly powerful and robust.

And with Duplicacy, the only limitation on performance is only hardware, CPU, RAM or disk IO. Arq 7, sometimes, does nothing for hours, no CPU load, no disk IO, no full RAM, then it goes back to its slow speed until the next shutdown.

With Duplicacy, I almost halved my backup times.

[gchen]

@saspus

I don’t think Kopia uses central databases; its design is very similar to duplicacy’s: they use content addressable storage (CAS).

I believe Kopia does use central databases, or what they called index. I briefly looked at how it worked when it just came out. In my opinion their way of handling the databases is too complicated, even more complicated than other competitors.

[saspus]

@gchen

when it just came out.

This has changed, I think around 0.9, now indexes are append-only and also live in CAS.

[saspus]

Can’t comment on arq6. It was very short lived, electron based UI, experiment. Not even a contender. It was rightfully scraped shortly after.

@AZERTY

For example, I encountered a lot of VMDK corruption issues on powered off Workstation VM backups, an issue I never had with Duplicacy.

I doubt it had anything to do with a backup tool. Any backup solution is as reliable as underlying storage. On the other hand, with encryption enabled with any tool, including Arq and duplicacy, there is no way for you to end up with bad data even if storage rots. You either get back exactly what was backed up or nothing.

@AZERTY

But the most annoying thing is that there is no procedure or possibility of repair in case of file corruption.

There are two factors that affect reliability (assuming your ram and cpu are working correctly): storage and networking.

For storage it’s easy: Use reliable storage that guarantees data consistency. Repairing data rot is a job of a filesystem, not applications. Datastore corruption is an impossibility.

Duplicacy does support erasure coding, so it can tolerate some rot — but the problem is, if even one byte is allowed to rot, who is to say that it will be limited to that one byte? Ultimately, the storage either guarantees data integrity or it does not. In the former case you don’t need to worry’s about corruption. In the latter — you don’t have any guarantees, so don’t use that storage.

The other factor — resilience to network interruptions — is indeed important, and both tools handle it very well, according to my limited synthetic testing and years of use.

@AZERTY

And with Duplicacy, the only limitation on performance is only hardware

Performance is never a selling point of a backup tool. It’s simply irrelevant. Yes, duplicacy is very fast, which is nice, but I take CPU throttling over any super-speed backup utilizing full resources. There is no reason to hurry up to finish backup in 5 minutes to then wait for 12 hours for another backup.

Duplicacy benefits from a very fast regex engine, this helps a lot with scanning when exclusions are specified. Arq5 used to be very, ridiculously slow for the reasons I did not look into. I don’t know about arq7, because I no longer use manual filters. Everything gets backed up indiscriminately.

@AZERTY

And with Duplicacy, the only limitation on performance is only hardware, CPU, RAM or disk IO.

This statement applies to any piece of software on earth

Arq 7, sometimes, does nothing for hours, no CPU load, no disk IO, no full RAM, then it goes back to its slow speed until the next shutdown

Have you got to the root cause of this? Apps don’t just do nothing, when they appear to be not doing anything with all stacks idle they are likely waiting for some asynchronous request, network, or disk IO. This is not specific to ark or duplicacy or Microsoft word. Nobody inserts delineate delays in the code

[AZERTY]

For the VMDK corruption with the support we think the corruption was during restore. We search but never find why.
I make a new test just few minutes and with last version of Arq 7 this seems working. If i reinstall the old Arq 7 that i use when i have the troubles, problem occurs again with a new one,

if I launch several backups taking care to have modifications in the VMDK (just start then stop the VM) in one snapshot out of two I have no files displayed and therefore nothing to restore.

This doesn’t seem to happen with the latest version, I couldn’t find anything interesting in the changelogs that might relate to this issue but it’s not very reassuring.

So the easiest thing for me was to go with Duplicacy, much less obscure, very practical in its CLI version, and just as little to work under linux, which allows me to have a single backup format for my three platforms.

For performance I don’t agree, nowadays it’s not up to humans to wait for tools, so I prefer a reliable and fast tool, than a reliable but slow tool. When I launch a new backup, I need the time needed to achieve it to be realistic, Arq 7 has improved on this point but not enough for my taste and given the articles found on the internet I am not not the only one to think that.

But fortunately there are many tools, this makes it possible to satisfy more people and find the tool best suited to their needs.

PS: I really like restic too which seems a very good tool

[saspus]

@AZERTY

I make a new test just few minutes and with last version of Arq 7 this seems working. If i reinstall the old Arq 7 that i use when i have the troubles, problem occurs again with a new one

Ha! This is very interesting. Maybe some weird versioning incompatibility. If this is repeatable – i.e. you backup a file, restore a file and it’s different – send that file to arq, they should be able to debug.

@AZERTY

if I launch several backups taking care to have modifications in the VMDK (just start then stop the VM) in one snapshot out of two I have no files displayed and therefore nothing to restore.

Can you clarify this a bit? are you saying arq does not detect changed files? Or does vm solution change files without changing timestamp?

@AZERTY

So the easiest thing for me was to go with Duplicacy, much less obscure, very practical in its CLI version, and just as little to work under linux,

Absolutely. Simplicity and robustness is what I value in duplicacy as well. This is great for servers. But:

• for workstations and personal laptops I value creature comfort more.
• Support for Archival storage is still missing. I don’t want to pay 4x in storage fees for no reason. for 4TB this is literary difference between $50 and and $200/year. I can find better use for $150.

@AZERTY

nowadays it’s not up to humans to wait for tools

If that was about anything else I would agree. But backup specifically is a background task, there is no hurry ever, nobody is waiting for it to end. If you backup once every hours – it has the whole hour to make that incremental backup.

Just like when some people don’t use dishwashers because “I can wash dishes faster!”. Well, guess what you are not doing when the dishwasher takes 2 hours to do that – you are not washing dishes!.

Also, when I’m running on 5% battery remaining, and duplicacy decides to complete backup in 30 seconds, fans blaring, immediately sending my laptop to sleep, I strongly dislike it. I’d prefer it to take 3 hours and not disturb me.

@AZERTY

s not up to humans to wait for tools, so I prefer a reliable and fast tool, than a reliable but slow tool. When I launch a new backup, I need the time needed to achieve it to be realistic

Two comments here:

• initial backup is a one-time operation, and therefore it does not matter how long it takes.
• I’m yet to see a backup tool that is limited by anything but network. Granted, I don’t run them on raspberry pis…

@AZERTY

articles found on the internet I am not not the only one to think that.

This is a bit sideways comment… but I increasingly grew to take stuff written on the internet with a giant boulder of salt In vast majority of cases there is some sort of undisclosed factor making the whole ordeal pointless and misleading. The remaining genuine testimonials are so far and between that they are not not worth the efforts to sifting though mountains of garbage. So, I don’t read reviews anymore, no longer ask questions on forums, and don’t form any opinion on anything, unless I’ve looked and tried things myself. Expecially when technology is concerned.

@AZERTY

But fortunately there are many tools, this makes it possible to satisfy more people and find the tool best suited to their needs.

Yes. Competition is good, and nothing is perfect.

@AZERTY

PS: I really like restic too which seems a very good tool

Restic is great. Borg is also fine. Some people get away with using git for backup. My main day-to-day backup is, in fact, Time Machine. Why? Because it’s integrated into the OS and provides excellent UI. It takes 4 hours to complete an incremental backup – because it’s a background service and it stays out of the way. It supports exclusions if needed, but I don’t use them, storage is cheap. Arq to AWS is for disaster recovery, I never expect to need to restore from it.

At some point it becomes too much to keep pouring over one solution, and you want something that “just works” without the need to read forums and write scripts. Time Machine and Arq allow me to do that, (so I can obsess over other things )

[duplic]

LoL dupliacy.com doesn’t even comply with GDPR, good luck getting sued.
(C) 2019 is just sad, who doesn’t auto update that.
You handle customer data, where are your privacy policies?
Even with external payment processing, you still have at least the name and email of the person.

[saspus]

@duplic

You handle customer data, where are your privacy policies?

This indeed looks like an oversight. @gchen?

[gchen]

@saspus

This indeed looks like an oversight. @gchen?

https://duplicacy.com/privacy_policy.html

Constructive criticisms are welcome, but labeling my paid software as a scam (while continuing to use my free software) is highly unprofessional, to say the least.

[saspus]

@gchen

@saspus

This indeed looks like an oversight. @gchen?

https://duplicacy.com/privacy_policy.html

There are no links to it from anywhere I looked — including the homepage of duplicacy.com. It’s impossible to find it. Per GDPR it shall be prominently displayed and easy to find.

@gchen

while continuing to use my free software) is highly unprofessional, to say the least.

Completely agree.

[duplic]

while continuing to use my free software) is highly unprofessional, to say the least.

Yes, ofc it is highly unprofessional. As a company, it is also highly unprofessional to completely ignore basic consumer rights and to interpret law in a very strange way. That’s the cost of doing worldwide business. GDPR is not an opt-out law, like install a browser plugins to disable Google Analytics in your policy. It’s the complete opposite, the customers have to actively agree and only after that you can send their data away. If the link is not accessible from the site, the policy basically doesn’t exist.
Yes, calling your product a scam is overdramatic, but there are some shady things going on and everything I touch feels deprecated.
The constructive part of the criticism above is the following.
Either redesign the whole site and make the paid version actually feature complete with a good gui or pay someone to do it.

[gchen]

This is for the forum: Privacy - Duplicacy Forum. You can access it by clicking the three line icon on the top right corner and then clicking Privacy. It is just the default Discourse template and was set up before there was GDPR. Certainly it needs an update.

[Tarnsten]

@saspus

The next best thing in each category is so far behind that it’s not with considering it.

What was your opinion/reasoning on Borg? The main reason I ask is because I used duplicacy for some time, but ended up switching back to Borg because:

a) Before using ZFS as a target for backups, I could use Borgs append-only mode to protect against hacks on the client-side (as in, if server/client was compromised I could guarantee storage side would not be also impacted)
b) The cache in duplicacy seems to blow out to prohibitively large proportions
c) I can mount Borg as a browsable filesystem

I like the 1-repo-many-clients aspect of duplicacy though, and would like to know (from your opinion/experience) if there is any compelling reason to reconsider duplicacy over borg?

[saspus]

TLDR – I like Borg, but I trust duplicacy

@Tarnsten

Before using ZFS as a target for backups, I could use Borgs append-only mode to protect against hacks on the client-side (as in, if server/client was compromised I could guarantee storage side would not be also impacted)

Duplicacy never modifies data on the target, it’s essentially always in append-only mode. So you can set permissions to allow read and write, but not modify or delete. Or, indeed, just take periodic filesystem snapshots, so this makes the point moot.

@Tarnsten

The cache in duplicacy seems to blow out to prohibitively large proportions

Hmm. I haven’t noticed. I was using it on a Mac to backup all local users, total about 4TB. I thought only chunks for the snapshots are cached – and those represent very small amount of space. Under what usecase do you see unbounded cache growth?

@Tarnsten

c) I can mount Borg as a browsable filesystem

This is a feature I suggested long ago, Mount duplicacy snapshot as a virtual filesystem, and it was recently implemented by a fellow forum member, see at the end of that thread, so you can build it and use if you want, but personally I now consider it just a gimmick. I would not install FUSE on a Mac, and the feature while nice is not a dealbreaker in any way, but does add complexity and unnecessary dependencies.

@Tarnsten

if there is any compelling reason to reconsider duplicacy over borg?

Yes, the reasons are the same as before – it’s stable as a rock and simple enough for me to trust the implementation. I’m prepared to pay with creature comfort for such assurances.

[Tarnsten]

Thanks for the response!

@saspus

So you can set permissions to allow read and write, but not modify or delete.

Is this also an option in terms of SFTP access though, for example? I.e. borg serve means that I can associate the SSH key with an append only flag; is this also an option in duplicacy (and if so, I have missed that)? Or are you just referring to standard options like object lock etc on B2/S3 compat. systems?

@saspus

Under what usecase do you see unbounded cache growth?

The backup size I was using was ~1.9TB at the time, and the cache was somewhere between 150-200GB, which was a far cry larger than borg’s 2GB for ~3TB atm. I tried recreating a few times but re-encountered this same growth.

@saspus

I would not install FUSE on a Mac, and the feature while nice is not a dealbreaker in any way, but does add complexity and unnecessary dependencies.

Yeah I agree re: not installing FUSE on a Mac. The only reason it was more of an “issue” for me with duplicacy was because I found some inconsistencies/difficulties in restoring compared to borg (and this is presumably on me; I had just hoped at that time for a “mount” option to quickly work around the issues I was having restoring - so that I could just use rsync instead).

[saspus]

@Tarnsten

Or are you just referring to standard options like object lock etc on B2/S3 compat. systems?

duplicacy is serverless, all it needs just a basic file access api. Object lock on S3 storage is not supported (there is a FR for it), but you can accomplish the bulk of this with server-side permissions; here is a relevant recent discussion with some details: Using CLI on headless linux server - #2 by saspus, there are a few details.

@Tarnsten

which was a far cry larger than borg’s 2GB for ~3TB atm. I tried recreating a few times but re-encountered this same growth.

I’m wondering if there were a lot of revision chunks that got downloaded? Were you doing just a backup or also check/prune? Just the backup should not have used cache at all.
(Arq keeps about 40GB of caches on my Mac, which is quite a bit more than Borg, but indeed not third of a terabyte. its weird)

@Tarnsten

I found some inconsistencies/difficulties in restoring

Yes, essentially you need to figure out exclusions patterns to filter what to restore, and it’s not straightforward.

I’m thinking of this as a last resort backup and I never expect to need to restore. So difficulties restoring are pretty much irrelevant – if I find myself in a situation where duplicity backup is my last hope – I won’t mind a few minutes of reading documentation. In the day-to-day life I’m relying a Time Machine to a remote (over ZeroTier) server – because of excellent UI, reliability, and system integration. Time machine keeps a bunch of local snapshots as well, and usually those are more than enough to rectify mishaps. If not – Time Machine bundle on my server is already almost 9 years old, and I’m not planning on starting over any time soon

Ideally duplicacy could implement a FileProvider client, instead of a fuse based vfs, to be a first class macOS citizen, or, to make it cross-platform and compatible with anything without the need of ugly kernel extensions – local NFS server service duplicacy’s CAS storage. Mountain duck takes this approach. And it can probably be even done outside of duplicacy as a separate utility, or via go api (if duplicity offers any). This of course all does not exist today, and should not be used in making a decision on which backup tool to use.

What in Borg made you reconsider other choices?

[Tarnsten]

@saspus

duplicacy is serverless, all it needs just a basic file access api. Object lock on S3 storage is not supported (there is a FR for it), but you can accomplish the bulk of this with server-side permissions; here is a relevant recent discussion with some details: Using CLI on headless linux server - #2 by saspus, there are a few details.

I will take a read of this - thanks!

@saspus

I’m wondering if there were a lot of revision chunks that got downloaded? Were you doing just a backup or also check/prune? Just the backup should not have used cache at all.
(Arq keeps about 40GB of caches on my Mac, which is quite a bit more than Borg, but indeed not third of a terabyte. its weird)

I had checks + prunes, but they were done via Web UI scheduler at that point, and it was a couple of years ago so I cannot attest to their success at the time.

@saspus

if I find myself in a situation where duplicity backup is my last hope – I won’t mind a few minutes of reading documentation

Fair point - though for me, as part of my backup procedure I also do regular restore testing, so I’d not feel comfortable until I’d “figured that out” so to speak.

@saspus

In the day-to-day life I’m relying a Time Machine to a remote (over ZeroTier) server – because of excellent UI, reliability, and system integration

Agreed - TimeMachine is great, though these days work is all cloud-based storage, so everything important goes there (and then I control the backups of that to various other storages for the company). I only own ipad + iphone for personal devices, so I run TrueNAS for my actual storage needs (and plug into that with FileBrowserGo on the devices).

@saspus

What in Borg made you reconsider other choices?

No dissatisfaction - quite the opposite, I am quite satisfied. However, I know borg is releasing a new version sometime soon, which will be a breaking change. Old version will be supported for some time but I will eventually need to re-upload my entire content. I am in Australia so my internet is not the greatest, and so hand-in-hand with this I thought I’d just revisit duplicacy and see if it makes sense to switch (or not).

Right now I get my easy-access backups from raw encrypted ZFS-send’s to a remote, so from that perspective my lack of “easy restore” would be covered for duplicacy here. I do also like the idea of being able to backup without leaving the “restore” key on the server, though realistically this doesn’t matter drastically given an attacker has server access anyway and can read files on source. I’ve partially mitigated this with borg anyway by loading the password from 1Pass CLI integrations via service account. The main protection I’m afforded this way is that if the server is physically stolen, the ZFS pool remains encrypted, and the borg password is not in plaintext on the server (so I can revoke the 1Pass token before the thieves got a chance to read the borg password out of that, for example).

At any rate, I’m rambling a little - but tl;dr is that I’m just re-assessing options, and from reading various posts here your opinion seemed informed, so thought I’d get another perspective!