Backup Immutability - Object Lock support?

[tranceFusion]

@dullage

I do exactly the same thing and it works well. The only other thing is to ensure the bucket Life Cycle settings keep prior versions (at least for a while) as writeFiles allows files to be overwritten (if you only keep the latest version). That said, I think this is a requirement for fossilisation anyway.

So you are saying that outside of pruning, duplicacy doesn’t need to delete anything? including metadata?

[tallgrass]

@towerbr is right indeed. Also linking to another thread where we talked about this in the context of Backblaze. I think I came to the conclusion that some of this is a happy accident of how Duplicacy works on b2 storage.

How secure is duplicacy?

You have to use the B2 API to assign specific capabilities (Application Keys). The website doesn’t let you get that granular. The easiest way is to use their B2 command line tool (Get the Command-Line Tool). Using the b2 cli tool: b2 authorize-account b2 create-key --bucket [bucket-id] [new-backup-key-name] listBuckets,listFiles,readFiles,writeFiles b2 create-key --bucket [bucket-id] [new-prune-key-name] listBuckets,listFiles,readFiles,writeFiles,deleteFiles Just be sure to save the Key ID …

[towerbr]

@tallgrass

Also linking to another thread where we talked about this in the context of Backblaze

Thanks! I was looking for exactly this topic to quote here, but I haven’t found it, I was probably not using the right words in the search.

[towerbr]

@tranceFusion

So you are saying that outside of pruning, duplicacy doesn’t need to delete anything? including metadata?

Yep!
…

[dgcom]

I did some testing and I still think that using B2 in this way is not very helpful

I really do not see any useful way to recover if adversary used b2 tool to soft-delete all your files.

And it is worse if you sync local backup to cloud using rclone - data is on B2, but there is no tool I know off which can recover B2 state to a point in time. Ideally, it should work as snapshot - you get hacked today, you clean up, go to your cloud storage and tell it to get you to the state you were yesterday.
I searched for a tool, which would allow B2 point-in-time recovery, but besides some hack-ish scripts didn’t find any. It would be great to have this functionality included in Duplicacy and rclone, but until it is there, I’ll stay with more expensive, but proven backend with snapshots

[towerbr]

@dgcom

I’ll stay with more expensive, but proven backend with snapshots

And what would these backends be?

[dgcom]

@towerbr

And what would these backends be?

This would be rsync.net (I mention this earlier in this thread) - it uses ZFS and provide 7 daily snapshots free, but you can setup flexible schedule and only pay for difference in data in older snapshots.
I use it for some time already, but I am always on a lookout for alternatives.

[towerbr]

@dgcom

This would be rsync.net (I mention this earlier in this thread)

Sorry, I had read it quickly and didn’t notice the snapshot aspect.

So I have two options:

• B2 (with its retention policies and different permissions per key) for 0.005 / GB / month

or

• rsync.net with its snapshot protection for 0.025 / GB / month

[dgcom]

Keep in mind that:

• You only pay for storage on rsync.net - they do not charge for ingress/egress
• B2 - you can get free egress if needed, but you pay for ingress and API calls (but probably not too much)
• If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

Still, B2 will probably be cheaper, especially if you are on a budget. But absence of the clear way to recover data to a previous state in case of attack makes this irrelevant for now.

[towerbr]

@dgcom

but you pay for ingress

No, there is no cost of ingress.

@dgcom

and API calls

Some, but more related to download / egress. For backups, class A (free) transactions are the most used.

https://www.backblaze.com/b2/b2-transactions-price.html

@dgcom

If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

Interesting . As they support SFTP, maybe I’ll do some tests later.

[dgcom]

@towerbr

No, there is no cost of ingress.

You are right, ingress is free, I mixed up with something else… With Cloudflare it makes storage relatively economical, but I am not sure how well this works with Duplicacy - I know there was a merged pull request for B2-custom backend, but I do not see much documentation or feedback on this feature…

I also like rsync.net because of ssh/sftp support - this makes it a bit more compatible with other data storage approaches

[tedsmith]

@dgcom

If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

I believe that if you read the fine print these cheaper accounts don’t support snapshots.

[dgcom]

@tedsmith

I believe that if you read the fine print these cheaper accounts don’t support snapshots.

Not sure where did you see that - I have such account and I do have snapshots

[saspus]

@dgcom

If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

Whoah! That’s a really nice door!

What is the catch though? (Reading fine print furiously)

[dgcom]

There is no catch… As I understand it, people who run the service are bunch of Unix techs, who know the value of borg and rclone tools and provide special discount for their users (you can replace rclone.html with borh.html and you’ll get same discount).

I exchanged email with their support too - pleasant experience, very knowlegable.

[tedsmith]

@dgcom

Not sure where did you see that - I have such account and I do have snapshots

Sorry I got the plans you were talking about confused with borg plans: Cloud Storage for Offsite Backups - borg support (rsync.net) which says “Free ZFS filesystem snapshots are not included since you’ll be doing versioning and retention with borg.”

[dgcom]

That’s interesting, I did not realize that borg plans don’t include snapshots!
Glad that I found rclone plans at one point

Also, keep in mind that pricing structure is different - you have to purchase 100GB min, after that it can auto-grow in 20% increments after reaching 90% use (this is from one of the emails with their support).

[tedsmith]

@dgcom

I really do not see any useful way to recover if adversary used b2 tool to soft-delete all your files.

And it is worse if you sync local backup to cloud using rclone - data is on B2, but there is no tool I know off which can recover B2 state to a point in time. Ideally, it should work as snapshot - you get hacked today, you clean up, go to your cloud storage and tell it to get you to the state you were yesterday.

Isn’t the case that if you don’t give yourself delete permissions on B2 (i.e. you can’t prune with duplicacy) then the adversary can’t delete its backup files? This has no worse storage downsides than the snapshot style backup (i.e. both will keep growing forever but will be deduplicated.) I use duplicacy to copy my local duplicacy backups to b2. Then restoring from B2 is no different than restoring from a local backup. You can use revision labels to simplify keeping track of which revisions from each repository go together - perhaps have the label be the date.

[saspus]

Thank you. I think I misread the pricing. This comes out to be $15/TB/month, which is 3x more than even BackBlaze. I don’t see the point in using them. Snapshots are questionable benefit to pay triple for.

[dgcom]

@tedsmith

Isn’t the case that if you don’t give yourself delete permissions on B2 (i.e. you can’t prune with duplicacy) then the adversary can’t delete its backup files?

Adversary cannot permanently delete files, but they can hide file or upload corrupted file version.
If that happens, there is no tool to recover previous, good state for large number of files - I so far could not find one which will restore bucket to previous state at a point I need - i.e. simulate reverting to the snapshot.
Basically, there is a way to protect from data loss, but there is no simple way to recover
Also, keep in mind, you’ll have to do prune manually all the time which might be good for lower-volume home backup.

@saspus

Snapshots are questionable benefit to pay triple for.

It depends on your acceptance of the risk - how much you value your data