Backup Immutability - Object Lock support?

[dgcom]

@towerbr

You have to use B2 CLI tool

Or just call an API.
B2 tool does not like restricted API keys, so you can’t upload:

 ConsoleTool cannot work with a bucket-restricted key and no listBuckets capability
 ERROR: application key has no listBuckets capability, which is required for the b2 command-line tool

I already tested this with small C# snippet and it seems to work, maybe useful for some other project I have.

I think with this it should be possible to get Duplicacy to create sort of write-only, no delete backups, although I am not sure if Duplicacy relies on listFiles…
And you’ll still need to be able to delete files somehow - possibly running cleanup manually with different API key.

[towerbr]

@dgcom

Or just call an API.

I didn’t understand your point, the above commands with the CLI tool are calling B2 API.

I use two keys, one for backup and one for prune.

The “backup” key has the permissions listBuckets,listFiles,readFiles,writeFiles

And the “prune” key the permissions listBuckets,listFiles,readFiles,writeFiles,deleteFiles

The particularity in my case is that I only execute prune manually (and rarely). The “prune key” is encrypted with GPG and I provide the password at run time.

I think the only vulnerability in my case is if a ransonware was able to install a keylogger and capture the password I type for the prune key when I use it.

[dullage]

@towerbr

I use two keys, one for backup and one for prune .

The “backup” key has the permissions listBuckets,listFiles,readFiles,writeFiles

And the “prune” key the permissions listBuckets,listFiles,readFiles,writeFiles,deleteFiles

The particularity in my case is that I only execute prune manually (and rarely). The “prune key” is encrypted with GPG and I provide the password at run time.

I do exactly the same thing and it works well. The only other thing is to ensure the bucket Life Cycle settings keep prior versions (at least for a while) as writeFiles allows files to be overwritten (if you only keep the latest version). That said, I think this is a requirement for fossilisation anyway.

Should an attacker choose to mess up all the chucks at least I’d have a copy. Albeit some DIY scripting would be needed to restore a backup from previous versions of chucks.

[dgcom]

This may work if you can manually run prune regularly - not really a good option in every case. I still think read only snapshots on the storage side are better solution.
I also prefer solution where Duplicacy creates local backup (on NAS or removable drive) and then separate job (on central NAS) pushes data to cloud storage using rclone.
Looks like rclone also now supports “soft delete”, I need to figure out what would be a minimal set of key capabilities for rclone to properly mirror Duplicacy backup to B2, including prune and let life cycle to take care of “soft deleted” files.

[tranceFusion]

@dullage

I do exactly the same thing and it works well. The only other thing is to ensure the bucket Life Cycle settings keep prior versions (at least for a while) as writeFiles allows files to be overwritten (if you only keep the latest version). That said, I think this is a requirement for fossilisation anyway.

So you are saying that outside of pruning, duplicacy doesn’t need to delete anything? including metadata?

[tallgrass]

@towerbr is right indeed. Also linking to another thread where we talked about this in the context of Backblaze. I think I came to the conclusion that some of this is a happy accident of how Duplicacy works on b2 storage.

How secure is duplicacy?

You have to use the B2 API to assign specific capabilities (Application Keys). The website doesn’t let you get that granular. The easiest way is to use their B2 command line tool (Get the Command-Line Tool). Using the b2 cli tool: b2 authorize-account b2 create-key --bucket [bucket-id] [new-backup-key-name] listBuckets,listFiles,readFiles,writeFiles b2 create-key --bucket [bucket-id] [new-prune-key-name] listBuckets,listFiles,readFiles,writeFiles,deleteFiles Just be sure to save the Key ID …

[towerbr]

@tallgrass

Also linking to another thread where we talked about this in the context of Backblaze

Thanks! I was looking for exactly this topic to quote here, but I haven’t found it, I was probably not using the right words in the search.

[towerbr]

@tranceFusion

So you are saying that outside of pruning, duplicacy doesn’t need to delete anything? including metadata?

Yep!
…

[dgcom]

I did some testing and I still think that using B2 in this way is not very helpful

I really do not see any useful way to recover if adversary used b2 tool to soft-delete all your files.

And it is worse if you sync local backup to cloud using rclone - data is on B2, but there is no tool I know off which can recover B2 state to a point in time. Ideally, it should work as snapshot - you get hacked today, you clean up, go to your cloud storage and tell it to get you to the state you were yesterday.
I searched for a tool, which would allow B2 point-in-time recovery, but besides some hack-ish scripts didn’t find any. It would be great to have this functionality included in Duplicacy and rclone, but until it is there, I’ll stay with more expensive, but proven backend with snapshots

[towerbr]

@dgcom

I’ll stay with more expensive, but proven backend with snapshots

And what would these backends be?

[dgcom]

@towerbr

And what would these backends be?

This would be rsync.net (I mention this earlier in this thread) - it uses ZFS and provide 7 daily snapshots free, but you can setup flexible schedule and only pay for difference in data in older snapshots.
I use it for some time already, but I am always on a lookout for alternatives.

[towerbr]

@dgcom

This would be rsync.net (I mention this earlier in this thread)

Sorry, I had read it quickly and didn’t notice the snapshot aspect.

So I have two options:

• B2 (with its retention policies and different permissions per key) for 0.005 / GB / month

or

• rsync.net with its snapshot protection for 0.025 / GB / month

[dgcom]

Keep in mind that:

• You only pay for storage on rsync.net - they do not charge for ingress/egress
• B2 - you can get free egress if needed, but you pay for ingress and API calls (but probably not too much)
• If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

Still, B2 will probably be cheaper, especially if you are on a budget. But absence of the clear way to recover data to a previous state in case of attack makes this irrelevant for now.

[towerbr]

@dgcom

but you pay for ingress

No, there is no cost of ingress.

@dgcom

and API calls

Some, but more related to download / egress. For backups, class A (free) transactions are the most used.

https://www.backblaze.com/b2/b2-transactions-price.html

@dgcom

If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

Interesting . As they support SFTP, maybe I’ll do some tests later.

[dgcom]

@towerbr

No, there is no cost of ingress.

You are right, ingress is free, I mixed up with something else… With Cloudflare it makes storage relatively economical, but I am not sure how well this works with Duplicacy - I know there was a merged pull request for B2-custom backend, but I do not see much documentation or feedback on this feature…

I also like rsync.net because of ssh/sftp support - this makes it a bit more compatible with other data storage approaches

[tedsmith]

@dgcom

If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

I believe that if you read the fine print these cheaper accounts don’t support snapshots.

[dgcom]

@tedsmith

I believe that if you read the fine print these cheaper accounts don’t support snapshots.

Not sure where did you see that - I have such account and I do have snapshots

[saspus]

@dgcom

If you go through the right door, you get 0.015/GB/month - rsync.net/products/rclone.html

Whoah! That’s a really nice door!

What is the catch though? (Reading fine print furiously)

[dgcom]

There is no catch… As I understand it, people who run the service are bunch of Unix techs, who know the value of borg and rclone tools and provide special discount for their users (you can replace rclone.html with borh.html and you’ll get same discount).

I exchanged email with their support too - pleasant experience, very knowlegable.

[tedsmith]

@dgcom

Not sure where did you see that - I have such account and I do have snapshots

Sorry I got the plans you were talking about confused with borg plans: Cloud Storage for Offsite Backups - borg support (rsync.net) which says “Free ZFS filesystem snapshots are not included since you’ll be doing versioning and retention with borg.”