Duplicacy chunks are immutable = I can use an immutable target storage? (where files can't be changed once uploaded)?

Support
#1

Wasabi has a “Compliance Mode” option for buckets, description below:

“Compliance mode prevents the deletion of any objects and provides additional information to prove that the original data is not modified since the time it was stored. Data is immutable for the length of the specified retention time.”

I’ve also setup my SFTP targets with a username which doesn’t have permission to delete/modify files.

Things look to be going ok so far. I’ve done a few smaller tests and they completed without issue. I was then able to restore without any errors.

I’m still on my initial backup for my main dataset which I’ve had to restart a few times, but over 2TB have been uploaded and I expect it to eventually complete successfully. The SFTP backup had trouble starting up initially, as it couldn’t delete a config temp file it created, so I relaxed the permissions until things got started then set them back without any further errors.

Is this expected to work, or does Duplicacy have a need to delete/modify existing files outside of pruning?

1 Like
How secure is duplicacy?
#2

I am looking at doing the same thing. Just started testing over the last few days and when uploading a second repository from a second machine got a compliance-related error:

So perhaps it won’t work…

EDIT:

However resumes fine when I restart the backup…

#3

Just guessing here: during fossil collection/retrieval chunks are renamed; some backends do not support rename operation, so Copy/delete is used instead. That could be failing.

That said, in my opinion using immutable storage with backup is not worth the trouble. A lot of space will end up being wasted, and if you need redundancy - just replicate that backup store to another destination.

#4

My understanding is only partial, but as I am not doing any pruning I would assume there are no fossils?

As for the immutable storage, my reasoning is that it would prevent a malicious actor with access to my laptop (and therefore the API keys perhaps) from deleting the cloud storage. Replicating to another (online, at least) destination I don’t think would solve that. And I am lazy with the offline backups. :slight_smile:

As I don’t have of a ton of data I’m fine with enforcing 1 year retention or something like that.

#5

My thoughts as well. Once I can confirm it’s setup well, I plan on hitting the “Lock Compliance Mode” button. That way I won’t be able to turn off Compliance Mode without calling Wasabi’s Support team. It’s still not as good as physical offline backups, as some social engineering can still trick Support into unlocking it, but it’s the best security option I’ve seen from any cloud provider.

My backups/restores are working fine so far. I haven’t tested adding a second repository, but I’m happy to hear you got it working!

#6

Wasabi’s Compliance mode, along with my ideal SFTP settings, would prohibit renaming operations anyway. The way I understand it, no space would be wasted with Compliance mode if you already planned on disabling pruning.

#7

According to this rename should not be happening during chunk upload for wasabi and s3 backends (unlike SFTP and other file based storages) so I think it should work just fine with wasabi.

Is this to SFTP? or S3? Can you run the backup with -d flag to get more verbose logging and see what is it actually doing?

#8

S3. I will start verbose logging in case it happens again but so far not repeatable.

#9

There is an alternate path for immutable for those that fit a specific configuration.

If you using Google Drive for your backup (many of us with unlimited accounts via school or work), you can do the traditional push up to Google Drive via Duplicacy. The only downside is that I have found that Google is rate limiting the upload for the API so you have to be more patient if moving TB’s.

With your unlimited account (many of us on this for free), you can put TB’s up there for $0 additional monthly fees.

Now the part on the immutable. Just put Spanning Backup against this Google Drive account. It cost $40 per year which is $3.33 per month. It is an unlimited backup and 100% immutable. You cannot delete the data even if you wanted to do this. Plus, you can go back to any previous time in the history of the Google Drive account. This gives me an unlimited backup (my 6TB+) and fully immutable as well. Additionally, I am getting a cloud backup of my cloud backup so essentially 2 cloud backup of my data. Having said this, there is the bundled risk that if my original Google Drive backup of data via Duplicacy is bad so is the Spanning backup copy.

I still have other other things in place as part of the 3-2-1 backup and even go beyond that rule. My data is worth just too much to me and hard drives are cheap. So, I add in local cold backup using entirely different software from Duplicacy to remove that variable.

If you wanted to add another layer here so you can get your data back fast versus worrying about the download from Google Drive in the case the house burned down, etc., you can use Syncovery ( https://www.syncovery.com/ ) at a friends house and have this software run a one-way download of the Google Drive account with email updates so you know it is running smoothly. That comes at a one time purchase of the software and hardware. And, it is one way so anything hitting my friend’s computer most likely does not impact my Google Drive account upstream.

And, if someone hits me with ransomware and actually infects my Google Drive backup, I restore to Google Drive to a previous point prior the ransomware with Spanning via their web interface and then pull the drive from my friend’s house that gets one-way updates via Syncovery.

Final side note for those that are admins for G Suite where you can set-up numerous accounts, I suggest setting up an account just for this backup that you never use for email and never use a local Google Drive client software as a link. For me, the only thing that is accessing this unique G Suite account is the Duplicacy credentials using the Google Drive API. It would be a really aggressive ransomware attack to get that refined to impact this unique G Suite account. And, even if they did, Spanning backup is there to bail me out.

#10

By the way, this approach also means you do not really need to worry about pruning.