How to add local storage when I already have B2

Support
#1

So I realize that I am doing this sort of backwards but I have repository already backed up to B2 and realize I would like to add a local copy. I am trying to figure out a way to accomplish this that will end up with repositories that are in sync so I can do a local backup and then do a copy to B2 in the future while also avoiding downloading the entire backup from B2 in order to set up the local. Can I:

  1. duplicacy add -copy my-b2-backup my-local-backup snapshot-id /local/file/path
  2. duplicacy backup -storage my-local-backup.
  3. From that point forward do my backups to local and then do a copy to the online?
#2

Effectively yes, and the same concept was recently discussed here.

You don’t have to keep the snapshot IDs as alluded to in that thread - in fact it makes things much easier to simply use new ones from that point forward (3) as you can follow precisely those steps.

However, if you did want to keep and reuse the IDs then some additional steps might be necessary:

  • 2b. duplicacy backup -storage my-b2-backup -hash
  • 2c. duplicacy copy -id <snapshot id> -r <last revision> -from my-b2-backup -to my-local-backup
  • 2d. Optional: manually delete the revision 1 file created in step 2a.

The -hash option makes sure every chunk that a fresh, local, backup generates, should also exist on B2. This may not be the case if your repositories evolve quite a bit over time. A chunk back at revision 1 may include parts of files that no longer exist, but they won’t get pruned because they’re still referenced.

So when the chunking algorithm looks at the way your files are now organised, while most chunks are likely de-duplicated and deterministic, some at the boundaries will be new.

Personally, I’d use a unique set of snapshot IDs for the new setup and start from revision 1.

You wouldn’t have to worry about running a -hash as these get recomputed on the local side anyway and uploaded if missing. Plus you can still keep the old snapshots around until they get pruned out of the B2 storage.

2 Likes
#3

Okay I like the idea of just setting up new snapshot ID’s since it would result in revision numbers being in sync as well as everything else. So I would end up with local and online that are in sync going forward, none to minimal downloads to start and I wouldn’t have to reupload the stuff that already exists? That would be perfect but I am unclear as to how to go about this. Do I do something like:

duplicacy init -e -storage-name my-b2-backup new-snapshot-id b2://old_b2_bucket_url
duplicacy add -copy my-b2-backup my-local-backup new-snapshot-id /local/file/path
duplicacy backup -storage my-local-backup
duplicacy copy -from my-local-backup -to my-b2-backup

Then from that point on I can just:

duplicacy backup -storage my-local-backup
duplicacy copy -from my-local-backup -to my-b2-backup

I realize I will need to use the same encryption password as the current online snapshot-id uses.

Also is it okay to have the local unencrypted and the online encrypted or will that break deduplication or result in unencrypted stuff in the online backup?

#4

That looks about right, but you may not need to do the init if you already have a .duplicacy/preferences file that points to the existing B2 storage. Even if you use a different snapshot ID (which is really just a dummy name) or storage name, you’ll likely get ‘The repository has already been initialized’.

I honestly don’t know, I guess you could try it with a local test repo and local test storage.

Personally, I wouldn’t recommend it, as the unencrypted config file may hold details that may be used to unlock the encrypted one(?). What would be the benefit of having no encryption anyway?

Edit: Just tested it…

Seems you can add a copy-compatible unencrypted storage from an encrypted one. In fact, that’s what would happen if you omit the -encrypt / -e option as per your second command above (add -copy).

The config file becomes fully readable, but I don’t know if any of the keys inside would assist a potential hacker to unlock your encrypted storage. Personally, I’d use add -encrypt -copy. :slight_smile:

#5

Okay thanks for that. Seems like everything should work the way I want.

I agree with you that the safe bet is to encrypt both local and online. However I can also see making an argument for a local backup that you have physical possession of being unencrypted in case of a worst case disaster scenario.

Thanks again for the help.