Clarification on copy to cloud of local network backup

Support
, , , , , ,
#1

Hello,

I have seen from documentation and other posts that it is possible to do the following:

  • backup my laptop repository to an SFTP storage on a NAS (using GUI on laptop to perform a backup)
  • copy the NAS backup storage to backblaze storage (using CLI on NAS to perform a copy)

I have successfully started the first step i.e. creating a backup of the laptop to a location on the NAS.

I am now completely confused as to where I should run the ADD command to create a secondary storage on backblaze… Do I run this on the laptop so I have two storage entries in the laptop preferences? I did this successfully but then couldn’t work out how the CLI on the NAS would know what storage entries I would be referring to if I ran a COPY command.

Or do I run the ADD command on the NAS? When I tried this, it said the repository had not been initialised. I expect in this case I need to run INIT first. However I need to specify a storage when running INIT - what should this be? I assume some dummy storage, if I would still need to perform ADD to create a storage with ‘- copy’ suitable to run the COPY command?

Is is possible for someone to outline the sequence of steps to perform to setup this 2 tier backup strategy?

Thanks
Nick

1 Like
#2

You can run the COPY command either on your laptop or on your NAS.

To run it on your laptop, run these commands:

cd /path/to/repository
duplicacy init repository_id sftp://user@NAS/storage/path
duplicacy add b2 repository_id sftp://user@NAS/storage/path -copy default
duplicacy backup
duplicacy copy -from default -to b2

To run it from your NAS, run these commands on your laptop:

cd /path/to/repository
duplicacy init repository_id sftp://user@NAS/storage/path
duplicacy backup

Then run these on your NAS:

cd /any/empty/directory
duplicacy init repository_id /path/to/storage/path
duplicacy add b2 repository_id sftp://user@NAS/storage/path -copy default
duplicacy copy -from default -to b2

The second way is recommended as running the COPY command on your laptop requires Duplicacy to download chunks over the network from your NAS to your laptop first and then upload to B2. To run the COPY command on your NAS, you basically create a shadow repository with the same repository id but the storage path becomes local, so you don’t need to download chunks over the network first.

1 Like
#3

I would like to ask a follow-up question to Nick’s: I am planning to setup Duplicacy for a small team. Individual clients will backup to a local SFTP server.

Q1: should I use the same repository for all clients for deduplication? But will Client A then be able to see all data from Client B?

I would then like to backup the local server to a remote SFTP server.

Q2: is the second method you recommend above suitable? if yes:

Q3: the command “duplicacy copy -from default -to b2” manually copys files from the local to the remote server, right?

Q3: how would a restore by client work from the remote server? how would a restore from remote server to the local server work?

Q4: if I set different prune schedules for the local (e.g. keep 1 year) and the remote server (e.g. 30 days), would this create conflicts?

#4

Q1: should I use the same repository for all clients for deduplication? But will Client A then be able to see all data from Client B?

Yes and Yes.

Q2: is the second method you recommend above suitable? if yes:

Yes.

Q3: the command “duplicacy copy -from default -to b2” manually copys files from the local to the remote server, right?

Right.

Q3: how would a restore by client work from the remote server? how would a restore from remote server to the local server work?

The client can add the remote B2 server as the additional storage and then specify the -storage option when restoring.

You can also run the restore command on the local server to restore from the remote server. This way you’ll get the original files in the repository directory (the place where you run the init command), not in the storage directory (where snapshot/chunk files are saved)

Of course you can copy from the remote server to the local server and run the restore command on the client using the local server as the storage.

Q4: if I set different prune schedules for the local (e.g. keep 1 year) and the remote server (e.g. 30 days), would this create conflicts?

No, but then you will be careful when you run the copy command. You’ll probably need to write a script to parse the output from the list command to get the snapshots in the last 30 days and only copy these snapshots. The copy command doesn’t support more options for snapshot selection like the prune command does.

#5

Q4: if I set different prune schedules for the local (e.g. keep 1 year) and the remote server (e.g. 30 days), would this create conflicts?

No, but then you will be careful when you run the copy command. You’ll probably need to write a script to parse the output from the list command to get the snapshots in the last 30 days and only copy these snapshots. The copy command doesn’t support more options for snapshot selection like the prune command does.

I am not sure I completely understand this. If I can’t have different prune schedules, what is then the advantage of the copy command? Would I ever prune the remote copy? I could simply do a robocopy or similar to the remote server.

Would an alternative be to:

  • duplicacy backup from clients to local server e.g. to folder local/mybackups
  • duplicacy init the folder local/mybackups on the local server
  • duplicacy backup the folder local/mybackups from local to remote server to remote/mybackups

That way I would have two pruning schedules. Let’s say I keep one revision every week on remote/mybackups. In terms of catastrophic failure of the local server I could restore the local server from the remote server how it was on “Tuesday”. This would also provide some safety if failures in the repository are backed-up.

#6

You can have different prune schedules – you’ll just need to specify the snapshots to copy otherwise it will copy snapshots older than 30 days to the remote server.

The alternative should work too.

#7

Maybe a follow up question on this one.

Rather than backing up from laptop to my local NAS and then running a copy command on the NAS (or laptop) and sending that offsite to another storage- I feel like I want to just run two backups from my laptop to the two different storages.

Is this OK? I realize that one downside is that the backups wont be identical because they would run at different times and all… but that doesnt seem bad to me.

Any thoughts?

#8

Yes, it is perfectly fine to run two backups to different storages, as long as the files do not change or you don’t consider the differences an issue.