Hi, had a question about the copy command performance and specifying a subset of snapshot ids instead of using the default of all ids.
Current setup:
- Have 6 or so snapshot ids configured on a single linux server (NAS). Such as photos, music, home videos, documents, etc.
- I have multiple storages - originally it was B2, then I later added a large USB3 HD (20TB) to have another local copy. I am using filters to restrict what is going to B2 (~ 3 TB), and what going to the USB HD (~10+ TB).
- The B2 storage was added/populated first (which is smaller), and the USB one later, so I am currently backing up to both locations as separate tasks.
- I’ve recently rediscovered some older USB HDs I’ve had laying around, a mix of sizes, USB2 vs USB3.
- I now have a 4 TB USB3 HD set up as another storage, which is set up as bit-identical to the 20 TB HD. It does have a different encryption password though.
What I’m currently trying is to do is a copy from the 20 TB USB3 HD to the 4 TB, but since there’s way more data than what would fit, I’m specifying multiple -id arguments to only copy over chunks that are in those snapshot ids (I’m excluding a couple of the snapshots to lower the size to fit).
The “issue”, is that it’s really slow, between 2 USB3 external hard drives, with 4 threads, it’s averaging only about 1.5 MB/s. I had used another pair of USB2 external HDs (2TB each), backup to one a smaller amount of data, then then copy from that, with all the snapshot ids, and it was over 30 MB/s (maxing out the USB2 connection I think).
So my question is… is it so slow because duplicacy needs to spend a bunch of processing to determine which chunks are in those snapshots I specified before it can copy them over? vs if its all snapshots, it doesn’t need to do any of that and copies everything? CPU usage is not very high. Is the differing encryption passwords for the storages to blame?
I realize the setup probably isn’t optimal, but I don’t want to necessarily blow it all up to redefine everything yet. In this case, would it be better to just do a straight backup to the 4TB drive, instead of trying to do a copy?
Thanks for any insight!