Questions about backup and restore functionality

Support
, , , ,
#1

Hey there everyone. Here’s my problem, and a related but possibly secondary problem that stemmed from it. Any advice welcome, I feel as if I must be missing something idiotically obvious that’s right in front of me, but who knows. So, here it goes:

Main Question:

Using the Duplicacy GUI (I haven’t yet used the CLI at all for anything) I set up a first backup ID to a specific B2 bucket in a first Duplicacy trial that I had running on a first laptop containing a large set of photo files in many folders (about 321GB worth). I uploaded this from that laptop through that trial, specific to that machine.

Later, on my main (second) laptop, I uploaded another but only slightly different copy of the same photo folder from the first laptop set to the SAME bucket but under a new backup ID specific to that second laptop’s copy of those photos.

Thus, there are two distinct backup IDs going to the same B2 bucket. And from what I could see of the increase in GB in that bucket inside my B2 “buckets” panel, the second backup uploaded well, with full deduplication and only a few extra, different subfolders worth of files in backup 2 from laptops 2 slightly increasing the bucket’s total size to 340GB, which makes sense to me.

After this, I deleted the entire storage profile and backup job profile in the first trial in my first laptop mentioned above, and also deleted that entire Duplicacy installation since I don’t plan on using that first laptop anymore.

My question for this is: Both sets of backups and their cross-matching (deduplicated) files are fully intact and referenced in my B2 backup for problem-free retrieval of either through my second laptop? This is the device in which I plan on continuing with a paid edition of that second Duplicacy trial.

Second Problem/Question:

In between running the first and second laptop uploads of the whole photo folder described above, I was still learning to use Duplicacy and made several partial folder/file backups of files from both computers full backup sets, as separate backups of their own, BUT all under the same Backup ID as the first FULL backup from the first laptop.

I now know you’re not supposed to do this, but though I later deleted these partial backup jobs themselves, some of them remain listed as “Revisions” (revision 2, revision 3, etc) with dates subsequent the original first “revision” (the first full backup under that original storage that was first initiated in laptop 1 under that original Backup ID. All of these revisions are now still visible when I open my “restore” section in my second laptop’s Duplicacy trial under that first backup ID. How do I get rid of these, and will they negatively affect the corresponding underlying files from the first, full, original backup copy in any way?

I looked into Prune in the GUI, but it only offers an option to delete snapshots older than X days. Since these partial backups under that same laptop 1 Backup ID were all done after the first full backup in that laptop, I don’t see how that Prune functionality helps me.

Those are my problems here. I hope the above is more or less understandable. Any help would be appreciated.

Thanks!

#2

Already posted this before but not a single reply, so here goes again:

Here’s my problem, and a related but possibly secondary problem that stemmed from it. Any advice welcome.

Main Question:

Using the Duplicacy GUI (I haven’t yet used the CLI at all for anything) I set up a first backup ID to a specific B2 bucket in a first Duplicacy trial that I had running on a first laptop containing a large set of photo files in many folders (about 321GB worth). I uploaded this from that laptop through that trial, specific to that machine.

Later, on my main (second) laptop, I uploaded another but only slightly different copy of the same photo folder from the first laptop set to the SAME bucket but under a new backup ID specific to that second laptop’s copy of those photos.

Thus, there are two distinct backup IDs going to the same B2 bucket. And from what I could see of the increase in GB in that bucket inside my B2 “buckets” panel, the second backup uploaded well, with full deduplication and only a few extra, different subfolders worth of files in backup 2 from laptops 2 slightly increasing the bucket’s total size to 340GB, which makes sense to me.

After this, I deleted the entire storage profile and backup job profile in the first trial in my first laptop mentioned above, and also deleted that entire Duplicacy installation since I don’t plan on using that first laptop anymore.

My question for this is: Both sets of backups and their cross-matching (deduplicated) files are fully intact and referenced in my B2 backup for problem-free retrieval of either through my second laptop? This is the device in which I plan on continuing with a paid edition of that second Duplicacy trial.

Second Problem/Question:

In between running the first and second laptop uploads of the whole photo folder described above, I was still learning to use Duplicacy and made several partial folder/file backups of files from both computers full backup sets, as separate backups of their own, BUT all under the same Backup ID as the first FULL backup from the first laptop.

I now know you’re not supposed to do this, but though I later deleted these partial backup jobs themselves, some of them remain listed as “Revisions” (revision 2, revision 3, etc) with dates subsequent the original first “revision” (the first full backup under that original storage that was first initiated in laptop 1 under that original Backup ID. All of these revisions are now still visible when I open my “restore” section in my second laptop’s Duplicacy trial under that first backup ID. How do I get rid of these, and will they negatively affect the corresponding underlying files from the first, full, original backup copy in any way?

I looked into Prune in the GUI, but it only offers an option to delete snapshots older than X days. Since these partial backups under that same laptop 1 Backup ID were all done after the first full backup in that laptop, I don’t see how that Prune functionality helps.

Those are my problems here. I hope the above is more or less understandable. Any help would be appreciated.

Thanks!

#3

Yes

You just need to delete the numbered revision files in snapshots\ID on the storage destination.

You can do with prune but then you’d need to use the --exclusive flag which you should avoid doing during any other operations. So it’s just simpler to manually delete the unwanted revisions, then clean-up with prune --exhaustive.

There’s no negative affects to other snapshots.

To clean-up in the GUI, just create a new schedule and untick all the days so you can run it manually, add a prune job (accept the defaults) and edit the flags to remove the --keep's and just put --exhaustive.

#4

Hello there! Thanks so much for answering my questions. I just have a few more, hopefully pretty simple ones.

First, I tried what you said about deleting revisions in the Snapshot folder inside my B2 bucket for that storage and it worked like a charm.

This brings me to my first additional question: Will the revision deletions create any orphan chunks? The revisions in this case were partial backups of folders from the original full backup. Thus, they were all for files that are also referenced elsewhere, but I still ask for the sake of avoiding orphans and for pruning.

Second question: I couldn’t find the specific controls for the prune commands you mentioned in the GUI. I mention that I’m using the newest web GUI. Am I missing something? Since I can’t upload images here apparently, i’ve added an IMGUR link of what prune looks like for me in the GUI https://imgur.com/hwNiZbx

Third question: I notice that under the GUI (If I go to the Restore option) each of the two different storage setups I have, each reference the same backup IDs that I have also set up.

These backup IDs share many of the same files and folders from my device, but I did allocate each to a particular, specific storage, so is it normal that they appear for both storage profiles? Again, all of them do go to the same B2 bucket for the sake of taking advantage of deduplication.

Final question, and sorry about so many, but still learning here: Backing up the contents of my external HD on my main working device with the Duplicacy GUI tends to slow down the computer, so I’m thinking about shifting the backup to a second laptop I have handy and just recreating the same storage and backup ID on the other machine. Since it’s all from an external HD, the data itself is easy to shift.
Is this a problem?

Thanks again, a lot, and hope it’s not a bother to give me some ideas with these further questions.

#5

A handful - that’s why I suggested you run prune -exhaustive at some point.

To be even more efficient, you might want to run the next backup job with the -hash flag, as you may have partial unreferenced data in chunks that also reference active data. (This always will happen gradually, but it may be more pronounced starting off, especially if you reorganised stuff between backups.)

Schedule > scroll down and click the green + icon in the bottom left to add a new schedule, untick all the days of the week like so:

image

On that schedule, Add a job, select Prune, choose destination and then Save (ignore the other options for now). Go back to the job and click the green options which currently shows -keep 0:1800... blah. Delete the whole set of options and put -exhaustive. You can add -exclusive if you want the chunks gone immediately, but make sure no other operation is running once you press the play button.

Not that I can see? If you’re just shifting data once to a new device, and use the same ID, there shouldn’t be an issue.

#6

So I tried the prune process you described in detail and it worked this time, or more so, I finally understood how to make it work. I didn’t know it was possible to put in the text for exhaustive and exclusive as instructions inside the GUI. Thanks a lot.

So quickly, two final questions: I asked this too in my previous reply, is it abnormal?

As for shifting my backup of an external drive to a new device, it’s okay to point it to the same backup ID as I was using in the first device with the same drive and content? Only now from a new storage name (same storage bucket location)? I’d read that using the same backup ID in any kind of entirely new backup session is a bad idea, but i’m unclear on why if one is trying to back up the same data from the same physical source.

Alternatively, I could start again backing up my external drive through my second laptop under en entirely new backup ID and perhaps run a prune to clear the orphan files from the first, incomplete backup when I was doing it in my first laptop?

Just want to clear up these last details and sorry for the many questions. Thanks!

Recreating backup job for future scheduling on new computer