Web UI Restore with RSA Key and Passphrase

Support
#1

Hi

First-time poster. Recently bought Duplicacy, tried the free trial for a while and it is fantastic. I got it uploading to my remote storage and am a few days in with a few weeks remaining (slow internet connection… welcome to Australia!)

Now that one of my smaller snapshots has finally finished, I of course wanted to do a test restore. However, that’s when I discovered my mistake: I had opted for the RSA key with a passphrase… one that contains spaces.

Firstly, during the restore it simply errored out because I hadn’t provided the key at all. There are no UI controls to supply the key, so indeed I had not.

Secondly, after figuring out the CLI arguments to provide the passphrase, I tried adding them in the “Options” field in the UI.

This also failed. It appears a lot like it simply can’t handle the spaces in the passphrase.

I tried using double quotes, single quotes, nested double/single quotes in various combination, I tried replacing the spaces with   and %20 and + but without success.

It appears I simply cannot restore from the UI at all, due to the RSA key and passphrase that I have chosen.

Is there any workaround for this, within the UI?

Of course, the CLI itself is a workaround! And I realise that this is my own fault for not testing restores before I had committed quite so much time to my initial backups. Maybe then I would have simply chosen a passphrase that didn’t include spaces.

But I really like the UI in every other respect, and would largely prefer to use it for casual restores. (I’d probably use the CLI if I was recovering an entire snapshot after total system failure… but if I’ve just accidentally damaged a file and want to get back to an older version, the UI is preferable).

If there’s not, I’d propose that providing dedicated UI fields for RSA key and passphrase provision would be a nearly essential feature for the UI Restore tool.

Another small issue: during my various failed attempts, Duplicacy DID create some zero-byte files at the path I was trying to restore to, which caused future attempts to fail before they even got to the RSA decryption stage because they wouldn’t overwrite an existing file, even though it was empty. Feels like if Duplicacy restore fails due to RSA issues, it should not leave a zero-byte file behind.

Thanks,
– neverness

2 Likes
#2

You can add -key <file> -key-passphrase <passphrase> to the restore options. See details at restore · gilbertchen/duplicacy Wiki · GitHub

#3

Thanks for the quick response!

I did figure out that -key and -key-passphrase should allow me to supply my RSA key and passphrase.

Except my passphrase has spaces in it, and I can’t get Duplicacy restore in the UI to accept it via something like -key private.key -key-passphrase "My Passphrase Has Spaces" (with or without " characters).

On the CLI however, -key private.key -key-passphrase "My Passphrase Has Spaces" does work fine.

#4

Maybe if you escape the quotes? So they get passed to the shell.

I.e. what happens if you write this in the UI: -key-passphrase \"My Passphrase Has Spaces\"

Or alternatively — use embedded quotes, e.g

-key-passphrase "'My Passphrase Has Spaces'"

Or alternatively, escape spaces in your pssphrase

-key-passphrase "My\ Passphrase\ Has Spaces"

I’m not sure how duplicacy strips quotes but worth a try.

#5

Definitely worth a try, or ten!

-key private.key -key-passphrase My Passphrase Has Spaces
-key private.key -key-passphrase My\ Passphrase\ Has\ Spaces
-key private.key -key-passphrase "My Passphrase Has Spaces"
-key private.key -key-passphrase "My\ Passphrase\ Has\ Spaces"
-key private.key -key-passphrase \"My Passphrase Has Spaces\"
-key private.key -key-passphrase 'My Passphrase Has Spaces'
-key private.key -key-passphrase \'My Passphrase Has Spaces\'
-key private.key -key-passphrase My%20Passphrase%20Has%20Spaces
-key private.key -key-passphrase My+Passphrase+Has+Spaces
-key private.key -key-passphrase My&nbsp;Passphrase&nbsp;Has&nbsp;Spaces

In every case, same error:

The restore command encountered an error:
Failed to parse the private key in /path/to/private.key: asn1: syntax error: sequence truncated

Exit code: 100
#6

According to the error above, Duplicacy couldn’t load your private key. Are you providing the entire pathname?

As for the key passphrase, set an environment variable for the user account and then use something like the following for the restore options (adjust as needed for Windows.):

-key /path/to/private.key -key-passphrase $KEY_PASSPHRASE

While it’s not the most secure way to handle passphrases, on a single-user system it’s no more risky than entering it into the web UI.

Might even be handy to do the same for the private key:

-key $KEY_PATHNAME -key-passphrase $KEY_PASSPHRASE
#7

I’m certain that I’m giving it the right path. I have tried cut-n-paste from a shell to confirm. And if I give an incorrect path on purpose, I get a different error:

Failed to read the private key file: open /some/bad/path.key: no such file or directory

The environment variable approach seemed like a great idea! I’m running Duplicacy in Docker so it was easy enough to set the variables. But still, with -key $KEY_PATHNAME -key-passphrase $KEY_PASSPHRASE:

The restore command encountered an error:
Failed to read the private key file: open $KEY_PATHNAME: no such file or directory

Exit code: 100

So I guess it doesn’t expand variables in the -key argument. Tried -key /path/to/private.key -key-passphrase $KEY_PASSPHRASE too, but it also seems to have maybe not expanded the variable:

The restore command encountered an error:
Failed to parse the private key in /config/keys/encryption_key.key: asn1: syntax error: sequence truncated

Exit code: 100

I think I’m going to restart my backup anyway, with a new key that doesn’t have spaces in it. Reading more about Duplicacy today, I realised I set my remote up as a whole separate backup, and looks like doing a backup to a local folder first and then a copy to the remote repo is better practice anyway.

I appreciate the help though, I can already see the community here is really nice! I would be happy to help continue testing if my feedback helps make the UI nicer for future users.

– neverness

#8

argh… my bad… I should have read the error message above more closely. It’s not a pathname issue, but rather a problem reading the contents of the key file. Since you’d already said it works fine via command-line, there must be an issue with the asn1.go library doing the decoding. It’s as though the decoder thinks the key string is incomplete. The error is preventing it from even parsing the passphrase, so unfortunately changing the passphrase to one without spaces won’t resolve the issue.

#9

Hehe all good. I never mind re-running experiments, you never know when you’ll spot that obvious-to-everyone-else typo!

My guess is that the string IS incomplete, by the time it gets there. I think that the Options field in the UI is being split on spaces, and there’s no way to protect from that. Therefore when I provided -key-passphrase My Passphrase Has Spaces or any of the other variations, it is being passed to the CLI as if I had put -key-passphrase My. Unsuccessfully, of course.

I’d love to see two changes to the UI here:

  1. Add specific fields to the UI for RSA Key File (with a browse button), and RSA Passphrase. This will mean no need to hack things into the Options field, and of course no restrictions with special characters since the passphrase would no longer need to be tokenised/split.
  2. For Options that might contain spaces, support some form of encoding. Escaping or URL encoding, either would do the trick. (Escaping is probably more traditional, but URL encoding might be easier since then the string can still be split on regular ol’ spaces as it currently is, with the arguments URL-decoded after splitting).

Of course, the Options field could get more complicated and become a list of strings or whatever, but I like the utility of it as a text field! If only it supported escaping/encoding of special characters.

Again, this is all wish-list stuff. I can work around it entirely via CLI. But I like the UI and want it to be usable in this circumstance!

–neverness

#10

Totally confused by this. What do you mean? I “backup” source to remote storage, so why wouldn’t it be a “whole separate backup”? Why would I replicate my backup locally, then copy it to remote storage…?

#11

From my experience, main reason would be to ensure backup consistency in case of corrupted backup files caused by storage problems.
This is Duplicacy specific, because Duplicacy lacks some eqivalent of purge-broken-files which exists in Duplicati.

Example: I have backup to remote sftp/fps/webdav where is where storage is provided by single HDD. One day bad blocks will appear and some files are corrupted.

  • If I have single Duplicacy job, I have an unsolvable problem - only solution to fix check command is to remove all affected versions - which can probably be all of them

  • If I have Duplicacy copy job with bit identical, I can easily download remote copy of corrupted files o local and continue with backup

  • And as I mentioned, if you have Duplicati, then in that case you can use purge-broken-files command that removes only specific corrupted file/files from all versions - but otherwise the versions will remain

The example I mentioned does not apply to backups to reliable locations where data consistency is guaranteed (S3, etc.)

If i’m wrong please correct me :slight_smile:

Robustness with respect to missing chunks
#12

This was the use-case I had in mind, yes. My off-site storage is a hard drive at my Mum’s house, so it sure isn’t eleven 9’s of durability! Also as a secondary justification, my uploads when backing up files directly were slower over my limited internet than the copy is. I guess it’s cheaper to copy a chunk than recompute it from scratch.

Though, as a Duplicacy novice, I’m still pretty much figuring things out as I go along.

Anyway, redoing all this and restarting my backup/copy from scratch gave me the opportunity to choose entirely new passphrases, no spaces this time. So… not quite a solution to my original problem, but at least I’m now able to restore just fine using the UI by providing the -key and -key-passphrase arguments in the Options field.

—neverness

#13

ok. I think I understand. My situation is a tad different;

  1. I have shared & user files on a NAS and more user files on various desktops/laptops/tablets/phones. I back up all the peripheral systems to a shared folder on the NAS. For the most part, those backups are versioned, deduped, and compressed with local proprietary software.

  2. My NAS has snapshots enabled. I also backup my nas files & configs to an external drive using HyperBackup. That backup is also versioned and compressed.

  3. Finally, with Duplicacy, I backup all my nas files and the user backup share to a B2 cloud.

With all that, I think I’m good.

#14

Yeah sounds like you’re basically getting the same outcome (local and remote backups), just using your own (impressive!) set of other assorted tools to make the local one.

Since I’m doing my local and remote backups with Duplicacy, the copy method seems superior to the “double backup from scratch” approach.

1 Like