How to view restore progress in web UI?

pieter · 6 January 2020 16:33

I lost a drive array, and I am trying to restore data using the web UI, using the SASPUS docker container on Unraid.
I started the restore, and the web UI has been saying “starting” for the last 30 minutes.

I am afraid to leave the restore page, as I’ve read reports that the restore stops when you leave the page.

I can see that files are starting to appear, so the restore is doing something, but what?
How do I view status of the restore?
Will the restore job really stop if I navigate away from the restore page?

pieter · 7 January 2020 02:26

I left the restore running, when I returned from the office, there was an error about an expired token.
The restore log does not report any info on actual file details, so I don’t know what was restored and what not:

2020/01/06 07:56:48 Total files: 1636626
2020/01/06 07:56:48 Listed 1627102 files for backup B2-Backup-Backup revision 101 in the storage B2-Backup
2020/01/06 07:57:19 192.168.1.113:62114 POST /list_restore_directory
2020/01/06 07:57:48 192.168.1.113:62114 POST /list_local_directory
2020/01/06 07:57:54 192.168.1.113:62114 POST /list_local_directory
2020/01/06 07:58:22 192.168.1.113:62114 POST /list_restore_directory
2020/01/06 07:58:42 192.168.1.113:62114 POST /list_restore_directory
2020/01/06 07:59:01 192.168.1.113:62114 POST /start_restore
2020/01/06 07:59:01 Created log file /logs/restore-20200106-075901.log
2020/01/06 07:59:01 Created restore session 1o8nqw
2020/01/06 07:59:01 Running /home/duplicacy/.duplicacy-web/bin/duplicacy_linux_x64_2.3.0 [-log restore -r 101 -storage B2-Backup -overwrite -stats --]
2020/01/06 07:59:01 Set current working directory to /cache/localhost/restore
2020/01/06 09:37:35 Deleted listing session jnji7l
2020/01/06 09:37:41 Invalid session
2020/01/06 12:07:57 Stopping the restore operation in session 1o8nqw
2020/01/06 12:07:57 Failed to restore files for backup B2-Backup-Backup revision 101 in the storage B2-Backup: Duplicacy was aborted
2020/01/06 12:07:57 closing log file restore-20200106-075901.log
2020/01/06 12:08:17 Deleted restore session 1o8nqw

The backup log just ends:

2020-01-06 12:07:55.832 INFO DOWNLOAD_PROGRESS Downloaded chunk 21636 size 2227236, 7.92MB/s 04:48:43 43.7%
2020-01-06 12:07:56.032 INFO DOWNLOAD_PROGRESS Downloaded chunk 21637 size 1792990, 7.92MB/s 04:48:43 43.7%
2020-01-06 12:07:56.879 INFO DOWNLOAD_PROGRESS Downloaded chunk 21638 size 8951659, 7.92MB/s 04:48:41 43.7%
2020-01-06 12:07:57.068 INFO DOWNLOAD_PROGRESS Downloaded chunk 21639 size 1891565, 7.92MB/s 04:48:42 43.7%

How do I get the restore to complete, and get logs on what files are being restored?

leerspace · 7 January 2020 02:37

The status is displayed at the bottom of the restore tab, right above the restore button (which turns into a stop button during the restore).

Yes. That’s how it currently works. See Web UI: navigate away from page during restore operation?

You could list the files in the restore directory to see what’s been restored already – assuming you were restoring to an empty directory. But if you just run the restore again I think it’ll basically pick up where it left off.

If you want the restore to go faster, I would suggest specifying -threads 8 (or something higher than 1) in the options field of the restore tab.

pieter · 7 January 2020 03:06

Wow, this is, in my opinion, a pretty terrible experience, close UI restore stops, what if my browser closes or my machine goes to sleep, what if it takes days to restore, and no way to see file by name restore progress reported in logs.

I really expected this to work like a start job from web, come back occasionally to see progress, use logs to troubleshoot.

I tried -threads 8, and I’ll see if it completes before failing.
Btw, it is a single 234GB file I’m trying to restore, ETA currently reported as 7 days…

pieter · 7 January 2020 12:23

Failed again after running for about 5 hours

Trying to do this using the CLI, but having difficulty with the list and restore command.

From the Web UI I enter the B2 bucket parameters, on the restore UI I list the backup ID’s in the bucket.
How do I list the available backup ID’s (or are they snapshots?) from the CLI?

After I selected an ID in the Web UI, I list the revisions.
In the CLI it looks like this:

C:\Users\piete\Downloads\Duplicacy>duplicacy -v list -id B2-Backup-Backup
Storage set to b2://ptr727-Duplicacy-Server-1-Backup
Using 16384 iterations for key derivation
[0] URL request 'HEAD https://f002.backblazeb2.com/file/ptr727-Duplicacy-Server-1-Backup/nesting'             returned status code 404
Listing revisions for snapshot B2-Backup-Backup
Snapshot B2-Backup-Backup revision 1 created at 2019-09-02 11:26 -hash
Snapshot B2-Backup-Backup revision 28 created at 2019-10-07 00:00
Snapshot B2-Backup-Backup revision 41 created at 2019-11-07 00:00
Snapshot B2-Backup-Backup revision 76 created at 2019-12-12 00:00
Snapshot B2-Backup-Backup revision 83 created at 2019-12-19 00:00
Snapshot B2-Backup-Backup revision 90 created at 2019-12-26 00:00
Snapshot B2-Backup-Backup revision 95 created at 2019-12-31 00:00
Snapshot B2-Backup-Backup revision 96 created at 2020-01-01 00:00
Snapshot B2-Backup-Backup revision 97 created at 2020-01-02 00:01
Snapshot B2-Backup-Backup revision 98 created at 2020-01-03 00:00
Snapshot B2-Backup-Backup revision 99 created at 2020-01-04 00:00
Snapshot B2-Backup-Backup revision 100 created at 2020-01-05 00:01
Snapshot B2-Backup-Backup revision 101 created at 2020-01-06 00:00

In the Web UI I can now select a revision, and then list the files.
In my CLI I can also list the files:

C:\Users\piete\Downloads\Duplicacy>duplicacy -v list -id B2-Backup-Backup -r 101 -files
Storage set to b2://ptr727-Duplicacy-Server-1-Backup
Using 16384 iterations for key derivation
[0] URL request 'HEAD https://f002.backblazeb2.com/file/ptr727-Duplicacy-Server-1-Backup/nesting' returned status code 404
Snapshot B2-Backup-Backup revision 101 created at 2020-01-06 00:100:
...
255834283004 2019-12-30 06:48:02 
7d96c2deac5629b9ac26f1a16d51a298faa58088cf69943b5723c77e1963b8c6 Unraid/2019-12-30@03.00/CA_backup.tar.gz
Files: 1636626, total size: 1347710659951, file chunks: 296653, metadata chunks: 164

When I try to restore the file from the CLI, it is using the wrong snapshot (or is it ID?):

C:\Users\piete\Downloads\Duplicacy>duplicacy -v restore -r 101 -stats -threads 8 -overwrite -ignore-owner "Unraid/2019-12-30@03.00/CA_backup.tar.gz"
Storage set to b2://ptr727-Duplicacy-Server-1-Backup
Loaded 1 include/exclude pattern(s)
Using 16384 iterations for key derivation
[0] URL request 'HEAD https://f002.backblazeb2.com/file/ptr727-Duplicacy-Server-1-Backup/nesting' returned status code 404
+Unraid/2019-12-30@03.00/CA_backup.tar.gz
[0] URL request 'HEAD https://f002.backblazeb2.com/file/ptr727-Duplicacy-Server-1-    
Backup/snapshots/B2-Backup/101' returned status code 404
Snapshot B2-Backup at revision 101 does not exist

Note it is using snapshot B2-Backup instead of B2-Backup-Backup.
How can I specify what ID / snapshot to use in the restore command?

gchen · 7 January 2020 13:15

The restore command doesn’t accept the -id option, so you can only restore for the id that was supplied when running the init command.

You can edit C:\Users\piete\Downloads\Duplicacy\.duplicacy\preferences to change the id from B2-Backup to B2-Backup-Backup.

pieter · 7 January 2020 15:05

Ok, thx, done, and now restore of the file is running via CLI.

How do I list the available backup ID’s (or are they snapshots?) from the CLI?
Similar to how the web UI flow works, config storage then select id then revision then file.

I had to restart the restore, after a few hours it failed with a connection error.

Downloaded chunk 34683 size 13140565, 15.05MB/s 01:20:44 70.1%
Failed to download the chunk 
1ff9d2c082d06226b0d81019338d048bf5a4428827a3fc0d3f6f337d66fd7fa9: read tcp 192.168.1.113:49858->206.190.215.16:443: wsarecv: An existing connection was forcibly closed by the remote host.

Downloaded chunk 49473 size 2706939, 13.52MB/s 00:00:01 100.0%
Downloaded Unraid/2019-12-30@03.00/CA_backup.tar.gz (255834283004)
Restored C:\Users\piete\Downloads\Duplicacy to revision 101
Files: 1 total, 243982.58M bytes
Downloaded 1 file, 243982.58M bytes, 14795 chunks
Total running time: 01:30:23

Is there no retry logic in case of a connection failure?