Restore (list files) takes too long

planned
gui

#1

Hi,

I’m trying to restore a few files from my backup set, but when I click “List Files” the GUI just hangs until I give up and kill it (some 30 minutes later).

Is it supposed to take such a long time and I should just keep waiting, or is this a bug?

Martin


#2

You can try the 2.1.0 beta which has an optimized workflow for listing and restoring:

At least you’ll have a progress indicator showing the number of files processed vs the total number of files.


#3

Thanks - I’ve tried the 2.1.0 beta. Definitely better having the indeterminate progress bar rather than just freezing.

I don’t see any info on number of files processed though, and the operation still seems to never complete which makes restore impossible :frowning:


#4

If you don’t see the number of files, then it is probably still downloading the file list. Can you run the CLI version to see how long it takes to list the files?

cd /path/to/repository
duplicacy list -r revision -files

The duplicacy executable can be downloaded from https://github.com/gilbertchen/duplicacy/releases


#5

Running that command took about 10 minutes.

Piping to wc took less time (about 1 minute) and gave a count of 2248064.


#6

Any more thoughts on what the issue in the GUI could be?


#7

Can you remove the content of .duplicacy/cache and run the list command (with the -files option) to see how fast it is? The code in the GUI version to parse the CLI output is known to be inefficient – I just wanted to know how much slower it is without the cache effect.


#8

Made a little difference - piping to wc was around 2 minutes whilst letting it print the output took 16 minutes.


#9

I plan to optimize the code that parses the CLI output in version 2.1.1.


#10

Thanks - let me know if you want me to test anything when you have it ready.


#11

Any update on when 2.1.1 will be released with the optimisations?


#12

Sorry I haven’t started working on 2.1.1. I plan to find some time next month.


#13

Hey. Any update on this issue?


#14

I looked into the the code that parses the output from restore and realized that it is already as efficient as it can get. The bottleneck is in populating the file list, but fixing it is much harder than I thought. First, we should change the order files and folders are listed in the snapshot file, so that we don’t need to download the entire snapshot file to get the top-level files and folders. Second, the tree list in wxWidgets doesn’t provide a way to create the top-level items first and then expand others on demand.

Hopefully we’ll be able to fix this issue in the new web-based GUI.


#15

Thanks for the update.


#16

Just checking in on this - has there been any improvement to restore speed through the GUI?


#17

Given that the web-based GUI (which has little in common with the current GUI version) has not yet been released, nothing has changed so far. For updates on the web-based GUI, you can watch the #web-ui tag.