I have a file which I may or may not have backed up using Duplicacy (it was a few months ago and now I can not remember if I included it in my Duplicacy backup or not). I want to search my Duplicacy backup storages for all revisions to try to find it. I am using the web GUI 1.6.3 (CLI 2.7.2) on Win10 Pro 21H2. There appears to be no way of conducting a search in the GUI. If I find it I want to restore it, so I would want to know all the revisions it belongs to and choose the latest revision.
The thing is, I do not know the file name I am looking for. I only know the directory structure so when I see the directory structure I can go down the paths and find the file (assuming that I did a backup of the file).
BTW: I don’t know how others work, but I very rarely remember a file name (there are simply too many). I always find files based on directory path location and not being able to search a directory path with all revisions is fustrating.
Since the filename and/or revision(s) are unknown, using Duplicacy’s CLI interface instead of its web GUI will be the easiest and fastest way by far to hopefully find what you’re looking for.
Based on your original post, I know you’ve got web edition v1.6.3 on Windows 10, but you didn’t mention…
… if Duplicacy is running as a user process or system service, so the following instructions will need to be adjusted for your particular setup.
… how many repositories (aka., backup sources) have been configured.
With quite a few assumptions…
Open a command-prompt.
Change directories into C:\path\to\.duplicacy-web\repositories\localhost\all (Adjust C:\path\to and the repository number to whatever is appropriate for your particular setup).
Run the following command: C:\path\to\.duplicacy-web\bin\duplicacy_win_x64_2.7.2.exe list -all -files > C:\duplicacy.txt
Once that’s done you can open C:\duplicacy.txt in a suitable text editor for review.
I try to keep consistent directory and file names regardless of what a file is for. This way I don’t have to recall each and every filename. I also try to minimize the number of directories so that it’s easier to file stuff away. For example, instead of this C:\Users\ABC\Photos\Hawaii\Day1\001.jpg I’d rename it to C:\Users\ABC\Photos\Hawaii.Day1.001.jpg. While I might also decide to put it in a dedicated subdirectory, I won’t have an image with a generic name such as 001.jpg (as years go by there could be hundreds, or even thousands, of them). Same naming scheme for music, videos, word processor documents, spreadsheets, etc. Each filename serves as a mnemonic device to help me easily reconstruct it.
While it might seem like a deficiency in Duplicacy, consider the file search feature in Windows… without remembering a filename – and/or something unique enough about its contents only if it’s a searchable document of some kind – it’s very difficult to find a file. For decades a common question among Windows users is how to generate a list of all the files on a computer (not an option in Windows Explorer).
Duplicacy doesn’t rely on a central database for managing backups, which is great, but the tradeoff is no global file search across backups. Contrast that to CrashPlan, where if its database is corrupted, many files could be lost – lose the database and all files are lost.
(There are also technical issues with presenting a large file list in a web browser, especially a file pick list.)
With offline backups on physical media such as CD/DVD/BD, tapes, USB flash drives, etc., a good practice is to generate a complete file index for reference so that any given file can be quickly located without manually searching every piece of media. The same can be done in Duplicacy (use the post command script feature to automate the process).
Over the years I’ve used a lot of backup software (commercial, freemium, open-source) all with different backup schemes. I can say with some certainty that if there was a perfect backup program, there wouldn’t be so many choices available, and seemingly a new one popping up every month.
I did manage to get a list of all files in all snapshot revisions and after going through about 1.5 million lines of text, I could not find the file I was looking for. So I may not have backed it up but I am also not confident in my search considering how many lines they were.
It really is a cumbersome way to search for a file somewhere in the history of the snapshots and nearly makes keeping many months/years of shapshots irrelevant for me if I can not search a file easily and efficiently
If you know what type of file – the file extension would be great – plus as much info about the potential folder names as possible, I can probably talk you through how to filter the file list instead visually reviewing it in a text editor.
I totally agree. It’s unfortunately a very difficult problem for any backup program to solve. Attempting to list millions of files in a file browser runs into all kinds of technical roadblocks.
A search feature would run into the same problems as you’re having with the file list because there’s no way to search for a folder path that’s only recognizable by sight. Each folder in the path triggers a memory which then leads you to the next subfolder in the chain, and so on, finally ending with the file you’re searching for, that you’ll recognize once you see it. Even the best search engines would have difficulty coping with such a scenario.
Not long ago I had a similar problem at work when a colleague asked for help with restoring a lost file that another former colleague had created. She didn’t know any part of the filename or the folder it was in. She didn’t know if it was in Word, a PDF, or some other type of document. The only details were a few snippets of info that the document was supposed to have and who created it. Now, picture trying to find it buried in years of backups containing more than 30 million unique files…
