Backup fails after on error. How to skip and finish the backup?

Support
#1

I started an initial backup today and it failed when it encountered an open file from QuickBooks which is always open on our server. This is a huge problem as the backup will always fail as we never close QuickBooks. How can I make the backup continue and skip files that are in use without stopping altogether?

Here’s the error log:

Running backup command from C:\Users\Administrator/.duplicacy-web/repositories/localhost/0 to back up C:/SERVER
Options: [-log backup -storage SERVER-BACKUP -threads 4 -stats]
2022-10-31 16:40:53.680 INFO REPOSITORY_SET Repository set to C:/SERVER
2022-10-31 16:40:53.680 INFO STORAGE_SET Storage set to one://SERVER-BACKUP
2022-10-31 16:40:57.596 INFO BACKUP_START No previous backup found
2022-10-31 16:40:57.596 INFO BACKUP_LIST Listing all chunks
2022-10-31 16:40:57.972 INFO BACKUP_INDEXING Indexing C:\SERVER
2022-10-31 16:40:57.972 INFO SNAPSHOT_FILTER Parsing filter file \\?\C:\Users\Administrator\.duplicacy-web\repositories\localhost\0\.duplicacy\filters
2022-10-31 16:40:57.972 INFO SNAPSHOT_FILTER Loaded 0 include/exclude pattern(s)
2022-10-31 16:43:49.815 INFO ONEDRIVE_RETRY Response code: 502; retry after 864 milliseconds
2022-10-31 17:02:06.213 INFO ONEDRIVE_RETRY Response code: 502; retry after 949 milliseconds
2022-10-31 17:04:55.651 INFO ONEDRIVE_RETRY Response code: 503; retry after 533 milliseconds
2022-10-31 17:08:14.541 INFO ONEDRIVE_RETRY Response code: 502; retry after 796 milliseconds
2022-10-31 17:16:22.566 INFO ONEDRIVE_RETRY Response code: 503; retry after 901 milliseconds
2022-10-31 17:24:44.599 INFO ONEDRIVE_RETRY Response code: 503; retry after 763 milliseconds
2022-10-31 17:24:46.978 INFO ONEDRIVE_RETRY Response code: 503; retry after 564 milliseconds
2022-10-31 18:04:12.610 INFO ONEDRIVE_RETRY Response code: 502; retry after 982 milliseconds
2022-10-31 18:04:27.880 WARN OPEN_FAILURE Failed to open file for reading: open \\?\C:\SERVER\QB\Company.QBW: The process cannot access the file because it is being used by another process.
2022-10-31 18:11:37.861 WARN OPEN_FAILURE Failed to open file for reading: open \\?\C:\SERVER\QB\Company.qbw.SearchIndex\SuggestionIndex\segments_39d: The system cannot find the file specified.
2022-10-31 18:12:42.581 INFO ONEDRIVE_RETRY Response code: 503; retry after 511 milliseconds
2022-10-31 18:12:56.094 INFO ONEDRIVE_RETRY Response code: 502; retry after 559 milliseconds
2022-10-31 18:12:57.586 INFO ONEDRIVE_RETRY Response code: 502; retry after 747 milliseconds
2022-10-31 18:13:17.245 INFO ONEDRIVE_RETRY Response code: 502; retry after 1183 milliseconds
2022-10-31 18:23:53.701 INFO ONEDRIVE_RETRY Response code: 502; retry after 887 milliseconds
2022-10-31 18:38:48.242 INFO ONEDRIVE_RETRY Response code: 502; retry after 697 milliseconds
2022-10-31 18:43:08.401 INFO ONEDRIVE_RETRY Response code: 502; retry after 854 milliseconds
2022-10-31 19:03:59.295 INFO ONEDRIVE_RETRY Response code: 502; retry after 668 milliseconds
2022-10-31 19:07:35.536 INFO ONEDRIVE_RETRY Response code: 503; retry after 812 milliseconds
2022-10-31 19:32:06.072 INFO ONEDRIVE_RETRY Response code: 502; retry after 620 milliseconds
2022-10-31 19:55:54.830 INFO ONEDRIVE_RETRY Response code: 502; retry after 547 milliseconds
2022-10-31 20:04:59.272 INFO ONEDRIVE_RETRY Response code: 503; retry after 804 milliseconds
2022-10-31 20:13:48.445 INFO ONEDRIVE_RETRY Response code: 502; retry after 596 milliseconds
2022-10-31 20:37:12.652 ERROR UPLOAD_CHUNK Failed to upload the chunk df930587cbe785d3ff11e35c11c8548f96e971bc72e391783309d29fd62bf666: open C:/Users/Administrator/Desktop/one-token.json: Access is denied.
2022-10-31 20:37:13.318 INFO INCOMPLETE_SAVE Incomplete snapshot saved to C:\Users\Administrator\.duplicacy-web\repositories\localhost\0/.duplicacy/cache/SERVER-BACKUP/incomplete_snapshot
Failed to upload the chunk df930587cbe785d3ff11e35c11c8548f96e971bc72e391783309d29fd62bf666: open C:/Users/Administrator/Desktop/one-token.json: Access is denied.
#2

Uninstall duplicacy. Then install it again, this time running the installer as administrator and install it “for all users”. Then create backup job and add -vss flag to the backup job in options.

#3

The backup is 50% done do I have to re-create it? Also, do I need to install as a service or it doesn’t matter?

#4

Yes, install it as a service, IIRC.

If you configure the same storage as target duplicacy will “adopt” it and the backup will continue. It may look like it is starting over, but the chunks that have already been uploaded won’t get re-uploaded.

Essentially, duplicacy will prepare each chunk, then realize that ah, that chunk already exist in the target, and move to the next one.

#5

On a separate note, reduce number of threads to 1 or max 2. OneDrive does not like multithreaded access like that and is throttling you.

#6

Thank you but I am now getting this error when running as a service:

Can’t start the web server at 127.0.0.1:3875: listen tep 127.0.0.1:3875: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.

#7

That means the other instance of duplicacy, installed as a user, is still running. Make sure you uninstall it, and kill any remaining duplicacy processes with the task manager.

#8

Actually, it work by going to http://127.0.0.1:3875/ but I guess I cannot use the desktop shortcut anymore or see the tray icon as it is now a service?

#9

Ah! Yes, you are right! It’s the second instance that attempted to start complained that the other one (running as a service, the correct one) holding the port already.

If the shortcut works you can bookmark it and access the UI that way, but IIRC when starting a new instance it should have launched the browser for the existing one. Maybe it broke? I vaguely remember this was being discussed in the past:… (I don’t use windows myself, sorry). @gchen?

In the meantime you can just edit the shortcut to point to the url instead.

#10

Makes sense! OK, I created a bookmark instead. Shame I don’t get to see the tray anymore though with it running as a service…

Final question: for the -vss flag - do I put it under Command options after the threads like so:

-threads 2 -vss

#11

Let’s wait for @gchen to comment — maybe there is a way to get it to work — e.g. via not installing as a service but forcing it to start as administrator via shortcut preferences. This may bring back the tray icon too.

The administrator requirement is for duplicacy to get access to VSS (Volume Shadow copy Service)

vss flag

Yes, it’s on the same level as -threads option. So your options will now look like -threads 2 -vss

#12

Awesome, thank you for all the help and prompt replies! Let’s see what @gchen says…

#13

saspus is correct, sadly you don’t get the tray icon when installed as a service. This is why I still run Duplicacy as admin user (non-service) on my own PC, and as a service on my brother’s PC (since he doesn’t need to see it), and server.

Typically, a separate user process that deals only with the tray, is spawned, and runs alongside service processes.

Incidentally, the legacy Duplicay GUI did this, but can’t remember if there was a rotating animation for when the backups ran, which would require some inter-process code to handle that.

@gchen Please change the installer to default to ‘all users’ and service mode (keeping user mode as an option), and please bring back the separate tray icon process to run alongside. Additionally, the desktop shortcut icon needs to change from regular exe shortcut, to URL shortcut - depending on install type.

#14

Thank you - all the notes to @gchen are on point.

#15

A Windows service can’t access the desktop GUI. Therefore we’ll need an ‘agent’ program to display the system tray icon and to communicate with the service. This increases the number of failure points.

This may make more sense for a brand new application. I think it is too late now – switching to this mode would create more trouble with existing installations.

#16

But this is how most tray programs work. I can think of Jellyfin, Asus fan controller software, various background scanning tools, in fact most of my tray icons on my systems alone.

Its only job would be to spin the icon when backups run (although it would be nice if it could show when errors occur), and launch the browser dashboard in the browser, stop/restart the service.

Example:

image

So many tray icons. Some of which spawn child processes to communicate with (user mode) and some which talk to service. A crashing tray icon shouldn’t stop the main GUI process.

Shouldn’t the installer detect for existing installations anyway? Even this part isn’t working properly, as it should stop existing processes or services, before attempting to overwrite files.

1 Like
#17

Why would it affect existing installs? Tray process is (by necessity) a completely separate monitor process. All it needs to be able to do is check if backup is in progress (duplicacy web can simply create named event for the duration of any operation. Or just check if duplicacy CLI is running with the parent process that’s the service — that way you don’t even need to modify duplicacy web) and “start” the URL from the duplicacy’s settings files.

It’s a little thing but with huge impact. And yes, installing as a service with -vss flag shall be default.