Duplicacy and VSS seems to fail to backup open files

mathome1 · 7 November 2018 06:21

I have an issue with Duplicacy being unable to backup open files. I am on windows, and for some reason backup jobs appear to be failing in some cases where files are open. I am using VSS, and would have assumed that should enable me to backup all files irrespective of if they are open or not. But whether I use -vss option or not the jobs are failing when some files are open.

e:\duplicacy-urgent-e>duplicacy -log backup -storage gui3 -stats -vss
2018-11-07 17:12:03.804 INFO STORAGE_SET Storage set to sftp://duplicacy@x.x.x.x//share/CACHEDEV1_DATA/duplicacy-backups/main
2018-11-07 17:12:08.584 INFO BACKUP_START Last backup at revision 10 found
2018-11-07 17:12:08.586 INFO VSS_CREATE Creating a shadow copy for e:\
2018-11-07 17:12:13.686 INFO VSS_DONE Shadow copy {8781DCD9-F839-4279-AFEB-461DB8CE6B3E} created
2018-11-07 17:12:13.689 INFO BACKUP_INDEXING Indexing e:\duplicacy-urgent-e
2018-11-07 17:12:13.690 INFO SNAPSHOT_FILTER Loaded 0 include/exclude pattern(s)
2018-11-07 17:12:16.686 WARN OPEN_FAILURE Failed to open file for reading: open e:\duplicacy-urgent-e/.duplicacy\shadow\\duplicacy-urgent-e/e-data/Interim/Spreadsheets/2018 - 2019/~$2019 - Q2.xlsx: The process cannot access the file because it is being used by another process.
2018-11-07 17:12:16.896 ERROR CHUNK_MAKER Failed to read 0 bytes: read e:\duplicacy-urgent-e/.duplicacy\shadow\\duplicacy-urgent-e/e
-data/user/Archive.pst: The process cannot access the file because another process has locked a portion of the file.
2018-11-07 17:12:16.999 INFO VSS_DELETE The shadow copy has been successfully deleted

Am I right in thinking duplicacy with VSS should be able to backup open files? If so is there anything wrong with my configuration? The only thing I can think of is :-

I am backing up a director which just has symbolic links to other directories (mklink /J).
back is for files on E: drive, rather than c: drive.

Out of curiosity I tried doing this with the old GUI, the old CLI (2.1.1), the new GUI and the new CLI (2.1.2) and all seem to give the same result.

All help in resolving this would be appreciated.

gchen · 7 November 2018 16:17

There were actually 2 files that couldn’t be opened:

2018-11-07 17:12:16.686 WARN OPEN_FAILURE Failed to open file for reading: open e:\duplicacy-urgent-e/.duplicacy\shadow\\duplicacy-urgent-e/e-data/Interim/Spreadsheets/2018 - 2019/~$2019 - Q2.xlsx: The process cannot access the file because it is being used by another process.

This one is related to VSS. You may be able to find more info in the Event Viewer. However, VSS doesn’t guarantee that any files opened by other processes can be successfully opened. It needs cooperation from other processes who need to implement the VSS writer interface correctly.

018-11-07 17:12:16.896 ERROR CHUNK_MAKER Failed to read 0 bytes: read e:\duplicacy-urgent-e/.duplicacy\shadow\\duplicacy-urgent-e/e
-data/user/Archive.pst: The process cannot access the file because another process has locked a portion of the file.

This means the file was locked by the other process using LockFile function | Microsoft Docs. The VSS won’t help here and I don’t know if there is a solution other than excluding the file.

saspus · 10 November 2018 07:32

AFAIK office files staring with ~ are transient files and they should not be backed up. Op should exclude them using filters.

For the pst — they are “special” in sense that Microsoft uses them in a way that causes issues with most backup software.

Here is relevant Crashplan article, with proposed workaround. Troubleshoot backups of Microsoft Outlook data - Code42 Support