Windows Server Deduplication

Bug
#2

Duplicacy should be capable of handing deduplicated files without special configuration.

File size being 0 is a bit weird. Can you run fsutil reparsepoint query on that file?

#3

Typical error in log file

2019-11-06 00:00:52.703 WARN LIST_LINK Failed to read the symlink Data/Administration/Hospitality/Guesthouse/guest house/Old guest house information/Information of Dormitory 2 & 3 - 7/Transport Instruction between Airport & Guesthouse.docx: 
The system cannot find the path specified.

fsutil reparsepoint query output:

Reparse Tag Value : 0x80000013
Tag value: Microsoft

Reparse Data Length: 0x0000007c
Reparse Data:
0000:  01 02 etc...
0010:  00 00 etc...
0020:  9d 58 etc...
0030:  dd 98 etc...
0040:  60 5b etc...
0060:  e0 90 etc...
0070:  61 ab etc...

All the files marked “WARN LIST_LINK Failed to read the symlink” does exist and can be read, so I don’t understand what is happening.

Thanks

#4

Maybe a permission issue?

#5

It does not appear to be a permission issue as the other files with the same permissions work.
However, from the logs all the “symlink errors stating the path cannot be found” are reparse points.

Any ideas why duplicacy does not like windows server deduplication?

I have duplicacy_web_installer_win64_1.1.0.exe installed as a service through admin console.

#6

The service runs under the localsystem account by default which may not have the right permission. Try change the account that the service runs under or simply run the web GUI as a normal application to see if it is permission related.

#7

Isn’t this a duplicate of this other thread?

#8

No. I am backing up local drives on local host, not network shares.

#9

Ok after some tinkering… I got it to work using admin account in the service and removing -vss options.

With -vss options I am getting “WARN LIST_LINK Failed to read the symlink _filename” Any idea how to fix

#10

In my test the -vss option works without an issue on a data deduplication volume.

Did you get the same The system cannot find the path specified error with -vss?

#11

When -vss is enabled, a file is accessed via a different path. For example, d:\repository\file1 will be accessed as d:\repository.duplicacy\shadow\repository\file1, where d:\repository.duplicacy\shadow is a symlink pointing to the newly created shadow copy of d:.

So maybe you don’t have permissions to this shadow symlink? While duplicacy is running, try to access this file in File Explorer.

#12

Yeah, you’re in a different situation than what I was. I am on V1.0 of the web GUI, and have had no problems backing up files on local drives with data deduplication enabled. (This is with Duplicacy running as admin.)

I have not tried the latest version yet, so I am curious what will happen when I update.

#13

Actually my problems is not with deduplication as first though but with -vss.

Did you get the same The system cannot find the path specified error with -vss ?

Yes

So maybe you don’t have permissions to this shadow symlink? While duplicacy is running, try to access this file in File Explorer.

I am unable to find the shadow symlink…

However from the logs I can see the creation of shadow copy…

2019-11-18 14:48:31.027 INFO VSS_CREATE Creating a shadow copy for F:\
2019-11-18 14:48:36.072 INFO VSS_DONE Shadow copy {4AEBF9B5-2799-4997-95CF-73F57A635E01} created
.
.
.
.
2019-11-18 14:48:49.264 INFO VSS_DELETE The shadow copy has been successfully deleted

I notice after running with -vss option the the vss writers get stuck in state 5.

 PS C:\Windows\system32> vssadmin list writers
 vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
 (C) Copyright 2001-2013 Microsoft Corp.
 
 Writer name: 'Task Scheduler Writer'
    Writer Id: {d61d61c8-d73a-4eee-8cdd-f6f9786b7124}
    Writer Instance Id: {1bddd48e-5052-49db-9b07-b96f96727e6b}
    State: [1] Stable
    Last error: No error
 
 Writer name: 'VSS Metadata Store Writer'
    Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
    Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
    State: [1] Stable
    Last error: No error
 
 Writer name: 'Performance Counters Writer'
    Writer Id: {0bada1de-01a9-4625-8278-69e735f39dd2}
    Writer Instance Id: {f0086dda-9efc-47c5-8eb6-a944c3d09381}
    State: [1] Stable
    Last error: No error
 
 Writer name: 'System Writer'
    Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
    Writer Instance Id: {5a4cfaeb-1e6a-4bce-a853-799ad751fc61}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'ASR Writer'
    Writer Id: {be000cbe-11fe-4426-9c58-531aa6355fc4}
    Writer Instance Id: {9a310c71-e64e-4607-85e7-b93d48f813d4}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'SqlServerWriter'
    Writer Id: {a65faa63-5ea8-4ebc-9dbd-a0c4db26912a}
    Writer Instance Id: {60f9efb8-5ecb-4a5c-b306-de11514ce0d3}
    State: [5] Waiting for completion
    Last error: No error
 
Writer name: 'Shadow Copy Optimization Writer'
    Writer Id: {4dc3bdd4-ab48-4d07-adb0-3bee2926fd7f}
    Writer Instance Id: {4cae5944-c54c-4119-97f0-ba9f99e297ac}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'FSRM Writer'
    Writer Id: {12ce4370-5bb7-4c58-a76a-e5d5097e3674}
    Writer Instance Id: {1d70a379-550d-433e-8ce8-642aa8ebf6cb}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'Dedup Writer'
    Writer Id: {41db4dbf-6046-470e-8ad5-d5081dfb1b70}
    Writer Instance Id: {5c377b12-c377-4107-8895-401acaefc3c5}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'IIS Metabase Writer'
    Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
    Writer Instance Id: {65cf5abd-7799-448d-9b15-b4c63ac379eb}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'IIS Config Writer'
    Writer Id: {2a40fd15-dfca-4aa8-a654-1f8c654603f6}
    Writer Instance Id: {21811a0b-99f0-4554-966b-d60bef9b65bc}
    State: [5] Waiting for completion
   Last error: No error
 
 Writer name: 'WMI Writer'
    Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
    Writer Instance Id: {1e47ca85-e44f-45e1-ae7e-b4069c5fb002}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'Registry Writer'
    Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
    Writer Instance Id: {0fb4ae1f-dedd-4db4-b38e-3499931db0f0}
    State: [5] Waiting for completion
    Last error: No error
 
 Writer name: 'COM+ REGDB Writer'
    Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
    Writer Instance Id: {f42d7831-2650-4353-8b78-ee1f04581913}
    State: [5] Waiting for completion
    Last error: No error

Any ideas how to get vss to work properly?

Thanks

#14

I have upgraded to web edition 1.1.0, and for the most part everything is working fine. I had a few scheduled jobs with the -vss option complete successfully over the weekend.

I checked my VSS writer state, and while I have a few that are in state 5 (IIS Metabase Writer, IIS Config Writer, BITS Writer, WMI Writer, NTDS), backup jobs with -vss still complete successfully.

My Duplicacy service is running on the Local System account.

#15

You can only see that shadow symlink when the backup is running. So while the program is running, open DOS prompt or Explorer to check files under this shadow symlink (especially those files that cause access errors).

#16

My repository is E:\ so the shadow symlink is E:\.duplicacy\shadow ? No success trying to find the shadow symlinks with duplicacy running…

#17

If you’re running the web GUI as service, the repository is actually c:\ProgramData\.duplicacy-web\repositories\localhost\n (where n is a number), and you can find the symlink under c:\ProgramData\.duplicacy-web\repositories\localhost\n\.duplicacy.

#18

It might be easier to isolate the issue if you can run the CLI from a DOS prompt (or Powershell). Copy the preference file from c:\ProgramData\.duplicacy-web\repositories\localhost\n\.duplicacy to e:\.duplicacy\ and run the backup directly from there:

cd e:\
mkdir e:\.duplicacy
copy c:\ProgramData\.duplicacy-web\repositories\localhost\n\.duplicacy\preferences e:\.duplicacy
c:\ProgramData\.duplicacy-web\bin\duplicacy_win_x64_2.2.3.exe backup -vss
#19

I tried it and it seems to work fine…Comparing with the files that fail when -vss is enabled in web gui…

I have theory that the “symlink errors: WARN LIST_LINK Failed to read the symlink” are caused because the file path is too long when the shadow symlinks are found in “c:\ProgramData.duplicacy-web\repositories\localhost\n.duplicacy\shadow” at least in Windows 2012 R2.

Could this be a possibility?

1 Like
#20

This must be it! To handle long file paths Duplicacy tries to access files using UNC paths, but earlier versions of go did not support accessing shadow copies via UNC paths, so this shadow symlink was used to work around the issue. I believe this issue has been fixed in the latest version of go – I’ll release a fix very soon.

4 Likes
#21

To recap, this issue occurs when the full path of a symlink path exceeds the Windows limit (about 260 characters). The reason why it happened only with the -vss option in this case is because with the shadow copy the symlink is accessed via an alternative path which can be much longer.

Also it can happen to any symlinks, even on a Windows volume without data deduplication. When the Windows data deduplication feature is enabled, however, this is more likely to happen because every deduplicated file becomes a symlink.

The fix is to use the UNC path when reading a symlink:

However, it appears to me that this fix is not necessary if you have the latest version of go, because now filepath.Join already produces UNC paths.

I’ll make a new release containing this fix by the end of this week.

6 Likes