Hi,
Just noticed that a backup I have configured to the root of a drive on a Windows Server was busy indexing the “$RECYCLE.BIN” folder.
There’s no point, usually anyway, backing up this folder. Would it be possible to have a default option which might automatically populate “Excluded Folders” with the common Windows system areas. Another candidate would be the “\System Volume Information” folder.
David
That is a good idea and should be in the Windows GUI version only. When a root drive is selected as the repository, those two directories should be automatically put into the exclude list.
I’ll plan this for the next version (2.0.3).
Did this feature end up making it in? I didn’t see it in the release notes.
I’ll add this feature to the coming 2.1.0 version (which should be available by the end of next week).
Are you aware that Windows provides its own exlusion list at HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToBackup?
It would be great if duplicacy would honour that list by default…
I didn’t know that before. Those look like good candidates for populating the exclude pattern list when backing up an entire drive.
so this is now implemented in the GUI version but not the CLI version?
No, I didn’t implement this feature in GUI 2.1.0. The exclusion list under that registry needs to be converted since it is in a different syntax, and then I thought there are too many of them and it would be better for the users decide what to add themselves.
In this case, shouldn’t this be #not-planned anymore?
(i will eventually add the filters and user-scripts to #how-to)
I see how writing a conversion algorithm might be cumbersome but perhaps there is a way to get Windows to meet you at least half way? Isn’t there some command that allows you to access those paths (instead of reading them directly from the registry)? Perhaps some powershell command?
Did anyone mention PPPPPPOWERSHELL?? (please read that in an announcer voice)
Start with this:
PS C:\Users\crist> Get-ChildItem -Recurse -Path HKLM:\SYSTEM\CurrentControlSet\Control\BackupRestore
Hive: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore
Name Property
---- --------
FilesNotToBackup ETW : {%SystemRoot%\system32\LogFiles\WMI\RtBackup\*.*}
FVE2_Control : {\System Volume Information\FVE2.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.*}
FVE2_Log : {\System Volume Information\FVE2.{c9ca54a3-6983-46b7-8684-a7e5e23499e3}}
FVE2_VBB : {\System Volume Information\FVE2.{24e6f0ae-6a00-4f73-984b-75ce9942852d}}
FVE2_Wipe : {\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}}
FVE2_WipeInfo : {\System Volume Information\FVE2.{aff97bac-a69b-45da-aba1-2cfbce434750}.*}
FVE2_WipeX : {\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}.*}
FVE_Control : {\System Volume Information\FVE.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.*}
FVE_Log : {\System Volume Information\FVE.{c9ca54a3-6983-46b7-8684-a7e5e23499e3}}
FVE_Wipe : {\System Volume Information\FVE.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}}
Internet Explorer : {%UserProfile%\index.dat /s}
Kernel Dumps : {%systemroot%\Minidump\* /s, %systemroot%\memory.dmp}
Memory Page File : {\Pagefile.sys}
Mount Manager : {\System Volume Information\MountPointManagerRemoteDatabase}
MS Distributed Transaction Coordinator : {C:\Windows\system32\MSDtc\MSDTC.LOG, C:\Windows\system32\MSDtc\trace\dtctrace.log}
Netlogon : {%SystemRoot%\netlogon.chg}
Power Management : {\hiberfil.sys}
Storage Tiers Management : {\System Volume Information\Heat\*.* /s}
Temporary Files : {%TEMP%\* /s}
VSS Default Provider : {\System Volume Information\*{3808876B-C176-4e48-B7AE-04046E6CC752} /s}
VSS Service Alternate DB : {\System Volume Information\*.{7cc467ef-6865-4831-853f-2a4817fd1bca}ALT}
VSS Service DB : {\System Volume Information\*.{7cc467ef-6865-4831-853f-2a4817fd1bca}DB}
WER : {%ProgramData%\Microsoft\Windows\WER\* /s}
WUA : {%windir%\softwaredistribution\*.* /s}
BITS_metadata : {C:\ProgramData\Microsoft\Network\Downloader\*}
BITS_LOG : {C:\Windows\System32\Bits.log}
BITS_BAK : {C:\Windows\System32\Bits.bak}
FilesNotToSnapshot FVE : {$AllVolumes$\System Volume Information\FVE.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}}
FVE2_Wipe : {$AllVolumes$\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}}
FVE2_WipeX : {$AllVolumes$\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}.*}
ModernOutlookOAB : {$UserProfile$\AppData\Local\Packages\Microsoft.Office.Desktop_8wekyb3d8bbwe\LocalCache\Local\Microsoft\Outlook\*.oab /s}
ModernOutlookOST : {$UserProfile$\AppData\Local\Packages\Microsoft.Office.Desktop_8wekyb3d8bbwe\LocalCache\Local\Microsoft\Outlook\*.ost /s}
OutlookOST : {$UserProfile$\AppData\Local\Microsoft\Outlook\*.ost}
Storage Tiers Management : {\System Volume Information\Heat\*.* /s}
WUA : {%windir%\softwaredistribution\*.* /s}
KeysNotToRestore Mount Manager : {MountedDevices\}
MS Distributed Transaction Coordinator : {CurrentControlSet\Control\MSDTC\ASR\}
Pending Rename Operations : {CurrentControlSet\Control\Session Manager\PendingFileRenameOperations}
Pending Rename Operations2 : {CurrentControlSet\Control\Session Manager\PendingFileRenameOperations2}
Session Manager : {CurrentControlSet\Control\Session Manager\AllowProtectedRenames}
PS C:\Users\crist>
so we have a name and a value to use.
Now we can iterate over the results of the previous command, and by using Get-ItemPropertyValue and some more magic, you finally get:
PS C:\Users\crist> Get-ChildItem -Recurse -Path HKLM:\SYSTEM\CurrentControlSet\Control\BackupRestore | foreach {
>> Get-ItemPropertyValue -LiteralPath "Registry::$($_.name)" $_.property}
%SystemRoot%\system32\LogFiles\WMI\RtBackup\*.*
\System Volume Information\FVE2.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.*
\System Volume Information\FVE2.{c9ca54a3-6983-46b7-8684-a7e5e23499e3}
\System Volume Information\FVE2.{24e6f0ae-6a00-4f73-984b-75ce9942852d}
\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}
\System Volume Information\FVE2.{aff97bac-a69b-45da-aba1-2cfbce434750}.*
\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}.*
\System Volume Information\FVE.{e40ad34d-dae9-4bc7-95bd-b16218c10f72}.*
\System Volume Information\FVE.{c9ca54a3-6983-46b7-8684-a7e5e23499e3}
\System Volume Information\FVE.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}
%UserProfile%\index.dat /s
%systemroot%\Minidump\* /s
%systemroot%\memory.dmp
\Pagefile.sys
\System Volume Information\MountPointManagerRemoteDatabase
C:\Windows\system32\MSDtc\MSDTC.LOG
C:\Windows\system32\MSDtc\trace\dtctrace.log
%SystemRoot%\netlogon.chg
\hiberfil.sys
\System Volume Information\Heat\*.* /s
%TEMP%\* /s
\System Volume Information\*{3808876B-C176-4e48-B7AE-04046E6CC752} /s
\System Volume Information\*.{7cc467ef-6865-4831-853f-2a4817fd1bca}ALT
\System Volume Information\*.{7cc467ef-6865-4831-853f-2a4817fd1bca}DB
%ProgramData%\Microsoft\Windows\WER\* /s
%windir%\softwaredistribution\*.* /s
C:\ProgramData\Microsoft\Network\Downloader\*
C:\Windows\System32\Bits.log
C:\Windows\System32\Bits.bak
$AllVolumes$\System Volume Information\FVE.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}
$AllVolumes$\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}
$AllVolumes$\System Volume Information\FVE2.{9ef82dfa-1239-4a30-83e6-3b3e9b8fed08}.*
$UserProfile$\AppData\Local\Packages\Microsoft.Office.Desktop_8wekyb3d8bbwe\LocalCache\Local\Microsoft\Outlook\*.oab /s
$UserProfile$\AppData\Local\Packages\Microsoft.Office.Desktop_8wekyb3d8bbwe\LocalCache\Local\Microsoft\Outlook\*.ost /s
$UserProfile$\AppData\Local\Microsoft\Outlook\*.ost
\System Volume Information\Heat\*.* /s
%windir%\softwaredistribution\*.* /s
MountedDevices\
CurrentControlSet\Control\MSDTC\ASR\
CurrentControlSet\Control\Session Manager\PendingFileRenameOperations
CurrentControlSet\Control\Session Manager\PendingFileRenameOperations2
CurrentControlSet\Control\Session Manager\AllowProtectedRenames
PS C:\Users\crist>
I gotta say though, i am unsure if that list is of any use to duplicacy if entered “as is”, as i think it should be read at each run (eg. for each computer), and even then, all the system variables (%windir%) would have to be expanded at runtime.
For what it’s worth, I just ran the same PS command and got mostly the same - even the UIDs seem to match, but with some extras:
Text Compare
Produced: 15/07/2018 16:15:19
Mode: Differences
Left file: E:\temp\duplicacy_excludes\crist.txt Right file: E:\temp\duplicacy_excludes\myout.txt
1 %ProgramData%\Microsoft\Network\Downloader\* /s
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
27 C:\ProgramData\Microsoft\Network\Downloader\*
28 C:\Windows\System32\Bits.log
29 C:\Windows\System32\Bits.bak
---------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------
36 $UserProfile$\AppData\Local\Microsoft\Office\16.0\OfficeFileCache\*.fsf
37 $UserProfile$\AppData\Local\Microsoft\Office\16.0\OfficeFileCache\*.fsd
38 $UserProfile$\Local Settings\Application Data\Office\16.0\OfficeFileCache\*.fsd
39 $UserProfile$\Local Settings\Application Data\Office\16.0\OfficeFileCache\*.fsf
40 $UserProfile$\AppData\Local\Microsoft\Office\16.0\OfficeFileCache\LocalCacheFileEditManager\*.fsf
41 $UserProfile$\AppData\Local\Microsoft\Office\16.0\OfficeFileCache\LocalCacheFileEditManager\*.fsd
42 $UserProfile$\Local Settings\Application Data\Office\16.0\OfficeFileCache\LocalCacheFileEditManager\*.fsd
43 $UserProfile$\Local Settings\Application Data\Office\16.0\OfficeFileCache\LocalCacheFileEditManager\*.fsf
44 $UserProfile$\AppData\Local\Microsoft\Outlook\*.oab
---------------------------------------------------------------------------------------------------------------
Basically, lines 36-44 must be for my Office 2016 install. I have line 1 (non-expanded, with /s) instead of your line 27 (expanded, without /s) and I don’t have lines 28/29.
I think what might be nice, for the command line version of Duplicacy, is to have some kind of helper command - say duplicacy dump --filters - also implemented in the GUI, with a tickbox (“exclude known system files”).
It could extract the above, expand the paths and maybe even look to see if those files exist and not bother with them if they don’t (to keep the list short). If the filters file already exists, prepend the list, otherwise create one. Put a special line separator e.g. ===='s, so as not to show the entries above, in the GUI.
The GUI version could amend the list on-the-fly (so long as the tickbox is still ticked) and the command line user could simply re-run the command, both with some logic to not touch the filters file if there’s no change.
OK, really old - but this would be nice. Ticking the exclude standard Windows system files would be great. The NTUSER.DAT file is a mongrel of a thing and hidden by default i think.
In the WebUI, identifying things to exclude is pretty tedious. I had a quick look at the filters instructions and OMG! 
I did the search to find this topic because i wanted to know how to, with one filter, exclude all the directories, subdirectories and files in different locations but of the same directory name. e.g.
users\test1\3D Objects
users\test2\3D Objects
I think it would be like this:
-*\3D Objects\*
Is that right? Because i don’t want to accidentally delete a file that might be called 3D Objects Family Fun Day.jpg i can’t just use -*3D Objects*. Also, how do i account for the space in the name?
Sorry, the above is partially directly related and partly not.
This is correct. I would prefer -users\*\3D Objects\ though.