I’m testing the QNAP qpkg for x86.
Files restored to a new location (in this case /share/homes/Keith/Downloads/Restored or Restored-ignore-owner instead of /share/homes/Keith) are inaccessible because of the “mask” setting on parent directories. This occurs with or without the -ignore-owner option.
Restored folders inaccessable.txt (5.1 KB)
I’ll look into this issue and the other issue about ACL tomorrow.
This commit should fix the ACL issue. I’m not sure if the restored directory being inaccessible was caused by the same bug, since I couldn’t reproduce it on my QNAP, but this commit may fix that too.
You can build a new CLI binary from the master branch if you know how to do that, and replace the existing one under /share/MD0_DATA/.qpkg/Duplicacy/.duplicacy-web/bin (it may be something other than MD0_DATA on your QNAP). I can get you a new CLI binary if you want.
Thanks. I’d appreciate a new CLI binary, but no rush. I’m still testing before migrating from CrashPlan (which doesn’t back up ACLs at all, BTW).
Does this affect backup of ACLs, or just restores?
Note that I initialized the storage and made initial snapshots with CLI version 2.4.1 before switching to the Web Edition qpkg – do I need to start over with Web Edition?
I cross-compiled a new CLI binary on Windows from the master branch (go get -u github.com/gilbertchen/duplicacy/...) with GOOS=linux and repeated the restore to the original location, but the restored ACLs were no different. Also, the Hidden and Archive flags were not restored.
[/share/CACHEDEV1_DATA/.qpkg/Duplicacy/.duplicacy-web/logs] # cat restore-20201109-164930.log
2020-11-09 16:49:30.356 INFO REPOSITORY_SET Repository set to /share/homes/Keith
2020-11-09 16:49:30.356 INFO STORAGE_SET Storage set to /share/TR-004/Duplicacy/storage
2020-11-09 16:49:30.356 INFO SNAPSHOT_FILTER Loaded 1 include/exclude pattern(s)
2020-11-09 16:49:30.358 INFO RESTORE_INPLACE Forcing in-place mode with a non-default preference path
2020-11-09 16:49:46.344 INFO SNAPSHOT_FILTER Parsing filter file /share/CACHEDEV1_DATA/.qpkg/Duplicacy/.duplicacy-web/repositories/localhost/restore/.duplicacy/filters
2020-11-09 16:49:46.344 INFO SNAPSHOT_FILTER Loaded 0 include/exclude pattern(s)
2020-11-09 16:57:23.274 INFO RESTORE_START Restoring /share/homes/Keith to revision 6
2020-11-09 16:57:23.694 INFO DOWNLOAD_PROGRESS Downloaded chunk 1 size 8057568, 7.68MB/s 00:00:01 100.0%
2020-11-09 16:57:23.695 INFO DOWNLOAD_DONE Downloaded Downloads/DuplicacyMetadataTest/test.txt (247)
2020-11-09 16:57:23.695 INFO DOWNLOAD_DONE Downloaded Downloads/DuplicacyMetadataTest/Subfolder/test.txt (256)
2020-11-09 16:57:23.695 INFO RESTORE_END Restored /share/homes/Keith to revision 6
2020-11-09 16:57:23.695 INFO RESTORE_STATS Files: 2 total, 503 bytes
2020-11-09 16:57:23.695 INFO RESTORE_STATS Downloaded 2 file, 503 bytes, 1 chunks
2020-11-09 16:57:23.695 INFO RESTORE_STATS Skipped 0 file, 0 bytes
2020-11-09 16:57:23.695 INFO RESTORE_STATS Total running time: 00:07:53
It took nearly 8 minutes to restore two small files – is that normal?
You need to run a new backup again and restore from this new backup to get the correct extended attributes. The previous bug was caused by adding a user. prefix to all extended attributes when backing up files.
You can see from the log that most of the time was spent downloading and processing the file metadata. If you restore other files it will be much faster since all metadata chunks are already in the local cache.
On Windows these attributes are not stored during the backup.
The version of duplicacy_utils_others.go I compiled from the master branch did not contain the fix you referenced in your earlier post.
What will trigger a “new backup” of files? I presume that all files already in snapshots will not have all extended attributes backed up, and will not be seen as changed, so will not be automatically backed up again, correct?
Would running a backup with -hash trigger a “new backup” of existing files?
I suppose I can delete all existing snapshots and start over.
Also, the Hidden and Archive flags were not restored.
On Windows these attributes are not stored during the backup.
I’m running the QNAP qpkg, so on linux the user.DOSATTRIB extended file attributes SAMBA uses to store the DOS file attributes are not backed up, correct?
No need to specify -hash. A normal backup without -hash will read extended attributes for all files.
user.DOSATTRIB should be backed up if you run the backup on linux. You can try to back up a small set and then run duplicacy cat -r revision to see the attributes.
The test.txt file has user.DOSATTRIB:
[/share/homes/Keith/Downloads/DuplicacyMetadataTest] # getfattr -n user.DOSATTRIB -e "text" test.txt
# file: test.txt
user.DOSATTRIB="0x22
But the cat command does not show it:
[/share/CACHEDEV1_DATA/.qpkg/Duplicacy/.duplicacy-web/repositories/localhost/3/.duplicacy] # duplicacy cat -r 32 Downloads/DuplicacyMetadataTest/test.txt
Repository set to /share/homes/Keith
[/share/homes/Keith/Downloads/DuplicacyMetadataTest] # getfacl test.txt
# file: test.txt
# owner: Keith
# group: everyone
user::rwx
user:admin:rwx
user:Keith:rwx
user:backup:rwx
group::---
group:administrators:rwx
mask::rwx
other::---If you don’t give a file name to the cat command, it will print out the content of the snapshot file for that revision. The extended attributes are included in the output.
OK, user.DOSATTRIB backup/restore worked with 2.7.1.
What do I need to do to build a CLI version that includes the fix for the ACL backup/restore bug?