QNAP files restored to a new location are inaccessible, CLI 2.7.2

gorlen · 17 November 2020 21:48

Please describe what you are doing to trigger the bug:

Using QNAP qpkg for x86 w/ CLI version 2.7.2.
Restore directory to new location:

2020-11-17 12:59:27.039 INFO REPOSITORY_SET Repository set to /share/homes/Keith/Downloads/Restored
2020-11-17 12:59:27.039 INFO STORAGE_SET Storage set to /share/TR-004/Duplicacy/storage
2020-11-17 12:59:27.039 INFO SNAPSHOT_FILTER Loaded 1 include/exclude pattern(s)
2020-11-17 12:59:27.041 INFO RESTORE_INPLACE Forcing in-place mode with a non-default preference path
2020-11-17 12:59:30.306 INFO SNAPSHOT_FILTER Parsing filter file /share/CACHEDEV1_DATA/.qpkg/Duplicacy/.duplicacy-web/repositories/localhost/restore/.duplicacy/filters
2020-11-17 12:59:30.306 INFO SNAPSHOT_FILTER Loaded 0 include/exclude pattern(s)
2020-11-17 12:59:30.306 INFO RESTORE_START Restoring /share/homes/Keith/Downloads/Restored to revision 36
2020-11-17 12:59:30.632 INFO DOWNLOAD_PROGRESS Downloaded chunk 1 size 11270829, 10.75MB/s 00:00:01 100.0%
2020-11-17 12:59:30.660 INFO DOWNLOAD_DONE Downloaded Household Documents/Music/music appreciation class 2020/Mid Baroque composers fugue.docx (19034)
2020-11-17 12:59:30.660 INFO DOWNLOAD_DONE Downloaded Household Documents/Music/music appreciation class 2020/OLAD baroque elements.docx (22104)
2020-11-17 12:59:30.660 INFO RESTORE_END Restored /share/homes/Keith/Downloads/Restored to revision 36
2020-11-17 12:59:30.660 INFO RESTORE_STATS Files: 2 total, 40K bytes
2020-11-17 12:59:30.660 INFO RESTORE_STATS Downloaded 2 file, 40K bytes, 1 chunks
2020-11-17 12:59:30.660 INFO RESTORE_STATS Skipped 0 file, 0 bytes
2020-11-17 12:59:30.660 INFO RESTORE_STATS Total running time: 00:00:03

See https://forum.duplicacy.com/t/qnap-files-restored-to-a-new-location-are-inaccessible/4350/3?u=gorlen

Please describe what you expect to happen (but doesn’t):

Restored parent directories should have same permissions as backed-up directories.

Please describe what actually happens (the wrong behaviour):

Permissions of parent directories are incorrect, making restored files inaccessible:

gchen · 19 November 2020 04:48

Did you pass the -ignore-owner option to the restore command? Without this option Duplicacy should update the owner and group of each restored directory.

gorlen · 19 November 2020 17:58

The problem occurs both with and without the -ignore-owner option.

gorlen · 24 December 2020 19:04

Is a fix for this planned?

The Web UI doesn’t default the “Restore to” field to the preference path, so an incorrect path is prone to be entered (https://forum.duplicacy.com/t/restore-location-protection-default/4441/4?u=gorlen), then when files are restored to a non-default preference path, they are invisible to Windows clients because of this bug, causing confusion.

gorlen · 29 November 2021 17:59

Any progress on fixing this bug?

gchen · 1 December 2021 04:01

Did you test it with CLI 2.7.2?

I mentioned in this post that the ACL fix may have fixed this issue too.

gorlen · 1 December 2021 21:10

I just retested with CLI 2.7.2. There are two problems when restoring a folder to a new location:

Permissions are still not set correctly on the parent directories of the restored folder.
Permissions are not correct for the files/folders restored from old snapshots, presumably because these were backed up before the ACL fix. The following shows the differences between restoring an old snapshot and the most recent, both with -ignore-owner (RESTORE_INPLACE Forcing in-place mode with a non-default preference path):

gchen · 7 December 2021 03:25

I’ll look into this bug.

Did you mean files/folders backed up after the ACL fix are restored correctly?

gorlen · 7 December 2021 19:42

Better still, why even create these folders?

Yes.

On further thought, why should this be? If the CLI pre-2.7.2 did not back up ACLs correctly in old shapshots, and the files haven’t been modified since CLI 2.7.2 was released in Nov 2020, what would trigger the ACLs to be backed up again?

gchen · 11 December 2021 04:01

Restore always preserves the folder structure. The destination folder is treated as another repository, the same way as the source directory.

All extended attributes are read when indexing files regardless of whether they have been modified or not.

gorlen · 26 October 2022 18:56

Not fixed in CLI version 3.0.1.

gorlen · 22 January 2023 20:14

Any plans to fix this? Still not fixed in CLI 3.1.0.

Screenshot 2023-01-22 121249

gchen · 23 January 2023 16:33

Can you elaborate on what has not been fixed?

gorlen · 23 January 2023 18:11

From December '21 post:

Thanks!

gchen · 23 January 2023 19:41

You’ll need to add those parent directories to the Options box (for example, -- dir1/ dir1/dir2 dir1/dir2/dir3/) for the permissions to restored correctly. Otherwise the CLI won’t touch those directories and thus leave the permissions unchanged.

gorlen · 22 May 2023 15:55

Yes, manually adding parent directories causes correct permissions to be set, but shouldn’t the CLI do this automatically when creating these directories? Many directories may need to be specified.

jhs · 20 January 2024 12:53

I am trialing duplicacy using QNAP package (duplicacy_web_qnap_x86_64_1.7.2.qpkg with CLI version 3.2.3)

I have a strong feeling this issue persists.
When restoring files, the parent folder defaulted to admin:administrator permission (while the file is correctly allocated to the right permission)

My scenario, I have these simple folder structure:

source folder: /share/backblaze-test/forum-backup-restore-test/source
restore folder: /share/backblaze-test/forum-backup-restore-test/restore

Folder permissions for source folder:

[jhs@server forum-backup-restore-test]$ getfacl -R source/
# file: source/
# owner: jhs
# group: everyone
user::rwx
group::r-x
other::r-x

# file: source//folderbyjhs1
# owner: jhs
# group: everyone
user::rwx
group::r-x
other::r-x

# file: source//folderbyjhs1/folderbyjhs2
# owner: jhs
# group: everyone
user::rwx
group::r-x
other::r-x

# file: source//folderbyjhs1/folderbyjhs2/test.txt
# owner: jhs
# group: everyone
user::rw-
group::r--
other::r--

When I restored the backup, the following are the results (notice ALL parent folders permission updated to admin:administrators - while the restored file shows correct permission):

[/share/backblaze-test/forum-backup-restore-test] # getfacl -R restore
# file: restore
# owner: jhs
# group: everyone
user::rwx
group::r-x
other::r-x

# file: restore/.duplicacy
# owner: admin
# group: administrators
user::rwx
group::r--
other::r--

# file: restore/folderbyjhs1
# owner: admin
# group: administrators
user::rwx
group::r--
other::r--

# file: restore/folderbyjhs1/folderbyjhs2
# owner: admin
# group: administrators
user::rwx
group::r--
other::r--

# file: restore/folderbyjhs1/folderbyjhs2/test.txt
# owner: jhs
# group: everyone
user::rw-
group::r--
other::r--

Restore log:

2024-01-20 23:35:22.304 INFO REPOSITORY_SET Repository set to /share/backblaze-test/forum-backup-restore-test/restore
2024-01-20 23:35:22.304 INFO STORAGE_SET Storage set to /share/backblaze-test/duplicacy-test
2024-01-20 23:35:22.304 INFO SNAPSHOT_FILTER Loaded 1 include/exclude pattern(s)
2024-01-20 23:35:22.319 INFO RESTORE_INPLACE Forcing in-place mode with a non-default preference path
2024-01-20 23:35:22.319 INFO RESTORE_INDEXING Indexing /share/backblaze-test/forum-backup-restore-test/restore
2024-01-20 23:35:22.319 INFO SNAPSHOT_FILTER Parsing filter file /share/CE_CACHEDEV1_DATA/.qpkg/Duplicacy/.duplicacy-web/repositories/localhost/restore/.duplicacy/filters
2024-01-20 23:35:22.319 INFO SNAPSHOT_FILTER Loaded 0 include/exclude pattern(s)
2024-01-20 23:35:22.478 INFO RESTORE_START Restoring /share/backblaze-test/forum-backup-restore-test/restore to revision 1
2024-01-20 23:35:22.478 INFO DOWNLOAD_PROGRESS Downloaded chunk 1 size 5, 250B/s 00:00:01 5000.0%
2024-01-20 23:35:22.479 INFO DOWNLOAD_DONE Downloaded folderbyjhs1/folderbyjhs2/test.txt (5)
2024-01-20 23:35:22.479 INFO RESTORE_END Restored /share/backblaze-test/forum-backup-restore-test/restore to revision 1
2024-01-20 23:35:22.479 INFO RESTORE_STATS Files: 1 total, 5 bytes
2024-01-20 23:35:22.479 INFO RESTORE_STATS Downloaded 1 file, 5 bytes, 1 chunks
2024-01-20 23:35:22.479 INFO RESTORE_STATS Skipped 0 file, 0 bytes
2024-01-20 23:35:22.479 INFO RESTORE_STATS Total running time: 00:00:01

I also tried to restore the file with -ignore-owner => with this option, it updates ALL restored folders and file permission to admin:administrator

gorlen · 20 January 2024 17:58

GC doesn’t consider this to be a bug: https://forum.duplicacy.com/t/qnap-files-restored-to-a-new-location-are-inaccessible-cli-2-7-2/4490/15?u=gorlen.

Manually entering the list of parent directories in the Options box as described is inconvenient, so I usually either (1) temporarily rename the original location and restore to the original location or (2) restore to a new location, then ssh into the QNAP and run chown -R and chgrp -R on the new location.

(BTW, if you decide to use duplicacy on QNAP with the Web UI, you may be interested in this wrapper script to log duplicacy messages to the QNAP Notification Center and healthchecks.io.)