Files skipped on backup - Showing OPEN_FAILURE with "permission denied"

Support
#1

As stated, some files are getting skipped on backup showing a permission denied error (OPEN_FAILURE then SKIP_FILE). I’m backing up from a Synology to iDrive E2 using the Duplicacy web GUI.

I checked Permissions, and duplicacy has read/write permissions to the full directory. Also, what is weird is that if a folder has ~150 files, only ~35-40 files have this error. All others seem to be backed up properly.

This issue is also across multiple directories. There seems to be no logical reason for this. I have checked exclusion patterns as well.

Blockquote
Running backup command from /tmp/duplicacy/repositories/localhost/0 to back up /volume1/xxxxx
Options: [-log backup -storage XxxXxx-Photos -threads 4 -stats]
2024-04-04 08:01:21.082 INFO REPOSITORY_SET Repository set to /volume1/xxxxx
2024-04-04 08:01:21.082 INFO STORAGE_SET Storage set to …path…
2024-04-04 08:01:21.595 INFO BACKUP_START Last backup at revision 2 found
2024-04-04 08:01:22.789 INFO BACKUP_INDEXING Indexing /volume1/xxxxx
2024-04-04 08:01:22.789 INFO SNAPSHOT_FILTER Parsing filter file /tmp/duplicacy/repositories/localhost/0/.duplicacy/filters
2024-04-04 08:01:22.789 INFO SNAPSHOT_FILTER Loaded 4 include/exclude pattern(s)
2024-04-04 08:01:27.882 WARN OPEN_FAILURE Failed to open file for reading: open /volume1/…/…: permission denied
.
.
.
2024-04-04 08:01:29.212 WARN SKIP_FILE File …/… cannot be opened
2024-04-04 08:01:29.834 INFO BACKUP_END Backup for /volume1/xxxxx at revision 3 completed
2024-04-04 08:01:29.834 INFO BACKUP_STATS Files: 191278 total, 1252G bytes; 1576 new, -1576 bytes
2024-04-04 08:01:29.834 INFO BACKUP_STATS File chunks: 263414 total, 1260G bytes; 0 new, 0 bytes, 0 bytes uploaded
2024-04-04 08:01:29.834 INFO BACKUP_STATS Metadata chunks: 29 total, 45,183K bytes; 0 new, 0 bytes, 0 bytes uploaded
2024-04-04 08:01:29.834 INFO BACKUP_STATS All chunks: 263443 total, 1260G bytes; 0 new, 0 bytes, 0 bytes uploaded
2024-04-04 08:01:29.834 INFO BACKUP_STATS Total running time: 00:00:08
2024-04-04 08:01:29.834 WARN BACKUP_SKIPPED 1576 files were not included due to access errors

About ~1500 files are affected out of ~200k files total.

Any help is much appreciated!

Exclusions
Exclusions905×671 73.6 KB

#2

I would start with re-applying ownership recursively. If you use ACLs — it’s would be easier in Synology UI.

#3

OK so, I removed and reapplied permission for the entire shared folder via Synology UI. I also went down to some of the directories that failed to upload and checked permission to see if they have been recursively applied to all files and subfolders.

All this checks out.

Ran backup again.

The same 1576 files are again skipped. It’s always the exact same files for some reason.

Question:

  1. Does path length matter?

Permissions
Permissions864×596 72.2 KB

#4

Is it possible those files are open by some process? Can you copy this file manually elsewhere successfully?

Do all those skipped files have anything in common? Any special filenames?

I remember Synology had some abhorrent limit on maximum file name length on encrypted shares. Is this share encrypted?

You can try find /volume1/parh/ -maxdepth 20 | grep -P '\/[^\/]{143,}[^\/]' to identify those.

But I doubt it would matter — if the file already exists — it exists.

#5

Yes, I can manually copy these files across locations, tried for a few different ones. As for them being open by other process, I’m not sure. I can only say I’m not currently accessing them in any way.

Honestly, nothing I can see offhand. There are some with weird random long names like:

471077e7-9d70-4ced-87f3-683fc59e9292.jpg

Some others are

IMG_3223.MOV
2018-04-17_18-06-56_448.jpeg

But nothing special or unique.

No the share is not encrypted.

#6

Does it always stumble on the same exact files? or always different?

I’m wondering if this is synology indexer crawling the share and locking files for the duration of indexing.

#7

It’s always the same files. I tried thrice this morning, and it’s always those files. Checking on indexing shows no active jobs, and status is “completed”…

#8

That’s baffling.

Compare attributes, linux permissions, and ACLs on one of the failed file vs passing one.

I don’t have synology nearby by I believe you can see extended linux permissions with ls -a@ ... and ACLs with synoacltool -get ...

If you use ACLs you expect linux permissions to be 7777. If they are anything but – and I have seen synology arbitrarily breaking them – it may cause weird issues.

Otherwise I"m out of ideas.