Error using backup between MacOS and Arm Linux

GuyHall · 25 March 2018 21:38

Hi,
I am backing up repositories located on a NAS drive using a Raspberry Pi and I get a different behaviour when compared to MacOS for the same repository. Both are Autofs mounted, the Pi using CIFS and the Mac using AFP.

Below is the output from the commands. As you can see I aborted the Pi backup as was skipping chunks and suggesting several days to complete. The key difference is the following error:

Failed to list subdirectory: readdirent: not a director

I have a separate ‘pictures’ folder at the same level as ‘video’ and this works without skipping chunks on both operating systems. In both cases ‘pictures’ and ‘video’ were initialised from MacOS.

I’m thinking it could be something in the folders but I’m not sure what to look for.

MacOS

Hostname: /nas02/sharedHD/video $ ~/apps/duplicacy_osx_x64_2.0.10 backup -stats -threads 2 -limit-rate 500
Storage set to one://backup.video
Last backup at revision 7 found
Indexing /nas02/sharedHD/video
Loaded 2 include/exclude pattern(s)
Use 2 uploading threads
Uploaded chunk 1 size 950, 950B/s 00:00:01 100.0%
Uploaded .log.video/backup_20180325_184014 (950)
Backup for /nas02/sharedHD/video at revision 8 completed
Files: 9486 total, 257,989M bytes; 1 new, 950 bytes
File chunks: 52502 total, 257,990M bytes; 1 new, 950 bytes, 738 bytes uploaded
Metadata chunks: 4 total, 7,899K bytes; 3 new, 6,006K bytes, 3,695K bytes uploaded
All chunks: 52506 total, 257,998M bytes; 4 new, 6,007K bytes, 3,695K bytes uploaded
Total running time: 00:00:38
Hostname: /nas02/sharedHD/video $ 

Hostname: /nas02/sharedHD/video $ df
//username@nasDrive02.local/sharedHD     5774294480 1984448480 3789846000    35% 248056058           473730750   34%   /nas02/sharedHD

MacOS 10.13.3

Raspberry Pi

user@hostname /nas02/sharedHD/video $ ~/apps/duplicacy_linux_arm_2.0.10 backup -stats -threads 2 -limit-rate 500
Storage set to one://backup.video
Last backup at revision 8 found
Indexing /nas02/sharedHD/video
Loaded 2 include/exclude pattern(s)
Failed to list subdirectory: readdirent: not a directory
Failed to list subdirectory: readdirent: not a directory
Use 2 uploading threads
Skipped chunk 2 size 7076550, 691KB/s 12 days 21:00:26 0.0%
Skipped chunk 3 size 5265202, 1004KB/s 8 days 20:36:51 0.0%
Skipped chunk 1 size 13228941, 2.03MB/s 4 days 06:37:01 0.0%
Skipped chunk 4 size 1897981, 2.18MB/s 3 days 23:31:35 0.0%
Skipped chunk 5 size 1882045, 2.15MB/s 4 days 00:51:03 0.0%
Skipped chunk 6 size 2322282, 2.16MB/s 4 days 00:39:11 0.0%
Skipped chunk 7 size 8224174, 2.24MB/s 3 days 21:10:15 0.0%
Skipped chunk 8 size 2115802, 2.23MB/s 3 days 21:40:59 0.0%
Skipped chunk 9 size 4314561, 2.21MB/s 3 days 22:23:51 0.0%
Skipped chunk 10 size 7927847, 2.25MB/s 3 days 20:41:37 0.0%
Skipped chunk 11 size 9200281, 2.33MB/s 3 days 17:35:26 0.0%
Skipped chunk 12 size 1215618, 2.28MB/s 3 days 19:17:15 0.0%
Skipped chunk 13 size 5178897, 2.30MB/s 3 days 18:46:45 0.0%
Skipped chunk 14 size 7073684, 2.29MB/s 3 days 18:57:29 0.0%
^Cuser@hostname /nas02/sharedHD/video $ df
//nasdrive02.local/sharedHD 209715200  37915092 171800108  19% /nas02/sharedHD

user@hostname /nas02/sharedHD/video $ uname -a
Linux pi1 4.1.19-v7+ #858 SMP Tue Mar 15 15:56:00 GMT 2016 armv7l GNU/Linux
(Raspberry Pi 2)

user@hostname /nas02/sharedHD/video $ ~/apps/duplicacy_linux_arm_2.0.10 -d check -r 8
Storage set to one://backup.video
Reading the environment variable DUPLICACY_ONE_TOKEN
Reading one_token from preferences
GET https://api.onedrive.com/v1.0/drive/root:/backup.video?select=id,name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks?select=id,name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/fossils?select=id,name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/snapshots?select=id,name,size,folder
Reading the environment variable DUPLICACY_ONE_TOKEN
Reading one_token from preferences
Reading the environment variable DUPLICACY_PASSWORD
Reading password from preferences
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/config?select=id,name,size,folder
GET https://api.onedrive.com/v1.0/drive/items/root:/backup.video/config:/content
Using 16384 iterations for key derivation
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/nesting?select=id,name,size,folder
Chunk read levels: [1], write level: 1
Compression level: 100
Average chunk size: 4194304
Maximum chunk size: 16777216
Minimum chunk size: 1048576
Chunk seed: c0519b8187d76432caaa3e58ffa0d0d260f454579747829a4dcc8b758ca1fb2f
Reading the environment variable DUPLICACY_PASSWORD
Reading password from preferences
id: video, revisions: [8], tag: , showStatistics: false, checkFiles: false, searchFossils: false, resurrect: false
Listing all chunks
Listing chunks/
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/fossils:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/00:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/0a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/0b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/0c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/0d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/0e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/0f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/01:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/1a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/1b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/1c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/1d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/1e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/1f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/02:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/2a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/2b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/2c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/2d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/2e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/2f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/03:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/3a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/3b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/3c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/3d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/3e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/3f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/04:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/4a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/4b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/4c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/4d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/4e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/4f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/05:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/5a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/5b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/5c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/5d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/5e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/5f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/06:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/6a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/6b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/6c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/6d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/6e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/6f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/07:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/7a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/7b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/7c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/7d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/7e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/7f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/08:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/8a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/8b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/8c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/8d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/8e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/8f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/09:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/9a:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/9b:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/9c:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/9d:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/9e:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/9f:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/10:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/11:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/12:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/13:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/14:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/15:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/16:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/17:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/18:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/19:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/20:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/21:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/22:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/23:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/24:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/25:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/26:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/27:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/28:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/29:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/30:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/31:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/32:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/33:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/34:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/35:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/36:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/37:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/38:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/39:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/40:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/41:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/42:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/43:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/44:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/45:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/46:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/47:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/48:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/49:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/50:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/51:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/52:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/53:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/54:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/55:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/56:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/57:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/58:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/59:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/60:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/61:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/62:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/63:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/64:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/65:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/66:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/67:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/68:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/69:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/70:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/71:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/72:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/73:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/74:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/75:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/76:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/77:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/78:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/79:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/80:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/81:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/82:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/83:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/84:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/85:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/86:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/87:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/88:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/89:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/90:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/91:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/92:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/93:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/94:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/95:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/96:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/97:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/98:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/99:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a0:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a1:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a2:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a3:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a4:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a5:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a6:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a7:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a8:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/a9:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/aa:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ab:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ac:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ad:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ae:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/af:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b0:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b1:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b2:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b3:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b4:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b5:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b6:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b7:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b8:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/b9:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ba:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/bb:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/bc:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/bd:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/be:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/bf:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c0:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c1:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c2:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c3:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c4:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c5:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c6:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c7:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c8:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/c9:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ca:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/cb:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/cc:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/cd:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ce:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/cf:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d0:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d1:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d2:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d3:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d4:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d5:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d6:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d7:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d8:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/d9:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/da:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/db:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/dc:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/dd:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/de:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/df:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e0:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e1:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e2:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e3:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e4:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e5:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e6:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e7:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e8:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/e9:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ea:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/eb:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ec:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ed:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ee:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ef:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f0:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f1:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f2:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f3:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f4:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f5:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f6:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f7:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f8:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/f9:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/fa:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/fb:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/fc:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/fd:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/fe:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/chunks/ff:/children?top=1000&select=name,size,folder
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/snapshots?select=id,name,size,folder
POST https://api.onedrive.com/v1.0/drive/items/9C448783DD0C4A72!282280/children
GET https://api.onedrive.com/v1.0/drive/root:/backup.video/snapshots/video/8?select=id,name,size,folder
Loaded file snapshots/video/8 from the snapshot cache
Chunk f611aabd0a9f44cb27d8cab25352340bd20dec62246db5e63812f0c3d1614ed1 has been loaded from the snapshot cache
All chunks referenced by snapshot video at revision 8 exist

gchen · 26 March 2018 01:59

If you run duplicacy -d backup it will show which subdirectories cause the listing errors.

GuyHall · 26 March 2018 20:51

Perfect gchen, found it.

An odd hidden file/link in the structure of Apple’s iMove library that points to the directory it is in and therefore concatenates and lists everything in the folder many, many times:

8 lrwxrwxrwx  1 user  wheel       1 11 Nov 12:51 .fcpcache -> .

[Linux fragment]
. . .

Listing imovie/
imovie/.DS_Store is excluded
Listing imovie/iMovie Library movielibrary2.imovielibrary/
Listing imovie/iMovie Library 2.imovielibrary/.fcpcache/
Listing imovie/iMovie Library 2.imovielibrary/.fcpcache/.fcpcache/
Listing imovie/iMovie Library 2.imovielibrary/.fcpcache/.fcpcache/.fcpcache/
Listing imovie/iMovie Library 2.imovielibrary/.fcpcache/.fcpcache/.fcpcache/.fcpcache/

etc. until:

Listing imovie/iMovie Library 2.imovielibrary/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/.fcpcache/
Failed to list subdirectory: readdirent: not a directory

[MacOS fragment]

Listing imovie/
imovie/.DS_Store is excluded
Listing imovie/iMovie Library 2.imovielibrary/
Listing imovie/iMovie Library 2.imovielibrary/.lock-dir/
Listing imovie/iMovie Library 2.imovielibrary/2015.08.03 Elephants/

For some reason the underlying MacOS command duplicacy calls to list folders and files does not list the .fcpcache link file or a couple of other files:

  0 drwxrwxrwx  1 user  wheel     264 11 Mar 09:39 .lock-dir
  8 -rw-rw-rw-  1 user  wheel     351 11 Mar 09:39 .lock-info
  8 -rwxrwxrwx  1 user  wheel      20 20 Mar  2016 .lockinfo

Don’t worry about looking into this further, what I’m doing is quite an edge case as iMovie is a MacOS application and therefore unlikely to be backed up by a Linux OS so I’m going to see if i can add it as an exclude filter for now while I work out if not having it breaks iMovie if I have to restore.

GuyHall · 26 March 2018 21:00

-*.fcpcache*

In the filters file lets the backup run clean in the unlikely event that someone else is trying to do what I’m doing.