Onedrive For business timeout or something else?

[danny1]

I have found a few forum posts describing the same issue (kind off) but it all appears to have fixed itself ? below is the error log it looks like to speed is decreasing after a few hours or something. the source folder is around 300gb. can someone please advice ? the backup runned 4 times a day before without issue, since the issue I have decreased to 2 times a day but it still does not manage to backup succesfully.

Back up /backup@localhost to storage Onedrive-Noreply, status: Aborted, log file: backup-20211116-000001.log

Running backup command from /cache/localhost/0 to back up /backup
Options: [-log backup -storage Onedrive-Noreply -threads 4 -stats]
2021-11-16 00:00:02.108 INFO REPOSITORY_SET Repository set to /backup
2021-11-16 00:00:02.122 INFO STORAGE_SET Storage set to odb://Backup-Duplicacy
2021-11-16 00:00:09.305 INFO BACKUP_START Last backup at revision 13 found
2021-11-16 00:00:09.305 INFO BACKUP_INDEXING Indexing /backup
2021-11-16 00:00:09.305 INFO SNAPSHOT_FILTER Parsing filter file /cache/localhost/0/.duplicacy/filters
2021-11-16 00:00:09.306 INFO SNAPSHOT_FILTER Loaded 2 include/exclude pattern(s)
2021-11-16 00:02:16.911 WARN LIST_SKIP Skipped non-regular file appdata/unifi-controller/run/mongodb-27117.sock
2021-11-16 00:02:17.044 INFO BACKUP_THREADS Use 4 uploading threads
2021-11-16 00:04:34.516 INFO ONEDRIVE_RETRY Response code: 429; retry after 57000 milliseconds
2021-11-16 00:04:34.559 INFO ONEDRIVE_RETRY Response code: 429; retry after 57000 milliseconds
2021-11-16 00:04:34.956 INFO ONEDRIVE_RETRY Response code: 429; retry after 35000 milliseconds
2021-11-16 00:04:36.263 INFO ONEDRIVE_RETRY Response code: 429; retry after 45000 milliseconds
2021-11-16 00:05:10.477 INFO ONEDRIVE_RETRY Response code: 429; retry after 319000 milliseconds
2021-11-16 00:05:23.704 INFO ONEDRIVE_RETRY Response code: 429; retry after 47000 milliseconds
2021-11-16 00:05:32.684 INFO ONEDRIVE_RETRY Response code: 429; retry after 327000 milliseconds
2021-11-16 00:05:32.741 INFO ONEDRIVE_RETRY Response code: 429; retry after 319000 milliseconds
2021-11-16 00:06:11.131 INFO ONEDRIVE_RETRY Response code: 429; retry after 304000 milliseconds
2021-11-16 00:10:30.186 INFO ONEDRIVE_RETRY Response code: 429; retry after 306000 milliseconds
2021-11-16 00:10:53.576 INFO ONEDRIVE_RETRY Response code: 429; retry after 300000 milliseconds
2021-11-16 00:11:00.550 INFO ONEDRIVE_RETRY Response code: 429; retry after 328000 milliseconds
2021-11-16 00:11:16.869 INFO ONEDRIVE_RETRY Response code: 429; retry after 313000 milliseconds
2021-11-16 00:15:49.346 INFO ONEDRIVE_RETRY Response code: 429; retry after 300000 milliseconds
2021-11-16 00:19:35.261 INFO ONEDRIVE_RETRY Response code: 429; retry after 37000 milliseconds
2021-11-16 00:19:35.424 INFO ONEDRIVE_RETRY Response code: 429; retry after 53000 milliseconds
2021-11-16 00:19:39.870 INFO ONEDRIVE_RETRY Response code: 429; retry after 46000 milliseconds
2021-11-16 00:20:16.116 INFO ONEDRIVE_RETRY Response code: 429; retry after 32000 milliseconds
2021-11-16 00:20:26.750 INFO ONEDRIVE_RETRY Response code: 429; retry after 37000 milliseconds
2021-11-16 00:20:29.018 INFO ONEDRIVE_RETRY Response code: 429; retry after 43000 milliseconds
2021-11-16 00:20:48.500 INFO ONEDRIVE_RETRY Response code: 429; retry after 327000 milliseconds
2021-11-16 00:20:50.404 INFO ONEDRIVE_RETRY Response code: 429; retry after 310000 milliseconds
2021-11-16 00:21:04.346 INFO ONEDRIVE_RETRY Response code: 429; retry after 309000 milliseconds
2021-11-16 00:21:12.907 INFO ONEDRIVE_RETRY Response code: 429; retry after 314000 milliseconds
2021-11-16 00:29:32.613 INFO ONEDRIVE_RETRY Response code: 429; retry after 57000 milliseconds
2021-11-16 00:29:34.365 INFO ONEDRIVE_RETRY Response code: 429; retry after 46000 milliseconds
2021-11-16 00:29:35.513 INFO ONEDRIVE_RETRY Response code: 429; retry after 48000 milliseconds
2021-11-16 00:29:37.832 INFO ONEDRIVE_RETRY Response code: 429; retry after 51000 milliseconds
2021-11-16 00:30:22.421 INFO ONEDRIVE_RETRY Response code: 429; retry after 303000 milliseconds
2021-11-16 00:30:23.817 INFO ONEDRIVE_RETRY Response code: 429; retry after 286000 milliseconds
2021-11-16 00:30:29.875 INFO ONEDRIVE_RETRY Response code: 429; retry after 302000 milliseconds
2021-11-16 00:30:30.419 INFO ONEDRIVE_RETRY Response code: 429; retry after 309000 milliseconds
2021-11-16 00:35:17.650 INFO ONEDRIVE_RETRY Response code: 429; retry after 317000 milliseconds
2021-11-16 00:35:29.961 INFO ONEDRIVE_RETRY Response code: 429; retry after 304000 milliseconds
2021-11-16 00:35:39.641 INFO ONEDRIVE_RETRY Response code: 429; retry after 305000 milliseconds
2021-11-16 00:35:43.773 INFO ONEDRIVE_RETRY Response code: 429; retry after 310000 milliseconds
2021-11-16 00:40:37.976 INFO ONEDRIVE_RETRY Response code: 429; retry after 300000 milliseconds
2021-11-16 00:40:46.061 INFO ONEDRIVE_RETRY Response code: 429; retry after 305000 milliseconds
2021-11-16 00:47:01.850 INFO ONEDRIVE_RETRY Response code: 429; retry after 38000 milliseconds
2021-11-16 00:47:02.117 INFO ONEDRIVE_RETRY Response code: 429; retry after 49000 milliseconds
2021-11-16 00:47:02.414 INFO ONEDRIVE_RETRY Response code: 429; retry after 50000 milliseconds
2021-11-16 00:47:04.371 INFO ONEDRIVE_RETRY Response code: 429; retry after 37000 milliseconds
2021-11-16 00:47:40.812 INFO ONEDRIVE_RETRY Response code: 429; retry after 317000 milliseconds
2021-11-16 00:47:42.859 INFO ONEDRIVE_RETRY Response code: 429; retry after 54000 milliseconds
2021-11-16 00:47:51.617 INFO ONEDRIVE_RETRY Response code: 429; retry after 320000 milliseconds
2021-11-16 00:47:52.510 INFO ONEDRIVE_RETRY Response code: 429; retry after 305000 milliseconds
2021-11-16 00:48:37.119 INFO ONEDRIVE_RETRY Response code: 429; retry after 309000 milliseconds
2021-11-16 00:56:34.128 INFO ONEDRIVE_RETRY Response code: 429; retry after 54000 milliseconds
2021-11-16 00:56:34.289 INFO ONEDRIVE_RETRY Response code: 429; retry after 54000 milliseconds
2021-11-16 00:56:34.971 INFO ONEDRIVE_RETRY Response code: 429; retry after 42000 milliseconds
2021-11-16 00:56:35.674 INFO ONEDRIVE_RETRY Response code: 429; retry after 58000 milliseconds
2021-11-16 00:57:18.673 INFO ONEDRIVE_RETRY Response code: 429; retry after 319000 milliseconds
2021-11-16 00:57:28.368 INFO ONEDRIVE_RETRY Response code: 429; retry after 59000 milliseconds
2021-11-16 00:57:28.628 INFO ONEDRIVE_RETRY Response code: 429; retry after 58000 milliseconds
2021-11-16 00:57:35.713 INFO ONEDRIVE_RETRY Response code: 429; retry after 266000 milliseconds
2021-11-16 00:58:26.754 INFO ONEDRIVE_RETRY Response code: 429; retry after 325000 milliseconds
2021-11-16 00:58:28.352 INFO ONEDRIVE_RETRY Response code: 429; retry after 319000 milliseconds
2021-11-16 01:06:08.575 INFO ONEDRIVE_RETRY Response code: 429; retry after 32000 milliseconds
2021-11-16 01:06:09.477 INFO ONEDRIVE_RETRY Response code: 429; retry after 46000 milliseconds
2021-11-16 01:06:11.217 INFO ONEDRIVE_RETRY Response code: 429; retry after 30000 milliseconds
2021-11-16 01:06:11.291 INFO ONEDRIVE_RETRY Response code: 429; retry after 51000 milliseconds
2021-11-16 01:06:42.257 INFO ONEDRIVE_RETRY Response code: 429; retry after 318000 milliseconds
2021-11-16 01:06:44.022 INFO ONEDRIVE_RETRY Response code: 429; retry after 34000 milliseconds
2021-11-16 01:06:56.783 INFO ONEDRIVE_RETRY Response code: 429; retry after 315000 milliseconds
2021-11-16 01:07:04.702 INFO ONEDRIVE_RETRY Response code: 429; retry after 309000 milliseconds
2021-11-16 01:07:19.294 INFO ONEDRIVE_RETRY Response code: 429; retry after 312000 milliseconds
2021-11-16 01:12:01.407 INFO ONEDRIVE_RETRY Response code: 429; retry after 306000 milliseconds
2021-11-16 01:12:11.869 INFO ONEDRIVE_RETRY Response code: 429; retry after 321000 milliseconds
2021-11-16 01:12:13.960 INFO ONEDRIVE_RETRY Response code: 429; retry after 317000 milliseconds
2021-11-16 01:12:31.687 INFO ONEDRIVE_RETRY Response code: 429; retry after 311000 milliseconds
2021-11-16 01:20:35.779 INFO ONEDRIVE_RETRY Response code: 429; retry after 49000 milliseconds
2021-11-16 01:20:36.054 INFO ONEDRIVE_RETRY Response code: 429; retry after 36000 milliseconds
2021-11-16 01:20:38.179 INFO ONEDRIVE_RETRY Response code: 429; retry after 41000 milliseconds
2021-11-16 01:20:38.652 INFO ONEDRIVE_RETRY Response code: 429; retry after 38000 milliseconds
2021-11-16 01:21:12.221 INFO ONEDRIVE_RETRY Response code: 429; retry after 300000 milliseconds
2021-11-16 01:21:17.136 INFO ONEDRIVE_RETRY Response code: 429; retry after 311000 milliseconds
2021-11-16 01:21:19.783 INFO ONEDRIVE_RETRY Response code: 429; retry after 59000 milliseconds
2021-11-16 01:21:24.885 INFO ONEDRIVE_RETRY Response code: 429; retry after 37000 milliseconds
2021-11-16 01:22:03.615 INFO ONEDRIVE_RETRY Response code: 429; retry after 309000 milliseconds
2021-11-16 01:22:19.245 INFO ONEDRIVE_RETRY Response code: 429; retry after 307000 milliseconds
2021-11-16 01:26:12.352 INFO ONEDRIVE_RETRY Response code: 429; retry after 310000 milliseconds
2021-11-16 01:26:28.685 INFO ONEDRIVE_RETRY Response code: 429; retry after 324000 milliseconds
2021-11-16 01:27:12.717 INFO ONEDRIVE_RETRY Response code: 429; retry after 325000 milliseconds
2021-11-16 01:27:28.012 INFO ONEDRIVE_RETRY Response code: 429; retry after 320000 milliseconds
2021-11-16 01:37:36.835 INFO ONEDRIVE_RETRY Response code: 429; retry after 37000 milliseconds
2021-11-16 01:37:36.856 INFO ONEDRIVE_RETRY Response code: 429; retry after 48000 milliseconds
2021-11-16 01:37:37.892 INFO ONEDRIVE_RETRY Response code: 429; retry after 57000 milliseconds
2021-11-16 01:37:45.170 INFO ONEDRIVE_RETRY Response code: 429; retry after 48000 milliseconds
2021-11-16 01:39:02.567 INFO ONEDRIVE_RETRY Response code: 429; retry after 31000 milliseconds
2021-11-16 01:39:06.803 INFO ONEDRIVE_RETRY Response code: 429; retry after 53000 milliseconds
2021-11-16 01:39:07.737 INFO ONEDRIVE_RETRY Response code: 429; retry after 47000 milliseconds
2021-11-16 01:39:16.252 INFO ONEDRIVE_RETRY Response code: 429; retry after 57000 milliseconds
2021-11-16 01:39:37.857 INFO ONEDRIVE_RETRY Response code: 429; retry after 305000 milliseconds
2021-11-16 01:39:56.606 INFO ONEDRIVE_RETRY Response code: 429; retry after 314000 milliseconds
2021-11-16 01:40:01.329 INFO ONEDRIVE_RETRY Response code: 429; retry after 304000 milliseconds
2021-11-16 01:40:16.830 INFO ONEDRIVE_RETRY Response code: 429; retry after 326000 milliseconds
2021-11-16 01:44:52.990 INFO ONEDRIVE_RETRY Response code: 429; retry after 300000 milliseconds
2021-11-16 01:56:17.998 INFO ONEDRIVE_RETRY Response code: 429; retry after 31000 milliseconds
2021-11-16 01:56:20.735 INFO ONEDRIVE_RETRY Response code: 429; retry after 45000 milliseconds
2021-11-16 01:56:21.349 INFO ONEDRIVE_RETRY Response code: 429; retry after 45000 milliseconds
2021-11-16 01:56:21.699 INFO ONEDRIVE_RETRY Response code: 429; retry after 36000 milliseconds
2021-11-16 01:56:50.588 INFO ONEDRIVE_RETRY Response code: 429; retry after 318000 milliseconds
2021-11-16 01:57:00.778 INFO ONEDRIVE_RETRY Response code: 429; retry after 308000 milliseconds
2021-11-16 01:57:05.839 INFO ONEDRIVE_RETRY Response code: 429; retry after 309000 milliseconds
2021-11-16 01:57:07.322 INFO ONEDRIVE_RETRY Response code: 429; retry after 309000 milliseconds
2021-11-16 02:10:34.307 INFO ONEDRIVE_RETRY Response code: 429; retry after 34000 milliseconds
2021-11-16 02:10:35.587 INFO ONEDRIVE_RETRY Response code: 429; retry after 56000 milliseconds
2021-11-16 02:10:37.622 INFO ONEDRIVE_RETRY Response code: 429; retry after 35000 milliseconds
2021-11-16 02:10:38.884 INFO ONEDRIVE_RETRY Response code: 429; retry after 42000 milliseconds
2021-11-16 02:11:08.414 INFO ONEDRIVE_RETRY Response code: 429; retry after 319000 milliseconds
2021-11-16 02:11:12.991 INFO ONEDRIVE_RETRY Response code: 429; retry after 45000 milliseconds
2021-11-16 02:11:21.322 INFO ONEDRIVE_RETRY Response code: 429; retry after 315000 milliseconds
2021-11-16 02:11:34.067 INFO ONEDRIVE_RETRY Response code: 429; retry after 34000 milliseconds
2021-11-16 02:11:58.105 INFO ONEDRIVE_RETRY Response code: 429; retry after 240000 milliseconds
2021-11-16 02:12:11.558 INFO ONEDRIVE_RETRY Response code: 429; retry after 328000 milliseconds
2021-11-16 02:21:06.823 INFO ONEDRIVE_RETRY Response code: 429; retry after 48000 milliseconds
2021-11-16 02:21:07.936 INFO ONEDRIVE_RETRY Response code: 429; retry after 37000 milliseconds
2021-11-16 02:21:10.088 INFO ONEDRIVE_RETRY Response code: 429; retry after 44000 milliseconds
2021-11-16 02:21:10.656 INFO ONEDRIVE_RETRY Response code: 429; retry after 43000 milliseconds
2021-11-16 02:21:46.713 INFO ONEDRIVE_RETRY Response code: 429; retry after 324000 milliseconds
2021-11-16 02:21:53.920 INFO ONEDRIVE_RETRY Response code: 429; retry after 33000 milliseconds
2021-11-16 02:21:55.067 INFO ONEDRIVE_RETRY Response code: 429; retry after 315000 milliseconds
2021-11-16 02:21:55.509 INFO ONEDRIVE_RETRY Response code: 429; retry after 52000 milliseconds
2021-11-16 02:22:27.068 INFO ONEDRIVE_RETRY Response code: 429; retry after 320000 milliseconds
2021-11-16 02:22:47.820 INFO ONEDRIVE_RETRY Response code: 429; retry after 316000 milliseconds
2021-11-16 02:27:10.184 INFO ONEDRIVE_RETRY Response code: 429; retry after 327000 milliseconds
2021-11-16 02:27:10.833 INFO ONEDRIVE_RETRY Response code: 429; retry after 300000 milliseconds
2021-11-16 02:27:48.261 INFO ONEDRIVE_RETRY Response code: 429; retry after 313000 milliseconds
2021-11-16 02:28:04.358 INFO ONEDRIVE_RETRY Response code: 429; retry after 321000 milliseconds
2021-11-16 02:32:48.948 INFO ONEDRIVE_RETRY Response code: 429; retry after 304000 milliseconds
2021-11-16 02:36:12.094 INFO ONEDRIVE_RETRY Response code: 429; retry after 41000 milliseconds
2021-11-16 02:36:12.380 INFO ONEDRIVE_RETRY Response code: 429; retry after 46000 milliseconds
2021-11-16 02:36:17.595 INFO ONEDRIVE_RETRY Response code: 429; retry after 57000 milliseconds
2021-11-16 02:36:56.380 INFO ONEDRIVE_RETRY Response code: 429; retry after 313000 milliseconds
2021-11-16 02:36:59.037 INFO ONEDRIVE_RETRY Response code: 429; retry after 44000 milliseconds
2021-11-16 02:37:14.989 INFO ONEDRIVE_RETRY Response code: 429; retry after 321000 milliseconds
2021-11-16 02:37:44.474 INFO ONEDRIVE_RETRY Response code: 429; retry after 316000 milliseconds
2021-11-16 02:37:53.057 INFO ONEDRIVE_RETRY Response code: 429; retry after 323000 milliseconds
2021-11-16 02:42:09.821 INFO ONEDRIVE_RETRY Response code: 429; retry after 323000 milliseconds
2021-11-16 02:42:36.252 INFO ONEDRIVE_RETRY Response code: 429; retry after 303000 milliseconds
2021-11-16 02:43:27.715 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/profiles/eba06c58-dc85-4c93-b923-6cba88bda70d/20211108_190001.zip: no such file or directory
2021-11-16 02:43:27.715 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/profiles/eba06c58-dc85-4c93-b923-6cba88bda70d/20211108_203008.zip: no such file or directory
2021-11-16 02:43:27.715 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/profiles/eba06c58-dc85-4c93-b923-6cba88bda70d/20211109_010001.zip: no such file or directory
2021-11-16 02:43:27.792 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/servers/eba06c58-dc85-4c93-b923-6cba88bda70d/theisland_20211101_190001.zip: no such file or directory
2021-11-16 02:43:27.792 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/servers/eba06c58-dc85-4c93-b923-6cba88bda70d/theisland_20211101_200001.zip: no such file or directory
2021-11-16 02:43:27.792 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/servers/eba06c58-dc85-4c93-b923-6cba88bda70d/theisland_20211101_210001.zip: no such file or directory
2021-11-16 02:43:27.792 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/servers/eba06c58-dc85-4c93-b923-6cba88bda70d/theisland_20211101_220001.zip: no such file or directory
2021-11-16 02:43:27.792 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/servers/eba06c58-dc85-4c93-b923-6cba88bda70d/theisland_20211101_230001.zip: no such file or directory
2021-11-16 02:43:27.792 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Ark-Backups/servers/eba06c58-dc85-4c93-b923-6cba88bda70d/theisland_20211102_000001.zip: no such file or directory
2021-11-16 02:43:47.175 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/1-Maatjes/2021-11-01_23-50-56.zip: no such file or directory
2021-11-16 02:43:47.175 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/1-Maatjes/2021-11-02_00-51-01.zip: no such file or directory
2021-11-16 04:28:55.960 INFO ONEDRIVE_RETRY Response code: 503; retry after 820 milliseconds
Duplicacy was aborted

[saspus]

@danny1

2021-11-16 00:02:17.044 INFO BACKUP_THREADS Use 4 uploading threads
2021-11-16 00:04:34.516 INFO ONEDRIVE_RETRY Response code: 429; retry after 57000 milliseconds

429 is “too many requests”. Reduce number of threads to 1.

[danny1]

i changed to 1 thread this is the logs from tonight ( I dedacted a lot because it was to big and I am not allowed to upload the logs as a file because I am a new user.

Back up /backup@localhost to storage Onedrive-Noreply, status: Aborted, log file: backup-20211116-200519.log

Running backup command from /cache/localhost/0 to back up /backup
Options: [-log backup -storage Onedrive-Noreply -threads 1 -stats]
2021-11-16 20:05:19.445 INFO REPOSITORY_SET Repository set to /backup
2021-11-16 20:05:19.445 INFO STORAGE_SET Storage set to odb://Backup-Duplicacy
2021-11-16 20:05:23.425 INFO BACKUP_START Last backup at revision 13 found
2021-11-16 20:05:23.425 INFO BACKUP_INDEXING Indexing /backup
2021-11-16 20:05:23.425 INFO SNAPSHOT_FILTER Parsing filter file /cache/localhost/0/.duplicacy/filters
2021-11-16 20:05:23.426 INFO SNAPSHOT_FILTER Loaded 2 include/exclude pattern(s)
2021-11-16 20:07:19.594 WARN LIST_SKIP Skipped non-regular file appdata/unifi-controller/run/mongodb-27117.sock
2021-11-16 22:52:10.111 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/1-Maatjes/2021-11-02_19-52-34.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-14_04-59-52.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-14_11-00-16.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-14_17-00-36.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-14_23-00-57.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-15_10-42-55.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-15_16-43-14.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-15_22-43-37.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-16_04-43-56.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-16_10-44-14.zip: no such file or directory
2021-11-16 22:52:10.119 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Minecraft/2-Maatjes-Survival/2021-11-16_16-44-35.zip: no such file or directory
2021-11-16 22:53:30.865 WARN OPEN_FAILURE Failed to open file for reading: open /backup/Backup-Star-Destroyer/Pterodactyl/f0fb3d36-beef-46b4-8601-82f1541c2466.tar.gz: no such file or directory
2021-11-17 00:45:06.005 INFO ONEDRIVE_RETRY Response code: 504; retry after 736 milliseconds
2021-11-17 01:24:06.357 INFO ONEDRIVE_RETRY Response code: 502; retry after 751 milliseconds
Duplicacy was aborted

[gchen]

This is not an OneDrive issue. Those retry messages were normal and Duplicacy successfully recovered from 50x response codes since there were no subsequence errors.

@danny1

Duplicacy was aborted

This means the CLI was likely killed due to out-of-memory. Check ~/.duplicacy-web/logs/duplicacy_web.log and you should find more details there.

[saspus]

You can add ``` before and after blocks of text to format it for better readability. Or select the text and press </> on the toolbar in the editor.

@danny1

-log backup -storage Onedrive-Noreply -threads 1 -stats

I would also add -vss, see below

@danny1

Skipped non-regular file appdata/unifi-controller/run/mongodb-27117.sock

It seems you are trying to backup mongo db while it’s running. This likely will result in it being in the inconsistent state in backup. Skip the database (and any other data for that matter from there, except config folder) and instead use UniFi controller’s periodic backup feature. (It does it by exporting data from the db atomically)

@danny1

open /backup/Backup-Star-Destroyer/Minecraft/1-Maatjes/2021-11-02_19-52-34.zip: no such file or directory

Either the user duplicacy is running under does not have access to that file, or it was removed between the point backup started and that file getting picked up. To avoid ending up with inconsistent backup revision where files are picked up at random time in random state you should take snapshot of the filesystem and backup that snapshot. Duplicacy will do that if you pass -vss flag to backup command. It is supported on macOS and Windows. On other OS-es you can use pre-backup and post-backup scripts to accomplish that manually.

@danny1

Response code: 504; retry after 736 milliseconds

This is a “Gateway timeout”, intermittent network issue, that got resolved by the time duplicacy retried.

@danny1

Response code: 502; retry after 751 milliseconds

This is “Bad Gateway” – another issue on the network side a minute later, also resolved

@danny1

Duplicacy was aborted

If it ran out of memory you can reduce memory requirements somewhat by setting DUPLICACY_ATTRIBUTE_THRESHOLD=1 environment variable.

What appliance is this, how much ram is there, and how many files are in the bakup set?

[danny1]

no, this is my own mistake i set a time limit in the schedule I removed it yesterday and the backup i started is still going now. ill guess we’ll need to be a bit more patient.

[danny1]

as soon as this attempt is either failed or succeeded ill add the -vss flag, what does this do ?

as for the second suggestion I don’t understand where this is coming from, the folder I’m trying to backup is already a local “Backup” folder so there are a lot of different compressed files from different services aswell as the autobackup folder from unifi but only the auto backup folder is directly passthrough from inside the docker container running unifi so in theory Duplicacy should not be able to see this file as there are only auto backup files in unf format and a json autobackup file. the appdata folder is being backed up to this folder aswell but it is compressed so it shouldn’t fail or even see the individual files on that right?

as for the 3rd those are deleted files, there is a recycle schedule in the Minecraft folder it backups 4 times a day and after a X amount of versions is reached it will delete the oldest one. but duplicacy should just “ignore” that right?

the backup job got aborted by the time limit so that was my mistake i removed that now.

this is a docker container instance from Hotio
the docker host is an Dell PowerEdge R720XD Unraid server with 192 gb of ram and 32 threads
the folder is as i mentioned before a local backup folder this folder is synced across a few servers with a Syncthing instance to collect all the backup data in one folder so that duplicacy can backup it externally. the Folder has, according to Syncthing 1344 files spread over 88 folders for a total of 130gb

[saspus]

@danny1

the -vss flag, what does this do ?

@TheBestPessimist

-vss, -vss-timeout <timeout>

The -vss option works on Windows and on MacOS (APFS format only) but not on linux.

It is used to turn on the Volume Shadow Copy (Windows) or AFPS snapshot (MacOS) services such that files opened by other processes with exclusive locks can be read as usual.

The -vss-timeout option changes the grace period in which duplicacy will wait for the system to enable or disable the vss. The minimum is 60 seconds. This option should be used only with -vss.

@danny1

as for the second suggestion I don’t understand where this is coming from, the folder I’m trying to backup is already a local “Backup” folder so there are a lot of different compressed files from different services aswell as the autobackup folder

From this line:

2021-11-16 20:07:19.594 WARN LIST_SKIP Skipped non-regular file appdata/unifi-controller/run/mongodb-27117.sock

This is mongo socket file, definitely not what you describe above.

@danny1

local backup folder this folder is synced across a few servers with a Syncthing instance to collect all the backup data in one folder so that duplicacy can backup it externally. the Folder has, according to Syncthing 1344 files spread over 88 folders for a total of 130gb

It seems -vss won’t work for you, you would need to ensure the backup is atomic in some other way.

Do I understand correctly that you are syncing data to a single folder that is later picked up by duplciacy? Why not backup from each host directly to the destination instead? Unless your local filesystem supports snapshotting it would be very hard to ensure atomicity of your backup snapshots.

Furthemore, duplicacy does not benefit from containerization, it’s already self-contained executable. Running it directly may provide you to access to your host filesystem features to implement snapshotting on your own via pre-backup scripts